Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 3.0.0 #255

Closed
7 tasks done
joaander opened this issue Jun 16, 2023 · 5 comments
Closed
7 tasks done

Release 3.0.0 #255

joaander opened this issue Jun 16, 2023 · 5 comments
Assignees

Comments

@joaander
Copy link
Member

joaander commented Jun 16, 2023

Release checklist:

  • Run bumpversion.
  • Review the change log.
  • Check for new or duplicate contributors since the last release:
    comm -13 <(git log LAST_TAG --format="%aN <%aE>" | sort | uniq) <(git log --format="%aN <%aE>" | sort | uniq).
    Add entries to .mailmap to remove duplicates.
  • Check readthedocs build, especially change log formatting.
  • Tag and push.
  • Update conda-forge recipe.
  • Update glotzerlab-software.
@joaander joaander self-assigned this Jun 16, 2023
@IAlibay
Copy link

IAlibay commented Jun 16, 2023

@joaander - we (MDAnalysis) started picking up the 3.0 release on our CI and we're coincidentally seeing a lot of weird coverage failures: https://dev.azure.com/mdanalysis/mdanalysis/_build/results?buildId=5846&view=logs&j=7a9c7d2c-346e-591b-84bb-5490fd358201&t=ac690a76-5e9e-5c44-1f80-b71727af3e77&l=123

By any chance would you have any ideas if this is expected / related to changes in this release?

edit: Looks like the introduction of signal in #237 breaks pytest-xdist and/or pytest-cov.

@joaander
Copy link
Member Author

This error:

ValueError('signal only works in main thread of the main interpreter')
may be the result of this code:

gsd/gsd/__init__.py

Lines 19 to 21 in 16d44e1

# Install a SIGTERM handler that gracefully exits, allowing open files to flush
# buffered writes and close.
signal.signal(signal.SIGTERM, lambda n, f: sys.exit(1))

The signal is not strictly necessary. I added it as GSD 3.0 aggressively buffers writes in memory to improve performance. Users would expect to find that data in the file when their scripts are asked to terminate.

@IAlibay
Copy link

IAlibay commented Jun 16, 2023

The signal is not strictly necessary. I added it as GSD 3.0 aggressively buffers writes in memory to improve performance. Users would expect to find that data in the file when their scripts are asked to terminate.

Thanks @joaander

Seems like this is killing pytest-cov which is, at least in my opinion, a pretty big deal downstream. I can't see any coverage calls in your CI, how are you getting around this issue? I don't want to force a decision now, especially since it's a Friday evening, but it might be good to get an official view from GSD on if this is a likely feature - should I raise a separate issue here?

A side thought, given that it's being throw on what appears to be some kind of multithreaded call - how well tested is the current release with multiprocessing approaches?

@joaander
Copy link
Member Author

I haven't experience any issues in my testing in production environments. I have some unit tests that use gsd in combination with multiprocessing to read frames in parallel. Nothing in GSD is engineered for thread or process safe read/write access to the same file.

Given the nature of your error, I assume it is related to something specific in how xdist operates and not how typical users will use the software. Feel free submit a PR that works around the issue.

@joaander
Copy link
Member Author

I opened #257 to discuss the unexpected behavior which is not related to the steps needed to release 3.0.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants