Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seemingly random MacOS failure in test_cstr #1473

Open
adam-a-a opened this issue Aug 1, 2024 · 13 comments
Open

Seemingly random MacOS failure in test_cstr #1473

adam-a-a opened this issue Aug 1, 2024 · 13 comments
Labels
Priority:Normal Normal Priority Issue or PR

Comments

@adam-a-a
Copy link
Contributor

adam-a-a commented Aug 1, 2024

I have seen the following glitch show up in MacOS checks which I can resolve by just rerunning the check. Just seeing this for the second time on the same file (test_cstr) so reporting it in case it pops up again.

image

@ksbeattie ksbeattie added the Priority:Normal Normal Priority Issue or PR label Aug 1, 2024
@lbianchi-lbl
Copy link
Contributor

lbianchi-lbl commented Aug 1, 2024

Zooming out a bit, we should consider dropping support for (i.e. stop testing in CI) the IDAES-less macOS environment. It's a non-negligible maintenance burden and we might decide it's not worth it.

See also: #1306

@adam-a-a
Copy link
Contributor Author

adam-a-a commented Aug 1, 2024

Noticed this failure on #1471

@lbianchi-lbl
Copy link
Contributor

Zooming out a bit, we should consider dropping support for (i.e. stop testing in CI) the IDAES-less macOS environment. It's a non-negligible maintenance burden and we might decide it's not worth it.

See also: #1306

This (i.e. replacing our "bootleg" macOS CI environment with Apple Silicon GHA runners using IDAES solvers) is being done in #1483.

@lbianchi-lbl
Copy link
Contributor

  • As @adam-a-a points out, this could be an issue with a change in IDAES between version 2.5.0 and the upcoming 2.6.0
  • The motivation for this hypothesis is that the changes in Enable RO0D to be dynamic #1471 are not related to the failing check, but Enable RO0D to be dynamic #1471 does update the idaes-pse requirement to be IDAES/idaes-pse:main
  • We expect to be able to verify this by creating a PR that contains only the change to the idaes-pse requirement, which is planned as soon as idaes-pse 2.6.0rc0 is available

@lbianchi-lbl
Copy link
Contributor

lbianchi-lbl commented Sep 7, 2024

The macOS CI check is indeed failing in the same way in #1486, which would confirm @adam-a-a's hypothesis/observation that the failure is possibly related to a change in IDAES between versions 2.5.0 and 2.6.0rc0: https://github.com/watertap-org/watertap/actions/runs/10747258986/job/29809470692

image

@adam-a-a
Copy link
Contributor Author

@lbianchi-lbl this same test exists on IDAES for the same unit. The only difference is in the WaterTAP version of this unit, we add a variable and simple constraint. That is, we add hydraulic retention time as a variable and the associated constraint for it, relating retention time to reactor volume and inlet flowrate.

I don't know if this cstr test may have had a similar macOS test failure on the IDAES repo.

@lbianchi-lbl
Copy link
Contributor

lbianchi-lbl commented Sep 11, 2024

@lbianchi-lbl this same test exists on IDAES for the same unit. The only difference is in the WaterTAP version of this unit, we add a variable and simple constraint. That is, we add hydraulic retention time as a variable and the associated constraint for it, relating retention time to reactor volume and inlet flowrate.

I don't know if this cstr test may have had a similar macOS test failure on the IDAES repo.

Thanks for the extra info @adam-a-a. On the IDAES side, I don't believe we have any CI job running without the IDAES solver, so it's possible (but agani, not proven either way) that the IDAES/idaes-pse counterpart of this test would fail in the same way if it was run in the same environment (i.e. non-IDAES Ipopt, etc).

@lbianchi-lbl lbianchi-lbl changed the title Seemingly random MacOS check fail Seemingly random MacOS failure in test_cstr Sep 11, 2024
lbianchi-lbl added a commit that referenced this issue Sep 11, 2024
* Update IDAES requirement for 2.6 release

* Update min Python version to 3.9 b/c Pint

* Try removing version constraint for NumPy with Conda

* Add requires_idaes_solver marker for CSTR test affected by #1473
@andrewlee94
Copy link
Collaborator

andrewlee94 commented Sep 12, 2024

@adam-a-a Would you be able to run the diagnostics tests (both structural and numeric) and paste the output here? This would help us try to work out what the root cause might be.

Edit: It should not matter which platform you run this on, as the underlying issues will likely be the same.

@adam-a-a
Copy link
Contributor Author

adam-a-a commented Sep 12, 2024 via email

@andrewlee94
Copy link
Collaborator

@adam-a-a Running it on your local machine should be good enough. If there is a modeling issue, it will likely show up no matter what platform you run on; structural issues depend only on the model, and even numerical issues will likely be apparent in the before and after solve runs unless you are really unlucky.

@andrewlee94
Copy link
Collaborator

@adam-a-a Also, I note you said this could be fixed by re-running the test - is that correct? If so, we are dealing with something more complicated (and something that I am not sure how to deal with).

@adam-a-a
Copy link
Contributor Author

@adam-a-a Also, I note you said this could be fixed by re-running the test - is that correct? If so, we are dealing with something more complicated (and something that I am not sure how to deal with).

@andrewlee94 I am not certain that this is the case at the moment but can check again.

@adam-a-a
Copy link
Contributor Author

Reminder: we can check if flow_vol_phase in the denominator of the HRT constraint for the WaterTAP CSTR unit is causing an issue. Currently, one of the initialization tests for the WaterTAP CSTR (which adds HRT to the model) is currently failing on Mac checks, but more specifically, when pointing to the latest version of IDAES (v2.6?).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority:Normal Normal Priority Issue or PR
Projects
None yet
Development

No branches or pull requests

4 participants