add mixed precision support to deepxde #1650
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lululxvi
reviewed
Feb 11, 2024
lululxvi
reviewed
Feb 11, 2024
lululxvi
reviewed
Feb 11, 2024
g-w1
force-pushed
the
mixed
branch
4 times, most recently
from
d3590e4
to
1352213
Compare
lululxvi
reviewed
Feb 11, 2024
lululxvi
reviewed
Feb 11, 2024
lululxvi
reviewed
Feb 12, 2024
lululxvi
reviewed
Feb 12, 2024
lululxvi
reviewed
Feb 12, 2024
lululxvi
reviewed
Feb 12, 2024
lululxvi
reviewed
Feb 12, 2024
lululxvi
reviewed
Feb 12, 2024
g-w1
force-pushed
the
mixed
branch
3 times, most recently
from
70d6c15
to
b30dc78
Compare
|
If I use the API here to set mixed precision, then all the demo code can run in mixed precision? |
|
Not all. The L-BFGS optimizer doesn't work in mixed precision. But if you add the line |
lululxvi
reviewed
Jun 26, 2024
| @@ -74,7 +74,7 @@ def set_default_float(value): | |||
| The default floating point type is 'float32'. | |||
|
|
|||
| Args: | |||
| value (String): 'float16', 'float32', or 'float64'. | |||
| value (String): 'float16', 'float32', 'float64', or 'mixed' (mixed precision in https://arxiv.org/abs/2401.16645). | |||
There was a problem hiding this comment.
value (String): 'float16', 'float32', 'float64', or 'mixed' (mixed precision).
lululxvi
reviewed
Jun 26, 2024
| @@ -74,7 +74,7 @@ def set_default_float(value): | |||
| The default floating point type is 'float32'. | |||
There was a problem hiding this comment.
The default floating point type is 'float32'. Mixed precision uses the method in the paper: `J. Hayford, J. Goldman-Wetzler, E. Wang, & L. Lu. Speeding up and reducing memory usage for scientific machine learning via mixed precision. Computer Methods in Applied Mechanics and Engineering, 428, 117093, 2024 <https://doi.org/10.1016/j.cma.2024.117093>`_.
lululxvi
reviewed
Jun 26, 2024
|
|
||
| self.opt.step(closure) | ||
| def closure_mixed(): |
lululxvi
reviewed
Jun 26, 2024
| @@ -85,6 +85,20 @@ def set_default_float(value): | |||
| elif value == "float64": | |||
| print("Set the default float type to float64") | |||
| real.set_float64() | |||
| elif value == "mixed": | |||
| print("Set the float type to mixed precision of float16 and float32") | |||
| real.set_mixed() | |||
There was a problem hiding this comment.
This code is confusing. Here you do real.set_mixed(), but later you do either real.set_float16() or real.set_float32(). It seems you only need a flag mixed. You can do this flag after line 42.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.