-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implementation of an OPTIMADE client #1154
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1154 +/- ##
==========================================
- Coverage 92.41% 91.95% -0.46%
==========================================
Files 68 72 +4
Lines 3914 4265 +351
==========================================
+ Hits 3617 3922 +305
- Misses 297 343 +46
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
ded5573
to
64a9e48
Compare
There is probably enough here now for people to test... not sure if we want to split some of the other features mentioned above into separate PRs. Would be helpful to have at least a minimal implementation of this working before the workshop. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My last changes will break the tests for now, will update them on the train home!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good work writing this in a short time.
Thanks for the ongoing reviews @JPBergsma, I think I have addressed everything and that this is now in reasonable shape for this PR. I have reworked the wishlist of new features above and we can raise some issues to track them, when we are happy with this. I will rebase this PR into 3 commits: implementation, tests and docs. Have you been able to get the client running yourself yet? |
Final to-do(ish): The client now falls back to synchronous mode when a running event loop is detected, which is the case in Jupyter notebooks and in pytest. This is a slightly tricky feature that I would like to work on in a new PR, but it does mean the test coverage for this PR will be slightly lower until we can run the async stuff properly. |
- Added config for tabbed code blocks - Limit all code blocks to 20em before overflow
Just played around with @CasperWA's test setup for the gateway without much luck. The additional trouble is that this client sets up multiple |
I have still have a problem When I execute the line: optimade-get --filter 'elements HAS "Ag"' I get the error message: |
I think I am missing files that are in this PR. So it seems Git did something wrong. I'll try to delete my local branch and to fetch it again. |
Ok, that seems to have worked the command line client is now working. |
8251d4f
to
d581e01
Compare
Just removed the CI tests for assessing coverage for the validator only; I don't think the numbers are correct (the report seems to also include all other coverage tests) and I have never found myself looking at them (cuts our build time by about 20% too). |
I could not see the comments that I made before, so I will now check whether they still apply. |
They should all be listed as "resolved" above, you can double check them |
I could not find back the pending comments for the review, so I could not see whether they still applied with the latest patches. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok everything seems to work now.
I had some issues with this PR and I noticed that the first commit is from 20 hours ago even though this PR was opened 17 days ago. Perhaps this caused some of the issues I experienced, and it is perhaps also the reason the codecov report seems to give a wrong percentage. Perhaps you can try to make a new branch and PR. Hopefully the code coverage is calculated correctly in that case.
Thanks for the review, glad it is working for you now. I will actually reintroduce those codecov tests before merging and see if they fix themselves on master, we can always remove in a separate PR. I will also hold off on releasing v0.18.0 until the weekend; I will write some tutorial notebooks with this development version and make sure everything is working. |
I have just pushed one additional docs section on querying other endpoints. If the tests pass, I rebase this PR and merge, unless you wanted to have one last look @JPBergsma. |
docs/getting_started/client.md
Outdated
@@ -28,11 +28,20 @@ By default, the client will query all OPTIMADE API URLs that it can find via the | |||
results = client.get() | |||
``` | |||
|
|||
At the command line, it may be immediately useful to redirect or save these results to a file (see [Accessing the results](#accessing-the-results): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you can leave out "immediately" I do not see what it adds to the sentence
Maybee as a more general note, You could mention that not all the databases have implemented all the feature or strictly adhere to the standard. When people are going to use this client in it's current form they will see quite some error messages popping up, so it may be good to mention it is not their fault. |
I have had a look and given my comments above. |
This is a very good point, I will put that at the top of the client guide. |
- Bump httpx & pytest-httpx - Add --pretty-print flag and disable by default for performance - Add a simple async CLI test in CI - Documentation tweaks - Added docs section about endpoints - Setting `max_results_per_provider` to -1 or 0 will remove the limit - Allow `extensions/<example>` endpoints to be queried - Final code review changes - Addded feature list to top of docs Co-authored-by: Johan Bergsma <29785380+JPBergsma@users.noreply.github.com>
An implementation of a client that pulls all known databases from the providers (or uses a defined base URL) and asynchronously queries them with a nice rich interface.
Still to-do for this PR:
Remove non-async version and replace with throttled async (i.e. one worker)Refactored such that keeping the non-async version is not a huge burden, which makes things easier to test in Jupyter etc.Future design to-do: