automatic pagination?

[ltalirz]

hi @CasperWA
I'm in the process of setting up some automatic monitoring of the optimade servers on materials cloud to make sure that they are up.

While doing so, I realized that the default response to
https://dev-aiida-dev.materialscloud.org/optimade-sample/optimade/structures
is ~2MB in size.

Would it perhaps make sense to adjust the default pagination in such a way that the default response does not exceed 100KB?

[CasperWA]

Sure. This can be set in the config.json file here.
The next variable "db_page_limit" is the total number of results that is allowed by the DB backend (when using the query parameter page_limit=<some-integer-value>.

[CasperWA]

This has been solved by lowering the default page_limit to 15 and introducing a page_limit_max in aiida-optimade as well as "upstream" in optimade-python-tools.