[Datahub] - Add filter options on the download menu

[tomalary]

On each datasets page, the user can access to options to filter download resources and not only per file extension. The filters can be set up (for example per year).

The filter is a drop-down menu with suggestion and autocomplete search on each drop-down menu.

Draft of the component...

For the display of the download links the component check the tag in the metadata. Then we create the download links and they are displayed. If the tag are not found the component work as the usual downloading component.

As our solution is to create the download links by using API Entrepot (API developed by IGN that allow to load/process/share data services), we are open to discuss about an enhancement that suits everyone.

If this feature is not clear please let us know or ask questions to clarify.

---

This work is sponsored by IGN.

[jahow]

Hi, thanks for your interest,

Could you please explain what are the tags you mention that let you apply these filters? are these standard ISO fields? Also, how many download links do you typically expect in order for such filtering to become valuable?

[IGNF-Xavier]

Hi,
API entrepôt will allow the owner of datasets to configure prepackage zip-files of data (PPK) depending on territories, date of products, format of files, etc.
Once done, these PPK can be downloaded at a specific URL thanks to API Telechargement
The user of the component in the dasasets page will be able to choose the PPK needed according to user-defined filters :

• polygon : x1,y1;x2,y2;… ( EPSG:4326),
• editionDateFrom : YYYY-MM-DD,
• editionDateTo : YYYY-MM-DD,
• lang : FR ou EN
• crs : epsg,
• zone : FRA (for France entière) or D075 (for Paris)...
• format : SHP (for Shapefile)...

The swagger of API telechargement is here : https://data.geopf.fr/telechargement/swagger-ui/index.html (today 16/01, it won't work because of maintenance).
The point of access of the capabilities of this download-service will be stored in the metadata of the dataset. If completed, the component appears, if not, the component is not displayed.

When you consider that there are 101 departments in France, maybe the results of this component will need to have pagination.

[jahow]

Thanks for the clarification, it definitely makes more sense! Do we agree that records with this feature would be service metadata, and not datasets?

There has been a discussion for a while about how/if we should show services in the datahub; currently they're filtered out. A component such as the one mentioned here would make sense for other kinds of services (WMS, OGC API etc.), so we could address this use case globally and do a specific implementation for the IGN APIs.

Also, could you please clarify if this work will be funded by IGN?

[IGNF-Xavier]

We (IGN) are currently developing this download component specifically for the datasets pages because there will be plenty of datasets with different years (and eventually format and territories).
For instance, a new edition of BD TOPO is available every 6 months since many years, on each departement with different CRS. So in the medatata of BD TOPO, there will be a link for this download service https://data.geopf.fr/telechargement/resource/BDTOPO?

Maybe we will dadapt this component for this specific PPK-download service page adding the filter dataset.

Then we (IGN) will develop a map-based download component principally used for datasets which have a high weight (for exemple LidarHD). Moreover this type of dataset are available as tiles of 1km to 1km (for LidarHD)

[jahow]

Ok, I see, then this interface would simply show up next to an API link pointing at the IGN Géoplateforme, regardless of the metadata type we're in. I think such a "capabilities explorer" component would be very useful even outside of the IGN use cases, e.g. listing layers in a WMS endpoint.

Would you agree to participate in a preliminary UX work to properly design this component and its integration in the datahub?

[IGNF-Xavier]

I think there are 2 subjects :

• a component n°1 in the dataset page which provides URL of zip-files thanks to a search API telechargement
• a component n°2 in the service page which allow to explore the getcapabilities of a service (WMS for example).

For Geoplateforme (the governance is not only supported by IGN) :

• Component n°1 is described in this issue. A priori for march 2024
• component n°2 is another problem. For the getcapabilities of Geoplateforme on public endpoint, the xml will become bigger and bigger as time goes by. We are under investigation in order to propose :

1. a tool which allow the user/developer to maintain chosen layers in the XML of the getcapabilities and store it
2. a way to record a XML getcap for the organisation during the workflow of API Entrepôt and store it
3. a way to make XML by the governance of Geoplateforme for authorities data in differents thematics with automatic updates if new layers are proposed.
   In fact, all the layers are in the getcapabilities of the service but we provide a way to obtain XML getcap with only layers pertinent or needed by the users

For 2 : the getcapabilities will be available on each dataset page and on the organisation page (we want to suggest the community to have a specific for each organization with a description (exeample : https://www.data.gouv.fr/fr/organizations/institut-national-de-la-statistique-et-des-etudes-economiques-insee/)
For 2 and 3 : the component n°2 will allow to search different XML getcap : organisation, thematic and data authorities.

[IGNF-Xavier]

But to answer : (sorry) yes we agree to participate in a preliminary UX Work. Note that the development teams at IGN for Geoplateforme may be different for each component

[jahow]

@IGNF-Xavier Could you please get in touch with someone at Camptocamp? (your contact information is not public) Thanks!

[IGNF-Xavier]

👍 @fgravin knows @tomalary and me

[jahow]

Meetings note are here (FR): https://semestriel.framapad.org/p/api-telechargement-datahub-a5o8?lang=fr

[mmohadIGN]

This is the workflow we plan to do. Did it seem alright to you? @fgravin and @jahow

---
title: Workflow IGN download API
---
%% doc mermaid ici https://mermaid-js.github.io/mermaid/#/flowchart?id=flowcharts-basic-syntax
flowchart TD
    A("record-apis-component") -->|choose proxy compenent API| B("feature/record/proxy-apis-component")
    B -->|do| C(feature/record/ign-api-dl)
    C -->|begin| D(IGN API client)
    C -->|display| E(ui/ign-dl-form)
    D <-->|look at| C

Loading

[jahow]

Thanks for working on this and giving more details:

• I think there is no need to introduce a "proxy" component; currently the gn-ui-record-api-form is used exclusively but I think it would make sense to only use this component if the protocol is ogcFeatures, and introduce a new component called e.g. gn-ui-record-ign-api-form based on the IGN-specific protocol code; the choice between both can be done in the record-apis-component
• a dedicated ign-dl-form presentation component might not be needed; IIRC the component will mostly be comprised of several dropdown/search fields and a list of results; the smart component (e.g. gn-ui-record-ign-api-form) could hold the logic, interact with the IGN API client and place the relevant presentation components; this would avoid introducing another layer of complexity

Pinging @fgravin because there may be things that were discussed before that I don't know about 🙂

[fgravin]

• I think there is no need to introduce a "proxy" component; currently the gn-ui-record-api-form is used exclusively but I think it would make sense to only use this component if the protocol is ogcFeatures, and introduce a new component called e.g. gn-ui-record-ign-api-form based on the IGN-specific protocol code; the choice between both can be done in the record-apis-component

Where would you do the switch statement (API type => corresponding form) ? in datahub-record-apis ? Was just wondering if it's not more a feature than a datahub thiing.

• a dedicated ign-dl-form presentation component might not be needed; IIRC the component will mostly be comprised of several dropdown/search fields and a list of results; the smart component (e.g. gn-ui-record-ign-api-form) could hold the logic, interact with the IGN API client and place the relevant presentation components; this would avoid introducing another layer of complexity

Yes I told them to start with implementing the feature component, then perhaps we could split it with some UI components or not. Might not be needed though.

[jahow]

Where would you do the switch statement (API type => corresponding form) ? in datahub-record-apis ? Was just wondering if it's not more a feature than a datahub thiing.

Right, good point! then a generic gn-ui-record-ign-api-form in the feature/record lib would make sense, and the switch would be done here.

[jahow]

Hi @mmohadIGN @tomalary, this PR might interfere with your work there? #875

We're going to merge this soon so as to make it part of the next release, let us know if there's anything that can be done to help with potential conflicts.

[mmohadIGN]

Hi @mmohadIGN @tomalary, this PR might interfere with your work there? #875

We're going to merge this soon so as to make it part of the next release, let us know if there's anything that can be done to help with potential conflicts.

Hi,

Thanks, it should be okay for us, but we will tell you if not

[mmohadIGN]

Hello @jahow ,

We’ve completed the development of the first version. This version includes a form to download data from our "API téléchargement".

Before proceeding, I would like to confirm if I should initiate a PR into the main branch. I will also link the relevant issue.

Please let me know if you need further details or if any additional steps are required before the merge.

[jahow]

Hi @mmohadIGN,

Before proceeding, I would like to confirm if I should initiate a PR into the main branch. I will also link the relevant issue.

yes please!

Please let me know if you need further details or if any additional steps are required before the merge.

Since this sounds like a significant issue we will go through a review process. Please make sure that the PR is correctly applied to the latest main branch, thanks!

[mmohadIGN]

Hi @jahow ,

Sorry for the delay.
I create the PR here : create api card and form to request Ign download api #989
But I can't assigne someone.