Georchestra gn4.4.5 rebase #297

f-necas · 2024-06-19T15:20:25Z

geOrchestra/geonetwork checklist

PR only involves cherry-picked commits from upstream.
PR contains custom code which will soon be available in an upstream release and can be overriden => mention core-geonetwork version if possible.
PR contains custom geOrchestra code, which need to be verified during future migrations.
- I have properly filled the migration-helper-changelog.md file.

… dynamic privileges are not set

…en the metadata has 1 WMS layer (#7646) * Metadata editor / Fix javascript error in the add thumbnail option when the metadata has 1 WMS layer: Error: [orderBy:notarray] http://errors.angularjs.org/1.8.2/orderBy/notarray\?p0\=%7B%22resource%22%3A%7B%22ref%22%3A%22172%22%2C%22refParent%22%3A%22171%22%2C%22name%22%3A%222%22%2C%22url%22%3A%22https%3A%2F%2Fmaps-cartes.ec.gc.ca%2Farcgis%2Fservices%2FDMS%2FStressCumulatifLittorales%2FMapServer%2FWMSServer%3Frequest%3DGetCapabilities%26service%3DWMS%22%2C%22title%22%3A%22Metadata%20WMS%20layer%22%2C%22abstract%22%3A%22The%20ISO19115%20metadata%20standard%20is%20the%20preferred%20metadata%20standard%20to%20use.%20If%20unsure%20what%20templates%20to%20start%20with%2C%20use%20this%20one.%22%2C%22protocol%22%3A%22OGC%3AWMS%22%7D%7D at lib.js?v=65b94734a63ca8491dba95d7439baf1be02c2450:21:168 * Metadata editor / Improve layer configuration init. (#96) * Metadata editor / Improve layer configuration init. Related to geonetwork/core-geonetwork#7646. * Metadata editor / Improve layer configuration init. * Metadata editor / Remove log. --------- Co-authored-by: François Prunayre <fx.prunayre@gmail.com>

Follow up of geonetwork/core-geonetwork#7591 (comment) Fix form content when adding * a resource maintenance and then User defined maintenance frequency * or temporal resolution in ISO19115-3

…eters. By default injectable parameters are excluded from request parameters. Map is one of those object however it does occur where map objects are supplied as parameters and they should be added to open api spec. There are currently no cases where a map is injected on purpose into the request parameters. This will fix issues with missing request parameters documentation which are based on Map objects.

This fixes bug where some apis will not execute correctly from the swagger pager due to the wrong enum value being supplied. i.e. visibility should be using enum values PUBLIC/PRIVATE instead of public/private in formatters/zip api, the format should be SIMPLE/PARTIAL/FULL instead of simple/partial/full

…sources being returned. Update attachment api "Get a metadata resource" should indicate that gets a file resource Also "Create a new resource" should identify that it consumes any resources Update keywords api "Download a thesaurus by name" should indicate that gets a file resource

…ceLib / Sonarlint improvements (#7692) * Remove empty class SourcesLib and deprecated/unused methods in ResourceLib * Sonalint improvements

Updated-dependencies: - dependency-name: commons-fileupload:commons-fileupload dependency-type: direct:production --- In GN it is only used in SiteApi class in the services module. Other occurrences are introduced by openrdf-utils dependency. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Juan Luis Rodríguez <juanluisrp@gmail.com>

…nd configuring-search-fields (#7696) * Addressing docs translation glitch #7687- fixes adding-static-pages and configuring-search-fields * Addressing docs translation glitch #7687- fixes creating-custom-editor

* Add a role and feature matrix to the GeoNetwork documentation * Removed to entries that are no longer available * Update docs/manual/docs/administrator-guide/managing-users-and-groups/index.md --------- Co-authored-by: Jose García <josegar74@gmail.com>

* Remove non required data-translate directive usage in gnMetadataGroupUpdater directive * Enable preemptive in CSW http requests with credentials. Fixes #5492 Co-authored-by: josegar74 <josegar74@gmail.com>

Bumps [stCarolas/setup-maven](https://github.com/stcarolas/setup-maven) from 4 to 5. - [Release notes](https://github.com/stcarolas/setup-maven/releases) - [Commits](stCarolas/setup-maven@v4...v5) --- updated-dependencies: - dependency-name: stCarolas/setup-maven dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump org.json:json from 20140107 to 20240205 Bumps [org.json:json](https://github.com/douglascrockford/JSON-java) from 20140107 to 20240205. - [Release notes](https://github.com/douglascrockford/JSON-java/releases) - [Changelog](https://github.com/stleary/JSON-java/blob/master/docs/RELEASES.md) - [Commits](https://github.com/douglascrockford/JSON-java/commits) --- updated-dependencies: - dependency-name: org.json:json dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Add org.json:json to dependency management Remove the version from common/pom.xml and add the dependency to dependencyManagement section of root pom.xml. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Juan Luis Rodríguez <juanluisrp@gmail.com>

Removed unused dependency on jslint-maven-plugin from web-ui.

* Vertical extent label modification * fetching unit from index - default frontend fallback - example default value in -3 indexer

* Update to Elasticsearch 8. Use of Elasticsearch Java API Client instead of Java High Level REST Client * Update to Elasticsearch 8 / WFS indexing draft. (#88) * Update to Elasticsearch 8 / remove TODOs * Update Elasticsearch client to version 8.11.3 * Elasticsearch / Update maven plugin. * Associated record / Store all relations in index / Remove experimental feature, not used. Related to #4912 * Elasticsearch / Update maven plugin configuration. Avoid error like ERROR: Elasticsearch exited unexpectedly, with exit code 143 * Elasticsearch / Update MetadataUtils.getAssociated to retrieve scripted overview field * Elasticsearch / Update MetadataUtils.getAssociated remove TODO comment * Elasticsearch / Fix and refactor index readonly health check * Elasticsearch / Log query error details * Elasticsearch / Sonarlint improvements * Elasticsearch / WrapperQuery use base64 encoded JSON string query. * Elasticsearch / Remove unused commented code from EsSearchManager * Elasticsearch / More strict Xlink query based on UUID and fix check on hits. A request may return no hits but can be used to check number of hits. In such case we should avoid using hits.hits.size and use hits.total.value to get number of match. * Elasticsearch / Health check / Fix number of hits info. * Elasticsearch / Cleaning / No need to retrieve hits to only get matches. * Elasticsearch / Deprecated field [include] used, expected [includes] instead. * Elasticsearch / Remove 'Clear XLink cache' from Administration > Tools, clear the Xlink cache automatically before indexing and remove non-implemented code to retrieve metadata with XLink (not required anymore) * Kibana / Update install instruction Related to elastic/kibana#82521. * Elasticsearch / Remove unused imports * Kibana / Update default dashboards. * Elasticsearch / Documentation / Update Elasticsearch version * Elasticsearch / Fix logger module name typo --------- Co-authored-by: François Prunayre <fx.prunayre@gmail.com>

…adata with the option 'Generate UUID'

Updated summary and description to differentiate these two end points.

* Documentation / Elasticsearch query endpoint - query samples * Add step by step instructions for running eamples * Update docs/manual/docs/api/search.md Co-authored-by: François Prunayre <fx.prunayre@gmail.com> * Update docs/manual/docs/api/search.md Co-authored-by: François Prunayre <fx.prunayre@gmail.com> * Update docs/manual/docs/api/search.md Co-authored-by: François Prunayre <fx.prunayre@gmail.com> --------- Co-authored-by: Jody Garnett <jody.garnett@gmail.com> Co-authored-by: François Prunayre <fx.prunayre@gmail.com>

…figuration (#7702) * Standard / ISO19115-3 / Editor configuration / Add support for custom label on date field Add support to label override for `cit:date` element: eg. ```xml <field xpath="/mdb:MD_Metadata/mdb:identificationInfo/*/mri:citation/*/cit:date" or="date" in="/mdb:MD_Metadata/mdb:identificationInfo/*/mri:citation/*" name="Date de la donnée"/> ``` * Standard / ISO19115-3 / Handle setting to hide time in calendar Was only supported in ISO19139

…t (#7700) * Standard / ISO19115-3 / Improve french translation for temporal extent * Update codelists.xml

Co-authored-by: Jose García <josegar74@gmail.com>

Make sure it is clear to report a vulnerability directly through Github.

This can facilitate customization of the style of the editor. Previously only the tab identifier was provided making more difficult to define global editor styles common for all tabs and was not providing simple way to customize the side panel.

A number of DCAT related vocabularies are published using OWL format describing a concept schema and concepts. eg. https://mobilitydcat-ap.github.io/controlled-vocabularies/network-coverage/latest/index.html ```xml <owl:NamedIndividual rdf:about="https://w3id.org/mobilitydcat-ap/mobility-theme/accesibility-information-for-vehicles"> <rdf:type rdf:resource="http://www.w3.org/2004/02/skos/core#Concept"/> <skos:broader rdf:resource="https://w3id.org/mobilitydcat-ap/mobility-theme/data-content-sub-category"/> <skos:broader rdf:resource="https://w3id.org/mobilitydcat-ap/mobility-theme/public-transport-non-scheduled-transport"/> <skos:inScheme rdf:resource="https://w3id.org/mobilitydcat-ap/mobility-theme"/> <skos:prefLabel xml:lang="en">Accesibility information for vehicles</skos:prefLabel> </owl:NamedIndividual> ``` Add the possibility to import those files directly by converting them from OWL to SKOS format supported by GeoNetwork. API changes: * when using the `stylesheet` parameter of the upload thesaurus operation, restrict the use to known file in `xslt/services/thesaurus/` folder. Also some Sonar lint items. Funded by Wallonia region (SPW)

…in the same group (#7679) * Create a metadata / Add dynamic and download privileges to the users in the same group * Update core/src/main/java/org/fao/geonet/kernel/datamanager/base/BaseMetadataOperations.java Co-authored-by: Ian <ianwallen@hotmail.com> --------- Co-authored-by: Ian <ianwallen@hotmail.com>

landryb · 2024-06-20T04:29:16Z

building this branch with the following steps on debian 12/java 17

georchestra $make war-build-gn
georchestra $mvn clean install -pl geonetwork/web
georchestra $mvn package deb:package -pl geonetwork/web -PdebianPackage

i get a proper debian package for testing:

[WARNING] dpkg-deb: building package 'georchestra-geonetwork' in ...
'geonetwork/web/target/georchestra-geonetwork_4.4.5-georchestra.20240620042515~48e91aa-1_all.deb'.

landryb · 2024-06-20T04:41:43Z

db upgrade on an empty 4.2.8 from current master blows hard:

2024-06-20T04:37:00,986 WARN  [geonetwork.databasemigration] -       No migration task found between webapp and database version.
      The system may be unstable or may failed to start if you try to run 
      the current GeoNetwork 4.4.5-georchestra with an older database (ie. 4.2.8
      ). Try to run the migration task manually on the current database
      before starting the application or start with a new empty database.
      Sample SQL scripts for migration could be found in WEB-INF/sql/migrate folder.

2024-06-20T04:37:08,381 WARN  [geonetwork.database] - SQL failure for:  INSERT INTO Settings (name, value, datatype, position, internal) VALUES ('region/getmap/useGeodesicExtents', 'false', 2, 9591, 'n'), error is:ERROR: duplicate key value violates unique constraint "settings_pkey"
  Detail: Key (name)=(region/getmap/useGeodesicExtents) already exists.

havent looked deeper in the other db upgrade bits.

i'll do a new branch in the geonetwork_minimal_datadir repo with the latest schema versions

edit that's https://github.com/georchestra/geonetwork_minimal_datadir/tree/gn4.4.5

landryb · 2024-06-20T04:44:58Z

in https://georchestra.example.org/geonetwork/srv/eng/admin.console#/dashboard/information it says 4.4.1 for Application version and git.closest.tag.name, rather unexpected :) swagger says the same on doc/api

f-necas · 2024-06-20T06:55:22Z

db upgrade on an empty 4.2.8 from current master blows hard:

I think it's because branch for 4.2 and 4.4 have diverged when 4.2.6 was out. but migrations scripts should have approximately the same things inside.

landryb · 2024-06-20T07:09:06Z

i have a bunch of this in the logs when visiting the ui, but it shows apparently fine:

2024-06-20T07:02:46,003 ERROR [geonetwork] - Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: com/sun/media/imageio/stream/RawImageInputStream

i suppose it's due to missing imageio jar somewhere or because java17 is used to run tomcat. It wasn't happening on the same environment with gn 4.2.8 so maybe a missing jar in the war ?

the caller is at org.fao.geonet.api.users.UsersApi.getUserIdenticon(UsersApi.java:266) ~[gn-services-4.4.5-georchestra.jar:?] so maybe something new in 4.4

landryb · 2024-06-20T07:34:01Z

i've had a look at the db content and system/platform/version is 4.4.1 so i guess the first run of the sql migration script failed halfway.

edit: that's probably geonetwork/core-geonetwork#7656

edit2: the commit adding the new region/getmap/useGeodesicExtents settings key is already in georchestra-gn4.2.x branch since 10e515672 so from my understanding hibernate already added the key, and the sql migration script in https://github.com/geonetwork/core-geonetwork/blob/main/web/src/main/webapp/WEB-INF/classes/setup/sql/migrate/v442/migrate-default.sql#L4 also tries to add it. so to work around that i'd remove the line from the SQL script.

landryb · 2024-06-20T09:07:43Z

datahub works/can display metadata & organizations coming from 4.4.5
ogc-api-records microservice from 4.2.8 still works (and there's not much in latest upstream releases from geonetwork-microservices, mostly ES8 updates)
xlinks for contacts work once you retoggle system/xlinkResolver/enable to true
editing/saving a new iso19139 works

…ntation

@f-necas

discussed with @f-necas, cf geonetwork/core-geonetwork#7656

f-necas · 2024-06-20T13:48:58Z

Ready for geOrchestra 24.1.x

landryb · 2024-06-27T14:58:54Z

as said in georchestra/georchestra#4273 merging this PR to a branch named 4.2.x doesn't really make sense :) it should rather be the new default branch of the repo, named georchestra-gn4.4.x to align with others ?

landryb · 2024-06-27T14:59:36Z

maybe it should also get the 5 commits that landed to 4.2.x cf georchestra-gn4.4.5-rebase...georchestra-gn4.2.x

f-necas · 2024-06-27T15:03:27Z

Agree ! I've updated the destination branch, and the 5 commits will be inside normally.

landryb · 2024-06-27T15:10:12Z

can't harm anyone to merge since the target branch is not used yet by anything...

josegar74 and others added 30 commits February 1, 2024 09:41

Add configuration to filter out Elasticsearch fields when download or…

3a01af3

… dynamic privileges are not set

Editor / Fix add element attribute (#7683)

360b5b2

Follow up of geonetwork/core-geonetwork#7591 (comment) Fix form content when adding * a resource maintenance and then User defined maintenance frequency * or temporal resolution in ISO19115-3

Remove empty class SourcesLib and deprecated/unused methods in Resour…

5123486

…ceLib / Sonarlint improvements (#7692) * Remove empty class SourcesLib and deprecated/unused methods in ResourceLib * Sonalint improvements

Automatic formatting

7c5e656

Addressing docs translation glitch #7687- fixes adding-static-pages a…

9cefa22

…nd configuring-search-fields (#7696) * Addressing docs translation glitch #7687- fixes adding-static-pages and configuring-search-fields * Addressing docs translation glitch #7687- fixes creating-custom-editor

Enable preemptive for csw requests with credentials (#5497) (#7706)

029ccd3

* Remove non required data-translate directive usage in gnMetadataGroupUpdater directive * Enable preemptive in CSW http requests with credentials. Fixes #5492 Co-authored-by: josegar74 <josegar74@gmail.com>

using gn- icon definitions for all types now

590211c

Remove unused jslint-maven-plugin (#7725)

090cbe4

Removed unused dependency on jslint-maven-plugin from web-ui.

Vertical extent label modification (#7604)

da9623f

* Vertical extent label modification * fetching unit from index - default frontend fallback - example default value in -3 indexer

Use the generated metadata UUID for resource links when importing met…

5afc515

…adata with the option 'Generate UUID'

Map viewer / Remove Stamen background layers - no longer available

176570e

Separate docs for _search and _msearch

d8fcfac

Updated summary and description to differentiate these two end points.

remove mention of q query parameter

0a14763

Standard / ISO19115-3 / Improve french translation for temporal exten…

7dd47a9

…t (#7700) * Standard / ISO19115-3 / Improve french translation for temporal extent * Update codelists.xml

Index / Add danish language. (#7697)

dca00bc

Co-authored-by: Jose García <josegar74@gmail.com>

Update SECURITY.md

e75ea4e

Make sure it is clear to report a vulnerability directly through Github.

josegar74 and others added 5 commits June 14, 2024 08:25

Documentation / Release notes 4.2.10 (#8170)

d889e68

Update version to 4.4.5-0

d9f211c

Merge branch '4.4.5-upstream' into georchestra-gn4.4.5-rebase

b1ba94d

feat: set to 4.4.5

1dca5f5

feat: remove duplicate

48e91aa

landryb added a commit to georchestra/geonetwork_minimal_datadir that referenced this pull request Jun 20, 2024

initial commit for GN version 4.4.5 (cf georchestra/geonetwork#297)

8bb494a

Update files with Prettier formatting

092e109

f-necas and others added 4 commits June 20, 2024 11:59

feat: update db config, remove deprecated ogc features custom impleme…

e7a16da

…ntation

feat: remove duplicate in cat controller and migr dev guide

514d90f

drop duplicate addition of region/getmap/useGeodesicExtents settings key

064cccd

discussed with @f-necas, cf geonetwork/core-geonetwork#7656

feat: disable xlink by default

f98ba76

f-necas force-pushed the georchestra-gn4.4.5-rebase branch from 4e80a4e to f98ba76 Compare June 20, 2024 13:14

f-necas added this to the 24.1.0 milestone Jun 26, 2024

f-necas marked this pull request as ready for review June 26, 2024 07:50

landryb mentioned this pull request Jun 27, 2024

switch to GN 4.4.5 georchestra/georchestra#4273

Merged

f-necas changed the base branch from georchestra-gn4.2.x to georchestra-gn4.4.x June 27, 2024 15:02

landryb merged commit 6cf862a into georchestra-gn4.4.x Jun 27, 2024
1 check passed

f-necas deleted the georchestra-gn4.4.5-rebase branch June 27, 2024 15:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Georchestra gn4.4.5 rebase #297

Georchestra gn4.4.5 rebase #297

f-necas commented Jun 19, 2024 •

edited

Loading

landryb commented Jun 20, 2024

landryb commented Jun 20, 2024 •

edited

Loading

landryb commented Jun 20, 2024

f-necas commented Jun 20, 2024

landryb commented Jun 20, 2024 •

edited

Loading

landryb commented Jun 20, 2024 •

edited

Loading

landryb commented Jun 20, 2024

f-necas commented Jun 20, 2024

landryb commented Jun 27, 2024

landryb commented Jun 27, 2024

f-necas commented Jun 27, 2024

landryb commented Jun 27, 2024

Georchestra gn4.4.5 rebase #297

Georchestra gn4.4.5 rebase #297

Conversation

f-necas commented Jun 19, 2024 • edited Loading

landryb commented Jun 20, 2024

landryb commented Jun 20, 2024 • edited Loading

landryb commented Jun 20, 2024

f-necas commented Jun 20, 2024

landryb commented Jun 20, 2024 • edited Loading

landryb commented Jun 20, 2024 • edited Loading

landryb commented Jun 20, 2024

f-necas commented Jun 20, 2024

landryb commented Jun 27, 2024

landryb commented Jun 27, 2024

f-necas commented Jun 27, 2024

landryb commented Jun 27, 2024

f-necas commented Jun 19, 2024 •

edited

Loading

landryb commented Jun 20, 2024 •

edited

Loading

landryb commented Jun 20, 2024 •

edited

Loading

landryb commented Jun 20, 2024 •

edited

Loading