Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check for duplicate keys in Java properties files, fix CodeMeta displayName #9176

Merged
merged 16 commits into from
Aug 5, 2024

Conversation

poikilotherm
Copy link
Contributor

What this PR does / why we need it:
Fix duplicated keys in all *.properties. Add Github Action to check for no re-introduction of dups.

Which issue(s) this PR closes:

Closes #9169

Special notes for your reviewer:
Feel free to push to this branch for fixing all of the dups in the same go. I don't mind!

Suggestions on how to test this:
Look at the PRs Action log

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Nope

Is there a release notes update needed for this change?:
Nope

Additional documentation:
None

As creating the properties files for metadata blocks is a
tedious manual process, this script ensures in CI everything
is present. It adds to checking for duplicates.
- Easier to run and debug as separate files
- Can be used locally, too
- Use GraalVM to compile native binary for accents removal with same Java code as used in application (also uses JBang as build system)
- "Just" using JBang is not fast enough, JVM startup times are making it sluggish!
@poikilotherm
Copy link
Contributor Author

This PR now contains a test commit (33731ad), which needs to be reverted before merging.

Here is the output of the verification job:
image

Here is the output of the duplication check job:
image

That one also leaves notes in the changes tab:
image

@poikilotherm
Copy link
Contributor Author

As you can see, it already detected a missing property in CodeMeta and a typo - fixed with 1aeb665.

@poikilotherm poikilotherm added this to the 6.3 milestone Jul 2, 2024
@poikilotherm poikilotherm added Feature: Metadata Feature: Performance & Stability NIH OTA: 1.3.1 3 | 1.3.1 | Support software metadata | 5 prdOwnThis is an item synched from the product planning... labels Jul 2, 2024
@poikilotherm poikilotherm added the Size: 0.5 A percentage of a sprint. 0.35 hours label Jul 2, 2024
@poikilotherm poikilotherm marked this pull request as ready for review July 2, 2024 15:22
@pdurbin pdurbin removed this from the 6.3 milestone Jul 3, 2024
@pdurbin pdurbin changed the title 9169 fix duplicate keys in Java properties files check for duplicate keys in Java properties files Jul 9, 2024
@cmbz cmbz added the GREI 6 Connect Digital Objects label Jul 9, 2024
@cmbz cmbz added this to the 6.4 milestone Jul 18, 2024
@pdurbin pdurbin changed the title check for duplicate keys in Java properties files check for duplicate keys in Java properties files, fix CodeMeta displayName Jul 25, 2024
@pdurbin pdurbin self-assigned this Aug 1, 2024
Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't run this on my Mac, but that's ok. I ran the check_duplicate_properties.sh script on Linux and it worked fine.

I didn't install jbang and Graal to run the other script. We only need these to work in CI.

Approved.

FAIL=1

echo "::group::$FILE"
for KEY in $(echo "$FILTER" | cut -d" " -f3); do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Mac I get cut: stdin: Illegal byte sequence when it reaches lines like datasetfieldtype.titulo.title=T?tulo

There's a long discussion at https://stackoverflow.com/questions/19242275/re-error-illegal-byte-sequence-on-mac-os-x

I dunno, I'm ok with this only working on Linux and in our CI.

@@ -1,5 +1,6 @@
metadatablock.name=codeMeta20
metadatablock.displayName=Software Metadata (CodeMeta 2.0)
metadatablock.displayName=Software Metadata (CodeMeta v2.0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good fix, making it the same as what's in scripts/api/data/metadatablocks/codemeta.tsv

@pdurbin pdurbin removed their assignment Aug 1, 2024
@stevenwinship stevenwinship self-assigned this Aug 5, 2024
@stevenwinship stevenwinship merged commit c5a6a8f into IQSS:develop Aug 5, 2024
4 checks passed
@stevenwinship stevenwinship removed their assignment Aug 5, 2024
@poikilotherm poikilotherm deleted the 9169-fix-dup-props branch August 6, 2024 05:27
@pdurbin
Copy link
Member

pdurbin commented Sep 6, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature: Metadata Feature: Performance & Stability GREI 6 Connect Digital Objects NIH OTA: 1.3.1 3 | 1.3.1 | Support software metadata | 5 prdOwnThis is an item synched from the product planning... Size: 0.5 A percentage of a sprint. 0.35 hours
Projects
Status: Done 🧹
Status: Small & Fast
Development

Successfully merging this pull request may close these issues.

Duplicate property key in Bundle.properties file V5.12.1
5 participants