Skip to content

Releases: OP-TED/ted-rdf-conversion-pipeline

2.0.0-rc.4

14 Aug 12:38
01673c0
Compare
Choose a tag to compare

What's Changed

  • RML,Saxon and Limes libraries are included in the source code in a dedicated folder
  • Changed Dockerfile for Airflow
  • Included filters for security scan

Full Changelog: 2.0.0-rc.3...2.0.0-rc.4

2.0.0-rc.3

31 Jul 15:10
f6feb9f
Compare
Choose a tag to compare

What's Changed

  • changed Dockerfile for airflow, fuseki and digest-api.
  • changed requirements.txt
  • added dependencies version in airflow image

Full Changelog: 2.0.0-rc.2...2.0.0-rc.3

2.0.0-rc.2

28 May 14:43
a857375
Compare
Choose a tag to compare

What's Changed

Full Changelog: 2.0.0-rc.1...2.0.0-rc.2

2.0.0-rc.1

07 May 11:27
a5a8973
Compare
Choose a tag to compare

Overview

This release focuses on enhancing the TED-SWS pipeline to support the conversion of eForms notices alongside standard Forms. The updates entail adjustments across various components to accommodate eForms notices.

Key Features

  • Notice Fetcher: This component is now compatible with the latest version of the TED API (v3)
  • Metadata Processor: Enhancements have been made to extract and normalise metadata specifically tailored for eForms notices.
  • Eligibility Tester: The Eligibility Tester component has been updated to accommodate both eForms and standard Forms mapping suites, ensuring eligibility validation for both types of forms.
  • Mapping Suite Processor: This component has been updated to effectively read eForm mapping suites, facilitating integration into the TED-SWS pipeline.
  • Mapping Suite Loader: The Mapping Suite Loader has been enhanced to load both eForms and standard Forms mapping suites into the database.
  • Library Upgrades: RML has been upgraded to version 6.2.2, ensuring compatibility and leveraging the latest features for mapping transformations.

What's Changed

New Contributors

Full Changelog: 1.2.0-rc.4...2.0.0-rc.1

1.2.0-rc.2

02 Jul 17:19
1289e20
Compare
Choose a tag to compare

What's Changed

New Contributors

Full Changelog: 1.2.0-rc.1...1.2.0-rc.2

1.2.0-rc.1

07 Jun 13:04
5a9e4d0
Compare
Choose a tag to compare

Main Improvements

  • Improved TEDSWS operator experience when fetching notices in Airflow, displaying a passing step even when no notices are present.
  • Enable the RE-publishing of a Notice in Cellar, facilitating smoother management of published content.
  • Ability for TED-SWS operators to import mapping suite packages tagged with semantic versions, allowing for more precise control over package versions.
  • Ensuring proper naming of received METS packages by Cellar operators, eliminating any potential naming inconsistencies.
  • Updated package zip file name for improved clarity and organization.
  • Investigation and resolution of saxon parser error trace, ensuring smoother processing of XML data.
  • Update of component detector for enhanced accuracy and reliability.
  • Implementation of advanced XPATH checking that allows for conditional XPATHs, providing greater flexibility in data mapping.
  • Fixing Airflow config resolver in the staging environment for improved configuration management.
  • Enhancement of validation/sparql/cm_assertions directory by deleting its contents before creating new files, ensuring clean and accurate validation reports.
  • Integration of pyinstaller for metabase-toolchain, simplifying the deployment and usage of the toolchain.
  • Cross-package SPARQL and SHACL Summary reports for comprehensive analysis and validation.
  • Improvement of CLI tools to work with absolute Field XPATHs, enabling precise mapping definitions.
  • Fixing CLI tools to work with empty Base XPATHs as well, accommodating various mapping scenarios.
  • Refactoring of the Mapping Workbench CLIs and TED-SWS services to enhance performance and maintainability.
  • METS package improvement for better compatibility and reliability.
  • Resolution of identified code smells in the TED-SWS codebase, ensuring high code quality.
  • Generation of complete SPARQL validation queries, including all required prefix declarations.
  • Adjustment of the ecosystem to ensure smooth and expected functioning of all components.
  • Refined SPARQL validation report generation, reconsidering result status assignment for cases when XPATH is not specified, particularly in Integration test status.
  • Correction of the mapping suite identifier generation in validation reports, eliminating incorrect CM versions.
  • Enhancement of XPATH coverage report to include XPATHs present in the "Mapping Remarks" sheet, providing a comprehensive view of coverage.
  • Fixing a potential ZeroDivisionError in sparql_runner that could occur during execution.
  • Test P3 delivery and preparation for release, ensuring a smooth and error-free release process.
  • CLI enhancement to copy necessary integration test files mentioned in column I ("Reference to Integration Tests"), simplifying test data management.
  • Addition of a new feature for CLIs to process subfolders from test_data into corresponding subfolders in the output, improving organization and efficiency.
  • Summary report improvements, including the inclusion of a count of notices for better overview and analysis.
  • Report generation enhancement, adding references from summary reports to detailed reports for easy navigation and analysis.
  • Consistent generation of validation report file names for improved organization and traceability.
  • Improved title of generated HTML report files for better readability and understanding.
  • Contextual information provision for each change in CM Diff, providing additional details for easier comprehension.
  • Integration of extra SPARQL queries for expanded validation capabilities.
  • Creation of a service and CLI for executing custom SPARQL unit tests, enabling tailored validation.
  • Development of SPARQL validation for all notices, similar to XPath validation, ensuring comprehensive data validation.
  • Display of the list of files where different result statuses occur, simplifying error identification and troubleshooting.

What's Changed

Full Changelog: 1.1.0-beta...1.2.0-rc.1

1.1.0-beta

22 Feb 20:53
b86cb82
Compare
Choose a tag to compare

What's Changed

Full Changelog: 1.0.0-beta...1.1.0-beta

1.0.0-beta

20 Nov 15:34
300ed23
Compare
Choose a tag to compare

What's Changed

Read more

0.0.9-gamma

08 Sep 15:29
833bf00
Compare
Choose a tag to compare
0.0.9-gamma Pre-release
Pre-release

What's Changed

Full Changelog: 0.0.9-beta...0.0.9-gamma

What's Changed

Full Changelog: 0.0.9-beta...0.0.9-gamma

0.0.9-beta

29 Aug 21:06
Compare
Choose a tag to compare

What's Changed

Read more