Support horizontal edc scaling in cp adapter extension

[marcingajek-zf]

No description provided.

[florianrusch-zf]

Just reviewed the changes in the documentation so far

[ghost]

@DominikPinsel can we handle this initial db initialisation somehow with the migration extension?

I think we should avoid, that the operator has to do something manually in the DB (expect creating DBs)!

[marcingajek-zf]

done

[ndr-brt]

Looks pretty much good, I would suggest to add an end to end test that verifies all the components integration

[marcingajek-zf]

@ndr-brt please take a look if the PR can be merged.

[ndr-brt]

As I said in the previous I think an integration test that verifies all the flow from start to end would make a lot of sense.
Furthermore, tests on the Sql stores implementation would be important to verify mappings and queries

[ndr-brt]

This class seems to be not tested, I would suggest at least a save/retrieve test against a sql server to verify mapping and queries.

[marcingajek-zf]

It's not tested as I've cretaed only unit test for the classes with logic. SqlQueueStore is a utily class.

Integration tests are created within: https://jira.catena-x.net/browse/A1IDSC-393

Task is inprogress, and I believe there could be tests that make use of DB to test if reads/writes are correct

[ndr-brt]

Business tests are a little far from the logic and hard to debug, I'm not in favor of delegating integration tests against DB to the E2E layer because this could become tedious.
And, we shouldn't forget that our Business tests currently are flaky

[marcingajek-zf]

@ndr-brt
But unit test are also not good place for testing integration with DB.
Do You know some example in product-edc where this kind of DB test is cerated?

[ndr-brt]

this is actually the first time we have a sql store in product edc, but in EDC they are well tested, e.g.:
https://github.com/eclipse-edc/Connector/blob/main/extensions/control-plane/store/sql/asset-index-sql/src/test/java/org/eclipse/edc/connector/store/sql/assetindex/PostgresAssetIndexTest.java

there the test suites are separated by meaning: unit tests, postgres integration tests, daps integration tests, ...
a good practice is to follow the "fail fast" pattern, furthermore business tests should test business, so e2e use cases, not integration.

[marcingajek-zf]

@ndr-brt
Ok, I was not aware that there are test in EDC that uses real DB during test run.
Looks to me that I can not use the "testFixtures" from core edc and the whole db setup.
I understand that You expect me to set it all up for product-edc?
I'm asking as I'm starting to have some doubts if this will be ready for next release.

[ndr-brt]

I'm not really inclined to have features that aren't duly covered but, if we're sure that this works, good for me. I will just approve the PR, eager to see a subsequent one that covers the missing tests.

[marcingajek-zf]

Thank You Andrea,
I really appreciate all Your comments, they are very usefull. I will try to add such test in future. Hope to find someone that knows how those EDC test works, as for now they are failing when I run them. Maybe there is some additional config that I've missed, but I would expect that running a method with @test annotation should be enough...

[ndr-brt]

to run them you should either add the includeTags property as it's done by the ci workflow:
https://github.com/eclipse-edc/Connector/blob/main/.github/workflows/verify.yaml#L212
or to comment the @PostgresqlIntegrationTest annotation.

I usually do the latter during development :)

[marcingajek-zf]

@florianrusch-zf is it ok for You to mrege the PR ?