Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolving of Elasticsearch Transport Client with Apache Ivy/Ant fails (since 5.4.0) #24490

Closed
uschindler opened this issue May 4, 2017 · 11 comments · Fixed by #24879
Closed
Labels
:Delivery/Build Build or test infrastructure :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts discuss >regression Team:Delivery Meta label for Delivery team

Comments

@uschindler
Copy link
Contributor

uschindler commented May 4, 2017

Hi,

I updated a Ivy-based Ant project that uses the Elasticsearch Transport Client a minute ago. After updating the dependency to the transport client (5.3.2 => 5.4.0) suddenly the project failed to build. Reason was, that some transitive dependencies of Elasticsearch were missing (e.g. Lucene itsself, various others).

The reason for this is #23809: Previously all the transitive dependencies in ES were excluded "one by one" in the POM file (see exclusions on each dependency): http://repo1.maven.org/maven2/org/elasticsearch/elasticsearch/5.3.2/elasticsearch-5.3.2.pom

In 5.4.0 it now has a *:* exclusion instead: http://repo1.maven.org/maven2/org/elasticsearch/elasticsearch/5.4.0/elasticsearch-5.4.0.pom

This seems to work with Maven (I built a plugin that uses Maven to refer to Elasticsearch and its transitive deps), but Apache Ivy/Ant breaks horribly. It seems that a bug in Ivy (2.3.0 and 2.4.0) causes that the * matches not only the transitives of elasticsearch, but also the declared dependency itsself. This causes that Ivy ignores all dependencies in Elasticsearch that are marked with this *:* exclusison pattern.

I am quite sure this is a bug in Ivy, but I don't think that gets fixed in time (Ant's Ivy project is almost dead...?).

As Gradle also uses Ivy internally, are we sure this works with Gradle?

@uschindler
Copy link
Contributor Author

uschindler commented May 4, 2017

There is already a bug open @ Ivy that @nik9000 also took place in last December. No activity 👎
https://issues.apache.org/jira/browse/IVY-1531

@uschindler
Copy link
Contributor Author

See also #21170

@danielmitterdorfer
Copy link
Member

danielmitterdorfer commented May 5, 2017

The likely cause for this is #24056. @rjernst any thoughts on how we should proceed with this ticket (also considering the fact that Ivy might not ever fix this...)?

@danielmitterdorfer danielmitterdorfer added :Delivery/Build Build or test infrastructure :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts >bug v5.4.0 >regression discuss and removed >bug v5.4.0 labels May 5, 2017
@rjernst
Copy link
Member

rjernst commented May 5, 2017

My changes in #24056 were intentional, because without them the generated pom is broken: it contains two exclusions sections, one with the wildcard exclusion added by gradle, and the other with our explicit exclusions. Unfortunately I had indeed forgot about #21170, and since we have no tests for any downstream build tools (including gradle and maven!), I did not catch this earlier. It is possible to fix this by reverting #24056, but will also require some additional change to find and remove the exclusions tag added by gradle. But I am a bit concerned we will keep hitting random issues with build tools until we add tests (which will be a bit tricky) which consume our generated poms.

@yrodiere
Copy link
Contributor

Is there any chance this could get fixed in a 5.4.1? The problem not only affects Apache Ivy users, but also maven-enforcer-plugin users (see #24626).

@jasontedor
Copy link
Member

Is there any chance this could get fixed in a 5.4.1?

This will definitely not be addressed in 5.4.1, we are still considering what to do here.

@jasontedor
Copy link
Member

@rjernst Any additional thoughts on this one?

@rjernst
Copy link
Member

rjernst commented May 24, 2017

We need to revert my original commit and modify the pom generation logic to replace the exclusions in the pom instead of add. I just have not had time to do this yet.

rjernst added a commit to rjernst/elasticsearch that referenced this issue May 25, 2017
When transitive dependencies are disable for a dependency, gradle adds a
wildcard exclusion to the generated pom. However, some external tools
like ivy have bugs with wildcards. This commit adds back the explicit
generation of transitive excludes, and removes the gradle generated
exclusions element from the pom.

closes elastic#24490
@rjernst
Copy link
Member

rjernst commented May 25, 2017

I opened #24879 with a fix.

rjernst added a commit that referenced this issue May 25, 2017
)

When transitive dependencies are disable for a dependency, gradle adds a
wildcard exclusion to the generated pom. However, some external tools
like ivy have bugs with wildcards. This commit adds back the explicit
generation of transitive excludes, and removes the gradle generated
exclusions element from the pom.

closes #24490
rjernst added a commit that referenced this issue May 25, 2017
)

When transitive dependencies are disable for a dependency, gradle adds a
wildcard exclusion to the generated pom. However, some external tools
like ivy have bugs with wildcards. This commit adds back the explicit
generation of transitive excludes, and removes the gradle generated
exclusions element from the pom.

closes #24490
rjernst added a commit that referenced this issue May 25, 2017
)

When transitive dependencies are disable for a dependency, gradle adds a
wildcard exclusion to the generated pom. However, some external tools
like ivy have bugs with wildcards. This commit adds back the explicit
generation of transitive excludes, and removes the gradle generated
exclusions element from the pom.

closes #24490
@jasontedor
Copy link
Member

Is there any chance this could get fixed in a 5.4.1?

This will definitely not be addressed in 5.4.1, we are still considering what to do here.

We are going to address this in 5.4.1 after all.

@uschindler
Copy link
Contributor Author

Great! Thanks.

trask added a commit to trask/jBrowserDriver that referenced this issue Jan 22, 2018
Sadly, wildcard exclusions have some issues for downstream projects
e.g. https://issues.apache.org/jira/browse/MASSEMBLY-675 and
elastic/elasticsearch#24490
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts discuss >regression Team:Delivery Meta label for Delivery team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants