-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
theforeman.foreman collection could not be installed when using Galactory as a proxy. #131
Comments
@mshonichev thank you again for such a detailed report, I really appreciate it. For my own reference, here's where the 2400 character limit is described: I'm glad the second upstream in I'll think on how to address this issue, perhaps with short and long-term options. Probably the first thing I'm going to change is separating out parts of the collection info into separate properties; at least, the pieces that glactory itself needs. It's not a full solution necessarily in that a single property could still end up > 2400 chars, but that would be an even more extreme edge case. This is a breaking change, but it has an advantage of making native Artifactory searches more useful by adding more individual properties that could be used in searches. A safe but messy approach is splitting collection info into multiple properties by length, essentially paginating it within the artifactory properties. This would be backward compatible with existing uploaded collections, though it wouldn't fix existing broken collections, which should be ok since they can be deleted and re-uploaded or re-proxied. Feels ugly though... |
I did not realise that Galactory uses Artifactory as a search engine aside to storage backend. That's actually pretty cool. IMO, splitting props to pages is cringe. It adds a whole lot of cases when meta changes - e.g. authors added/removed. Here, suppose we need to find a collection by its author. A first layer might be smth like
each individual
So, to get the actual archive out of search results you'll need a second query. It would too hit hard on corner cases to support such a structure, but at least it could be introduced without b/w compatibility break. Remember, that 'broken' collection with bloated meta was never actually uploaded, so 'broken' props need no fix. We just start up a whole new life aside, adding a few new search capabilities, and all previously uploaded files need no rehaul. |
Hi there again!
We've found another fancy issue in our setup of Galactory as a proxy / inner source for Ansible collections.
Fast forward to root cause: storing collection metadata in the Artifactory file properties was a nice idea, however for some collections, specifically
theforeman
.foreman
, metadata is such huge in size that it hits the Artifactory property value limit and thus is failed to properly deserialize.Reproducer:
ansible.cfg
is configured to use galactory as default galaxy server.and some collections don't:
container logs reveal stack trace:
The upstream collection was successfully downloaded from https://galaxy.ansible.com and stored in the Artifactory repository.
However, following are the properties of file (actually a tail of):
Apparently, The Foreman team decides to write down all the seven tribes of relatives (cats included!) as a collection authors list and while Galactory tries to put
collection_info
into file properties, an 2400 chars limit was hit and JSON was corrupted.Collection meta:
https://github.com/theforeman/foreman-ansible-modules/blob/a69d83fb8681ca80878d8143d27594eb72ee295d/galaxy.yml#L4
From the Artifactory docs:
I'm not sure that limit can be raised up easily, at least documentation is obscure about. Also I'm not sure if removing metadata from properties would not break the Galactory codebase to pieces.
So, currently we had to add upstream galaxy as a backup server as a workaround:
The text was updated successfully, but these errors were encountered: