Implement Proxy Indexer

[jlind23]

The goal of this issue is to implement a new indexing mode - Proxy Indexer.

Proxy Indexer combines results from the https://epr.elastic.co/ together with other indexers. This way users don't have to download all packages to expose it, but just "link it" with the remote endpoint.

The feature can be used together with elastic-package to decrease the size of the EPR Docker image.

[mtojek]

@jsoriano Please let me know what do you think about this idea.

[jsoriano]

I like this idea. In combination with a local indexer it can be also a way for real users to combine custom packages with the oficial registry. Or this proxy can include more features like a local cache, that could maybe help in some use cases.

[jsoriano]

We may need to revisit how we configure the registry now. How would we configure a registry that uses a proxy indexer plus a local one?

[mtojek]

In combination with a local indexer it can be also a way for real users to combine custom packages with the oficial registry. Or this proxy can include more features like a local cache, that could maybe help in some use cases.

Yes, I was thinking about extensions and extra features, but maybe after MVP :)

We may need to revisit how we configure the registry now. How would we configure a registry that uses a proxy indexer plus a local one?

The original plan is to keep the local indexer enabled by default and make the proxy indexer switchable on a feature flag (similar to the storage indexer). I'm also fine with adjusting the configuration (YAML file).

[jsoriano]

Yeah, a --proxy-to https://epr.elastic.co flag may work by now 🙂

[mtojek]

After spending some time on implementation, I realized that it won't be a true indexer, as an indexer needs access to unexposed package properties (BasePath, semVer). Also, an indexer is not aware of the call context - all properties vs. base search ones.

I will give try a different approach - more like a proxy mode than an indexer.

[mtojek]

Released, resolving.