Configurable URL replacements for the analyzer / downloader

[dgutson]

We need to set some connection configuration of the vcs in general and git in particular, for example whether to use https instead of ssh.
Our current workaround is to place this in the docker entry point:

[sschuberth]

Similar feature requests occasionally came up in the past already, I remember @heliocastro having similar requirements. A configurable analyzer mechanism to do regex-based search & replace on URLs could also be used to replace hard-coded substitutions like not using git:// anymore for GitHub repos, see

ort/utils/ort/src/main/kotlin/Utils.kt

Lines 180 to 183 in 2e78d0f

	// Avoid using the unauthenticated Git protocol, which is blocked by many VCS hosts.
	if (url.startsWith("git://")) {
	url = "https://${url.removePrefix("git://")}"
	}

[fviernau]

A configurable analyzer mechanism to do regex-based search & replace on URLs could also be used to replace hard-coded substitutions like not using git:// anymore for GitHub repos, see

Note that this needs to work for submodules as well. (As long as we rely on recursive cloning)

[dgutson]

@fviernau s/not/note/ ?

If so, indeed I think a substitution wouldn't work for automatic submodules cloning, where ORT does not control the URL.

[dgutson]

I found more configuration rules that we had to set, which I'm not sure that can be specified as regexes:

git config --global url."https://github.com/".insteadOf ssh://git@github.com/
git config --global url."https://github.com/".insteadOf git@github.com:

[sschuberth]

Thanks to #6387 this could probably quite elegantly implemented as a special curation providers that does regex search & replaces on URLs from package metadata.

[sschuberth]

Note that this needs to work for submodules as well.

True, so any such URL replacement would also need to be applied by the downloader, not only the analyzer. A repo to test this with could be https://github.com/TritonDataCenter/node-http-signature/blob/v1.2.0/.gitmodules as it uses SSH / SCP URLs like url = git@github.com:davepacheco/javascriptlint for submodules.