Epic about using custom / newer ScanCode versions

[sschuberth]

Currently, the ScanCode version to use is hard-coded in ORT. This tight coupling prevents mixing of arbitrary ORT / ScanCode versions, and hinders a fast / user-specific upgrade of ScanCode (also see #4296).

To solve this, the following things would need to be addressed:

• Improve the boostrapping of scanners:
  • Accept a scanner in any version, as long as it is present on the system (leaving it to the user to install a compatible version; also see Make versions of local scanners configurable #4582); see LocalScanner: Remove the version requirement for scanners #4669
  • Bootstrap to a cached location for reuse (like done for Flutter; also see scan downloading scancode-toolkit each time. please made a cache of it once per installation session #1580).
  • Alternatively: Remove bootstrapping for scanners completely. However, that would requite CI to always install all supported scanners, and not only those whose tests are actually run.
• Improve handling of license texts (also see this comment):
  • Refactor the utils/core module so that utils/spdx can depend on non-ORT-specific parts of it (the plan is to introduce utils/common; see Start splitting :utils:core #4647, Move more code from :utils:core to :utils:common #4652, Move remaining common code from spdx-utils to common-utils #4670, Finalize the utils module refactoring #4671)
  • Read license texts from the ScanCode version in use rather than embedding them, see Refactor scancode license texts #4523.
• Verify that the scanner criteria for looking up stored scan results can be configured flexible enough to e.g. declare ScanCode results of all versions from 3.2.1-rc2 to 30.1.0 as compatible:
  • Make isCompatible() Scanner-aware #3247
  • Filter for scanner configuration in ScanResult storages is too restrictive #3223
• Make the ScanCode version in the Docker image easily configurable (see Expose the ScanCode version as a Docker build argument #4721)

FAQ

Why not just always use the latest version of ScanCode in ORT?

• Different findings might render existing package configurations invalid.
• Expensive testing might be required.

[maxhbr]

#1580 is closed. Does that mean that all boxes in "Improve the boostrapping of scanners" can be checked?

[sschuberth]

Yes, thanks for the reminder, done.

[mnonnenmacher]

@sschuberth Don't you think we can close this epic as the two open tickets are not required for choosing a custom ScanCode version?

[sschuberth]

Agreed. #3247 is anyway n/a anymore by now, and #3223 should not block / be part of this epic.