Create a Jenkinsfile to run scan on OpenSearch-projects repos

[zelinh]

Signed-off-by: Zelin Hao zelinhao@amazon.com

Description

Since #1181 is not working because of GHA. We will now use Jenkins to automate WhiteSource scan. All repos included in wss-scan.config are tested to be running properly in Jenkins. "OpenSearch" is not included since we have encountered some issues scanning it within docker container.

Refer to #1157, WhiteSource integration may not work correctly at this time. They have an issue on their side. We are switching to our Plan B using Jenkins for automating WhiteSource scan based on old script from ODFE. This workflow will update the scan results in main product "OpenSearch" on WhiteSource dashboard.

Issues Resolved

part of #1157

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #1268 (9292325) into main (e162aec) will increase coverage by 1.22%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #1268      +/-   ##
============================================
+ Coverage     92.87%   94.09%   +1.22%     
- Complexity        9       11       +2     
============================================
  Files            97      127      +30     
  Lines          2441     2863     +422     
  Branches          3       10       +7     
============================================
+ Hits           2267     2694     +427     
+ Misses          170      159      -11     
- Partials          4       10       +6     

Impacted Files	Coverage Δ
src/jenkins/InputManifest.groovy	58.06% <0.00%> (-16.94%)	⬇️
src/run_integ_test.py	86.66% <0.00%> (-6.02%)	⬇️
src/test_workflow/perf_test/perf_test_cluster.py	95.74% <0.00%> (-4.26%)	⬇️
src/jenkins/BuildManifest.groovy	73.33% <0.00%> (-3.59%)	⬇️
src/test_workflow/integ_test/integ_test_suite.py	91.48% <0.00%> (-1.85%)	⬇️
src/test_workflow/dependency_installer.py	93.75% <0.00%> (-1.71%)	⬇️
src/assemble_workflow/dist.py	94.64% <0.00%> (-1.44%)	⬇️
src/run_build.py	91.30% <0.00%> (ø)
src/manifests/manifests.py	100.00% <0.00%> (ø)
src/paths/script_finder.py	100.00% <0.00%> (ø)
... and 60 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e162aec...9292325. Read the comment docs.

[dblock]

What do you think about moving the list of repos to scan into Jenkinsfile and use the list to generate stages that are then executed in parallel? (e.g. kicks off a job with the argument of repo to scan, then waits for all the started jobs, or maybe there's a way to create stages dynamically).

[zelinh]

What do you think about moving the list of repos to scan into Jenkinsfile and use the list to generate stages that are then executed in parallel? (e.g. kicks off a job with the argument of repo to scan, then waits for all the started jobs, or maybe there's a way to create stages dynamically).

The reason I keep the list of repos in wss-scan.config is that I want to make minimum change to what we had in our repo. But I do like the idea that we have this list in Jenkinsfile as stages; passing the repo as argument or just use the default list seems to be a good idea for other people to kickoff scan. Maybe we could create an issue for this feature as enhancement?