Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Black Duck Hub as a snippet scanner #4632

Open
porsche-rishisaxena opened this issue Oct 28, 2021 · 6 comments
Open

Support Black Duck Hub as a snippet scanner #4632

porsche-rishisaxena opened this issue Oct 28, 2021 · 6 comments
Labels
new feature Issues that are considered to be new features scanner About the scanner tool

Comments

@porsche-rishisaxena
Copy link

As ORT is an orchestrator, it should allow to configure BlackDuck as scanner where code snippet can be scanned and result can be stored in ORT backend storage i.e. PostgreSQL

High Level Consideration

  1. Analyzer-result.yml generated by running analyzer
  2. API call request to BlackDuck transforming .yml meta-data to .json format
  3. API response from BlackDuck in .json format
  4. scan-result from black-duck stored in ORT backend storage i.e., PostgreSQL
  5. scan-result is fed into ORT evaluate to report for generating various output formats such as Web App, and JSON.
@sschuberth sschuberth added new feature Issues that are considered to be new features scanner About the scanner tool labels Oct 28, 2021
@sschuberth
Copy link
Member

sschuberth commented Nov 18, 2021

Also see #3265 and #2819, FYI.

@sschuberth
Copy link
Member

Maybe also @JeroenKnoops's BlackDuck GitHub Action is of interest in this context.

@tsteenbe

This comment was marked as resolved.

@MarcelBochtler MarcelBochtler changed the title Black Duck Integration with ORT Black Duck Hub Integration with ORT Jul 7, 2022
@tsteenbe

This comment was marked as resolved.

@nnobelis
Copy link
Member

Hello,

We are in the process of designing a common abstraction to represent the snippets in the ORT model. This abstraction will be submitted to the ORT community.
Our plan is to support FossID and SCANOSS but we would like, if possible, to support also Blackduck.

Could someone provide a sample response of Blackduck (ideally on the Semver4j project), so we can have a look at their data model for snippets ?

@JeroenKnoops
Copy link

@nnobelis What kind of format do you require? The SPDX output?

nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 3, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties such as matched lines (present in both results) will
be added in the future.

Blackduck [2] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 3, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties such as matched lines (present in both results) will
be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 3, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties such as matched lines (present in both results) will
be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 3, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties such as matched lines (present in both results) will
be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 4, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 12, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 12, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 20, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 24, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue Apr 24, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue May 8, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue May 11, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are not License nor Copyright findings as a human
operator needs to review them and either accept or flag them as
false positives.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings.
This model has been created by comparing the results from FossID and
ScanOSS and trying to find a common abstraction. This is currently the
minimal model required to handle snippets.
Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet Data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue May 11, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are difference in nature from License and Copyright
findings as they reference a third party sourcecode.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings. This model has been
created by comparing the results from FossID and ScanOSS and trying to
find a common abstraction. This is currently the minimal model required
to handle snippets. Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue May 11, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are difference in nature from License and Copyright
findings as they reference a third party sourcecode.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings. This model has been
created by comparing the results from FossID and ScanOSS and trying to
find a common abstraction. This is currently the minimal model required
to handle snippets. Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue May 11, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are difference in nature from License and Copyright
findings as they reference a third party sourcecode.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings. This model has been
created by comparing the results from FossID and ScanOSS and trying to
find a common abstraction. This is currently the minimal model required
to handle snippets. Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
nnobelis added a commit to boschglobal/oss-review-toolkit that referenced this issue May 12, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are difference in nature from License and Copyright
findings as they reference a third party sourcecode.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings. This model has been
created by comparing the results from FossID and ScanOSS and trying to
find a common abstraction. This is currently the minimal model required
to handle snippets. Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet data
model for ORT.

Fixes: oss-review-toolkit#3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: oss-review-toolkit#4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
MarcelBochtler pushed a commit that referenced this issue May 15, 2023
Snippet scanners such as ScanOSS [1] and FossID [2] can identify code
snippets potentially coming from a third party source. To do so, they
scan the Internet for source code and build a Knowledge Base (KB). Then,
the source code to check for snippets is scanned and compared against
this KB.

Snippet Findings are difference in nature from License and Copyright
findings as they reference a third party sourcecode.

Therefore, this commit adds a new property ORT data model in the
`ScanSummary` to carry these snippet findings. This model has been
created by comparing the results from FossID and ScanOSS and trying to
find a common abstraction. This is currently the minimal model required
to handle snippets. Further properties will be added in the future.

Blackduck [3] is another scanner considered for integration in ORT [4]
which supports snippets. However since it does not deliver snippets
through its API, it was not considered when designing the snippet data
model for ORT.

Fixes: #3265.

[1]: https://www.scanoss.com/
[2]: https://fossid.com/
[3]: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html
[4]: #4632

Signed-off-by: Nicolas Nobelis <nicolas.nobelis@bosch.io>
@sschuberth sschuberth changed the title Black Duck Hub Integration with ORT Support BlackDuck Hub as a snippet scanner Jun 10, 2024
@sschuberth sschuberth changed the title Support BlackDuck Hub as a snippet scanner Support Black Duck Hub as a snippet scanner Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature Issues that are considered to be new features scanner About the scanner tool
Projects
Status: Future
Development

No branches or pull requests

5 participants