Adding plots to ITSTPC matching QC #11974

chiarazampolli · 2023-09-27T16:13:52Z

Added:

matching with ITS,
mathcing vs 1./pt,
TPC tracks DCA,
matching vs pt removing eta ~0.

To test:

o2-global-track-cluster-reader --tpc-track-reader "tpctracks.root --reader-delay 10" --shm-segment-size 64000000000 --disable-mc --hbfutils-config o2_tfidinfo.root --track-types "ITS,TPC,ITS-TPC" --cluster-types "TPC,ITS" | o2-qc --config json:///home/zampolli/work/O2/RelVal/CPU_6/QC_GLO.json -b

Update of O2DPG json to be done.

Corresponding changes in QC: AliceO2Group/QualityControl#2001

Comparison of matching vs pt with previous version:

alibuild · 2023-09-27T21:07:18Z

Error while checking build/O2/fullCI for 7f8f6f1 at 2023-09-27 23:07:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/QualityControl-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:67:55: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPt'; did you mean 'getHistos'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:68:55: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPtTPC'; did you mean 'getHistoPtNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:69:77: error: no matching function for call to 'o2::globaltracking::MatchITSTPCQC::getFractionITSTPCmatch()'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:71:55: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPhi'; did you mean 'getHistoPhiNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:72:55: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPhiTPC'; did you mean 'getHistoPhiNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:73:80: error: no matching function for call to 'o2::globaltracking::MatchITSTPCQC::getFractionITSTPCmatchPhi()'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:74:84: error: no matching function for call to 'o2::globaltracking::MatchITSTPCQC::getFractionITSTPCmatchPhiVsPt()'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:76:55: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoEta'; did you mean 'getHistoEtaNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:77:55: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoEtaTPC'; did you mean 'getHistoEtaNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:78:80: error: no matching function for call to 'o2::globaltracking::MatchITSTPCQC::getFractionITSTPCmatchEta()'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:79:84: error: no matching function for call to 'o2::globaltracking::MatchITSTPCQC::getFractionITSTPCmatchEtaVsPt()'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:91:57: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPtPhysPrim'; did you mean 'getHistoPtPhysPrimNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:92:57: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPtTPCPhysPrim'; did you mean 'getHistoPtPhysPrimNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:93:87: error: no matching function for call to 'o2::globaltracking::MatchITSTPCQC::getFractionITSTPCmatchPhysPrim()'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:95:57: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPhiPhysPrim'; did you mean 'getHistoPhiPhysPrimNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:96:57: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPhiTPCPhysPrim'; did you mean 'getHistoPhiPhysPrimNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:97:90: error: no matching function for call to 'o2::globaltracking::MatchITSTPCQC::getFractionITSTPCmatchPhiPhysPrim()'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:99:57: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoEtaPhysPrim'; did you mean 'getHistoEtaPhysPrimNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:100:57: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoEtaTPCPhysPrim'; did you mean 'getHistoEtaPhysPrimNum'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:101:90: error: no matching function for call to 'o2::globaltracking::MatchITSTPCQC::getFractionITSTPCmatchEtaPhysPrim()'
ninja: build stopped: cannot make progress due to previous errors.

Full log here.

chiarazampolli · 2023-09-28T07:57:45Z

Moving to WIP while I fix the formatting etc.

chiarazampolli · 2023-09-28T21:39:01Z

Hello @shahor02 , @noferini , @JianLIUhep, @knopers8 , @Barthelemy ,
Again we have interdependecies bwtween QC and O2.
I think we should merge this one, if O2 compiled (QC will not for sure). Then QC will be broken, and we'll need to merge AliceO2Group/QualityControl#2001.
Then I will update the json for sync and async in O2DPG.
Cheers,
Chiara

shahor02 · 2023-09-28T21:46:21Z

@chiarazampolli if you keep unchanged old getters (even ie making them dummy when needed) and add new ones, the QC should not fail. Then, after merging this PR, then QC one, one can suppress old methods.

chiarazampolli · 2023-09-28T21:50:27Z

Hello @shahor02 ,
Since I now removed everything that was old, isn't it enough to see that the O2 compilation works, and QC breaks where it is expected to break?
Cheers,
Chiara

chiarazampolli · 2023-09-28T21:59:35Z

Uploading json file to test. In order to upload, I had to change the extension, but it is a json file.
QC_GLO.txt

shahor02 · 2023-09-28T22:30:26Z

Then the QC will be broken for a day or so. Could just add the lines below somewhere before private?

  // to remove after merging QC PR
  TH1D* getHistoPt() const { return nullptr; }                               // old
  TH1D* getHistoPtTPC() const { return nullptr; }                            // old
  TEfficiency* getFractionITSTPCmatch() const { return nullptr; }            // old
  TH1F* getHistoPhi() const { return nullptr; }                              // old
  TH1F* getHistoPhiTPC() const { return nullptr; }                           // old
  TEfficiency* getFractionITSTPCmatchPhi() const { return nullptr; }         // old
  TH2F* getHistoPhiVsPt() const { return nullptr; }                          // old
  TEfficiency* getFractionITSTPCmatchPhiVsPt() const { return nullptr; }     // old
  TH1F* getHistoEta() const { return nullptr; }                              // old
  TEfficiency* getFractionITSTPCmatchEta() const { return nullptr; }         // old
  TH2F* getHistoEtaVsPt() const { return nullptr; }                          // old
  TH2F* getHistoEtaVsPtTPC() const { return nullptr; }                       // old
  TEfficiency* getFractionITSTPCmatchEtaVsPt() const { return nullptr; }     // old
  TH1F* getHistoPtPhysPrim() const { return nullptr; }                       // old
  TH1F* getHistoPtTPCPhysPrim() const { return nullptr; }                    // old
  TEfficiency* getFractionITSTPCmatchPhysPrim() const { return nullptr; }    // old
  TH1F* getHistoPhiPhysPrim() const { return nullptr; }                      // old
  TEfficiency* getFractionITSTPCmatchPhiPhysPrim() const { return nullptr; } // old
  TH1F* getHistoEtaPhysPrim() const { return nullptr; }                      // old
  TH1F* getHistoEtaTPCPhysPrim() const { return nullptr; }                   // old
  TEfficiency* getFractionITSTPCmatchEtaPhysPrim() const { return nullptr; } // old

chiarazampolli · 2023-09-29T04:39:53Z

Hello @shahor02 ,
Sure, I will change it.
Cheers,
Chiara

alibuild · 2023-09-29T05:52:33Z

Error while checking build/O2/fullCI for 518c920 at 2023-09-29 07:52:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2Physics-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[0 more errors; see full log]

Full log here.

alibuild · 2023-09-29T12:26:04Z

Error while checking build/O2/fullCI for c3d8cda at 2023-09-29 14:26:

## sw/BUILD/QualityControl-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:77:55: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoEtaTPC'; did you mean 'getHistoPtTPC'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:96:57: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPhiTPCPhysPrim'; did you mean 'getHistoPtTPCPhysPrim'?
ninja: build stopped: cannot make progress due to previous errors.


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/2d0fba1dbf0aa9bd6f64745178054444bce2757d/slc8_x86-64/o2checkcode/1.0-local137/etc/modulefiles
++ cat
--

Full log here.

alibuild · 2023-09-29T13:39:54Z

Error while checking build/O2/fullCI for c9df921 at 2023-09-29 16:32:

## sw/BUILD/O2Physics-latest/log
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.


## sw/BUILD/QualityControl-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:77:55: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoEtaTPC'; did you mean 'getHistoPtTPC'?
/sw/SOURCES/QualityControl/v1.118.0/v1.118.0/Modules/GLO/src/ITSTPCMatchingTask.cxx:96:57: error: 'class o2::globaltracking::MatchITSTPCQC' has no member named 'getHistoPhiTPCPhysPrim'; did you mean 'getHistoPtTPCPhysPrim'?
ninja: build stopped: cannot make progress due to previous errors.


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/9e48d1610447e6b7648533e47b5650c314e71321/slc8_x86-64/o2checkcode/1.0-local139/etc/modulefiles
++ cat
--

Full log here.

chiarazampolli · 2023-09-29T15:46:32Z

Hello @shahor02 ,
I have the impression that now the failures are unrelated. Could you please check?
Cheers,
Chiara

shahor02 · 2023-09-29T15:51:53Z

Actually, the CIs fail to start with Git fetch failed.
@TimoWilken could you please check?

alibuild · 2023-09-29T16:33:11Z

Error while checking build/O2/fullCI for 9c079a2 at 2023-10-02 08:02:

## sw/BUILD/O2Physics-latest/log
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.


## sw/BUILD/QualityControl-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2-full-system-test-latest/log
Detected critical problem in logfile reco_NOGPU.log
reco_NOGPU.log-[ERROR] pid 45919 (qc-task-GLO-MTCITSTPC) crashed with or was killed with exit code 1
reco_NOGPU.log:[ERROR] Workflow crashed - pid 45916 (qc-task-FDD-DigitQcTaskFDD) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
reco_NOGPU.log:[ERROR] Workflow crashed - pid 45917 (qc-task-FT0-DigitQcTaskFT0) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
reco_NOGPU.log:[ERROR] Workflow crashed - pid 45918 (qc-task-FV0-DigitQcTaskFV0) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
reco_NOGPU.log:[ERROR] Workflow crashed - pid 45937 (qc-task-TRD-Tracklets) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
reco_NOGPU.log:[ERROR] Workflow crashed - pid 45936 (qc-task-TRD-Digits) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] pid 45919 (qc-task-GLO-MTCITSTPC) crashed with or was killed with exit code 1
[ERROR] Workflow crashed - pid 45916 (qc-task-FDD-DigitQcTaskFDD) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] Workflow crashed - pid 45917 (qc-task-FT0-DigitQcTaskFT0) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] Workflow crashed - pid 45918 (qc-task-FV0-DigitQcTaskFV0) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] Workflow crashed - pid 45937 (qc-task-TRD-Tracklets) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] Workflow crashed - pid 45936 (qc-task-TRD-Digits) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/209917df70cced6bbf99b68a9d802089d9e49d04/slc8_x86-64/o2checkcode/1.0-local1307/etc/modulefiles
++ cat
--
[0 more errors; see full log]

Full log here.

chiarazampolli · 2023-10-01T17:08:15Z

Hello @shahor02 , @TimoWilken ,
I don't think the errors are related to the PR. Can it be reviewed?
Cheers,
Chiara

shahor02 · 2023-10-01T23:18:33Z

Hi @chiarazampolli

the macOS-arm errors are genuine: to compose histo title you use more printf format specifiers than arguments:

/Volumes/build/alice-ci-workdir/o2/sw/SOURCES/O2/11974/0/Detectors/GlobalTracking/src/MatchITSTPCQC.cxx:149:105: error: more '%' conversions than data arguments [-Werror,-Wformat-insufficient-args]
    mPhiDen[i] = new TH1F(Form("mPhiDen_%s", title[i].c_str()), Form("Phi distribution of %s tracks %s %s; Phi [rad] %s; dNdPhi", title[i].c_str(), etaSel[i].c_str()), 100, 0.f, 2 * TMath::Pi());
   ...

Besides that the fullCI seg.faults in the qc-task-GLO-MTCITSTPC: I guess this is because current QC calls the dummy getters we put in this PR to not break the QC compilation. If in you local tests with modified QC it works fine, we can ignore the fullCI error.

chiarazampolli · 2023-10-02T07:25:06Z

Hello @shahor02 ,
I saw the failure in the CI, and did not understand it was a crash. Locally, with updated QC, it works. Unfortunately, when two packages depend on each other, I think some failures are to be expected.
I will fix mac, sure.
Thanks for taking a look.
Chiara

alibuild · 2023-10-02T11:45:43Z

Error while checking build/O2/fullCI for a31ff55 at 2023-10-02 15:02:

## sw/BUILD/O2Physics-latest/log
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.


## sw/BUILD/QualityControl-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2-full-system-test-latest/log
Detected critical problem in logfile reco_NOGPU.log
[ERROR] pid 47929 (qc-task-GLO-MTCITSTPC) crashed with or was killed with exit code 1
[ERROR] Workflow crashed - pid 47926 (qc-task-FDD-DigitQcTaskFDD) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] Workflow crashed - pid 47927 (qc-task-FT0-DigitQcTaskFT0) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] Workflow crashed - pid 47928 (qc-task-FV0-DigitQcTaskFV0) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] Workflow crashed - pid 47977 (qc-task-TRD-Digits) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...
[ERROR] Workflow crashed - pid 47980 (qc-task-TRD-Tracklets) was killed abnormally with exit code 128, could be out of memory killer, segfault, unhandled exception, SIGKILL, etc...


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/daaa7a1116c94b61080fae207ec5204dd99189e0/slc8_x86-64/o2checkcode/1.0-local34/etc/modulefiles
++ cat
--

Full log here.

alibuild · 2023-10-02T15:07:11Z

Error while checking build/O2/fullCI for 1c14f70 at 2023-10-05 02:44:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2Physics-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
Error in cling::AutoLoadingVisitor::InsertIntoAutoLoadingState:
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[0 more errors; see full log]

Full log here.

chiarazampolli · 2023-10-02T19:16:32Z

Hello @shahor02 ,
I would say that now it is fine, right?
Chiara

martenole · 2023-10-03T07:58:11Z

But isn't there a way to avoid breaking the fullCI?

[47960:internal-dpl-ccdb-backend]: [05:36:27][INFO] ccdb reads http://alice-ccdb.cern.ch/MCH/Calib/CTFDictionaryTree/1546300800000/e4eb37f0-06d8-11ee-b4d8-200114580201 for 1550600800000 (load to memory, agent_id: alimetal04.cern.ch-1696311382-bI3HMd), 
[48073:qc-task-GLO-MTCITSTPC]: *** Program crashed (Segmentation fault, FPE, BUS, ABRT, KILL, Unhandled Exception, ...)
[48073:qc-task-GLO-MTCITSTPC]: Backtrace by DPL:
[48073:qc-task-GLO-MTCITSTPC]: SEGMENTATION FAULT
[48073:qc-task-GLO-MTCITSTPC]: Executable is /sw/slc8_x86-64/QualityControl/v1.118.0-local48/bin/o2-qc
[47960:internal-dpl-ccdb-backend]: Info in <TJAlienFile::Open>: Accessing file /alice/data/CCDB/MFT/Calib/CTFDictionaryTree/05/53623/a61db061-06d8-11ee-b4d8-200114580201 in SE <ALICE::CERN::OCDB>
[48073:qc-task-GLO-MTCITSTPC]:     /lib64/libc.so.6:     ?? ??:0
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local48/lib/libO2QualityControl.so: o2::quality_control::core::ObjectsManager::startPublishing(TObject*)
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local48/lib/libO2QcGLO.so: o2::quality_control_modules::glo::ITSTPCMatchingTask::initialize(o2::framework::InitContext&)
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local48/lib/libO2QualityControl.so: o2::quality_control::core::TaskRunner::init(o2::framework::InitContext&)
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local48/lib/libO2QualityControl.so: o2::framework::adaptFromTask<o2::quality_control::core::TaskRunner, o2::quality_control::core::TaskRunner>(o2::quality_control::core::TaskRunner&&)::{lambda(o2::framework::InitContext&)#1}::operator()(o2::framework::InitContext&) const
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local48/lib/libO2QualityControl.so: std::_Function_handler<std::function<void (o2::framework::ProcessingContext&)> (o2::framework::InitContext&), o2::framework::adaptFromTask<o2::quality_control::core::TaskRunner, o2::quality_control::core::TaskRunner>(o2::quality_control::core::TaskRunner&&)::{lambda(o2::framework::InitContext&)#1}>::_M_invoke(std::_Any_data const&, o2::framework::InitContext&)
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/O2/11974-slc8_x86-64-local2/lib/libO2Framework.so: o2::framework::DataProcessingDevice::Init()
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/FairMQ/v1.8.0-6/lib/libfairmq.so.1.8.0: fair::mq::Device::InitWrapper()
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/FairMQ/v1.8.0-6/lib/libfairmq.so.1.8.0: boost::detail::function::void_function_obj_invoker1<std::function<void (fair::mq::State)>, void, fair::mq::State>::invoke(boost::detail::function::function_buffer&, fair::mq::State)
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/FairMQ/v1.8.0-6/lib/libfairmq.so.1.8.0: boost::signals2::detail::signal_impl<void (fair::mq::State), boost::signals2::optional_last_value<void>, int, std::less<int>, boost::function<void (fair::mq::State)>, boost::function<void (boost::signals2::connection const&, fair::mq::State)>, boost::signals2::mutex>::operator()(fair::mq::State)
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/FairMQ/v1.8.0-6/lib/libfairmq.so.1.8.0: fair::mq::fsm::Machine_::ProcessWork()
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/FairMQ/v1.8.0-6/lib/libfairmq.so.1.8.0: fair::mq::StateMachine::ProcessWork()
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/FairMQ/v1.8.0-6/lib/libfairmq.so.1.8.0: fair::mq::DeviceRunner::Run()
[48073:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/O2/11974-slc8_x86-64-local2/lib/libO2Framework.so: doChild(int, char**, o2::framework::ServiceRegistry&, o2::framework::RunningWorkflowInfo const&, o2::framework::RunningDeviceRef, o2::framework::DriverConfig const&, o2::framework::ProcessingPolicies, std::__cxx11::basic_string<char,

E.g. changing QC first and add a protection or simply disabling the task in the O2DPG json file in O2DPG used for the fullCI?

chiarazampolli · 2023-10-03T08:13:48Z

Hello @martenole ,
Either O2 or QC will be broken, I don't know how to overcome it. QC is now waiting for this PR.
Cheers,
Chiara

shahor02 · 2023-10-03T11:52:55Z

This would require making old (obsolete) getters (used by QC) aliased to new getters instead of returning nullptr (which makes to QC crash). I don't think it is worth it: the fullCI fails where it is expected to fail in the QC and locally, with fixed QC Chiara can run fullCI (@chiarazampolli can you confirm that you run fullCI and not the FST only)

WORKFLOW_EXTRA_PROCESSING_STEPS=TPC_DEDX,MFT_RECO,MID_RECO,MCH_RECO,MATCH_MFTMCH,MATCH_MCHMID,MUON_SYNC_RECO,ZDC_RECO FST_SYNC_EXTRA_WORKFLOW_PARAMETERS=QC,CALIB_LOCAL_AGGREGATOR,CALIB_LOCAL_INTEGRATED_AGGREGATOR QC_REDIRECT_MERGER_TO_LOCALHOST=1 GEN_TOPO_WORKDIR=`pwd` ALICE_O2SIM_DUMPLOG=1 NEvents=5 NEventsQED=100 O2SIMSEED=12345 $O2_ROOT/prodtests/full_system_test.sh
$O2_ROOT/prodtests/full_system_test_ci_extra_tests.sh

chiarazampolli · 2023-10-03T12:02:30Z

Hi @shahor02 ,
The command I ran is:

o2-global-track-cluster-reader --tpc-track-reader "tpctracks.root --reader-delay 10" --shm-segment-size 64000000000 --disable-mc --hbfutils-config o2_tfidinfo.root --track-types "ITS,TPC,ITS-TPC" --cluster-types "TPC,ITS" | o2-qc --config json:///home/zampolli/work/O2/RelVal/CPU_6/QC_GLO.json -b

as reported at the beginning of the PR. I can also test the fullCI.
Chiara

chiarazampolli · 2023-10-03T14:38:54Z

Hello,

fullCI needs an update in the json in O2DPG, as I planned to do after this PR. At this point, I will do it now.
But the command:

{code}
$O2_ROOT/prodtests/full_system_test_ci_extra_tests.s
{code}

did not work for me, something unrelated, I would say:

{code}
[O2PDPSuite/latest-o2] ~/work/O2/fullCI $> more o2-raw-file-reader-workflow.log
Running: TIME="#walltime %e" /usr/bin/time --output=o2-raw-file-reader-workflow.log_time ./o2-raw-file-reader-workflow.log_tmp.sh
[INFO] WorkflowSpec defineDataProcessing
[INFO] WorkflowSpec FLPWorkflow
[INFO] ft0-datareader-dpl
[ERROR] error parsing options of o2-ft0-flp-dpl-workflow: the required argument for option '--shm-segment-size' is missing
TASK-EXIT-CODE: 1
{code}

I am now opening the PR in O2DPG.

Chiara

chiarazampolli · 2023-10-03T14:41:50Z

Here it is:

https://github.com/AliceO2Group/O2DPG/pull/1257/files

@shahor02 , if you agree, we can merge it.

chiarazampolli · 2023-10-04T11:55:17Z

Hello @shahor02 ,
The CI re-run, but indeed it is failing in publishing.

[47568:qc-task-GLO-MTCITSTPC]:     /lib64/libc.so.6:     ?? ??:0
[47568:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local65/lib/libO2QualityControl.so: o2::quality_control::core::ObjectsManager::startPublishing(TObject*)
[47568:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local65/lib/libO2QcGLO.so: o2::quality_control_modules::glo::ITSTPCMatchingTask::initialize(o2::framework::InitContext&)
[47568:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local65/lib/libO2QualityControl.so: o2::quality_control::core::TaskRunner::init(o2::framework::InitContext&)
[47568:qc-task-GLO-MTCITSTPC]:     /sw/slc8_x86-64/QualityControl/v1.118.0-local65/lib/libO2QualityControl.so: o2::framework::adaptFromTask<o2::quality_control::core::TaskRunner, o2::quality_control::core::TaskRunner>(o2::quality_control::core::TaskRunner&&)::{lambda(o2::framework::InitContext&)#1}::operator()(o2::framework::InitContext&) const

Since the second test you proposed failed, but not due to QC, maybe we can anyway merge? Have you reviewed the PR?
Cheers,
Chiara

shahor02 · 2023-10-04T15:47:21Z

Hi @chiarazampolli ,

Sorry, I don't understand, why it is failing in publishing if you are using locally fixed QC?
I've checked your PR and haven't seen obvious problems.

Cheers,
Ruben

chiarazampolli · 2023-10-04T15:54:32Z

Hello @shahor02 ,

The publishing is failing in the fullCI not in my local test. The log in #11974 (comment), is from fullCI.

My local test fails in the second test you proposed, see #11974 (comment)

[O2PDPSuite/latest-o2] ~/work/O2/fullCI $> more o2-raw-file-reader-workflow.log
Running: TIME="#walltime %e" /usr/bin/time --output=o2-raw-file-reader-workflow.log_time ./o2-raw-file-reader-workflow.log_tmp.sh
[INFO] WorkflowSpec defineDataProcessing
[INFO] WorkflowSpec FLPWorkflow
[INFO] ft0-datareader-dpl
[ERROR] error parsing options of o2-ft0-flp-dpl-workflow: the required argument for option '--shm-segment-size' is missing
TASK-EXIT-CODE: 1

Cheers,

Chiara

shahor02 · 2023-10-04T16:07:23Z

I understand that it is the fullCI, this was the point: to run locally fullCI with local QC patch applied. If this was the case, why does it fail?
The ull_system_test_ci_extra_tests.sh is irrelevant, this is FT0 raw data reading test.

chiarazampolli · 2023-10-04T16:24:14Z

I meant it is fullCI from the CI tests here.

chiarazampolli · 2023-10-05T07:52:35Z

Hello @shahor02 ,

I am sorry to bother again, I would like to understand how we should proceed here. I repeat the situation:

local fullCI works;
fullCI from the CI is broken due to not updated QC.

Cheers,

Chiara

shahor02 · 2023-10-05T07:59:43Z

Hi @chiarazampolli
then I misunderstood you: if you confirm that locally (with fixed QC) the fullCI with QC (with the command line as in #11974 (comment)) works then I merge this PR.

chiarazampolli · 2023-10-05T08:25:44Z

Yes, it does.

cheers,

chiara

chiarazampolli requested a review from shahor02 as a code owner September 27, 2023 16:13

chiarazampolli mentioned this pull request Sep 27, 2023

Adding plots to ITSTPC matching QC AliceO2Group/QualityControl#2001

Merged

chiarazampolli changed the title ~~Adding plots to ITSTPC matching QC~~ [WIP] Adding plots to ITSTPC matching QC Sep 28, 2023

chiarazampolli force-pushed the updateITSTPCQC branch from 7f8f6f1 to 518c920 Compare September 28, 2023 15:46

chiarazampolli changed the title ~~[WIP] Adding plots to ITSTPC matching QC~~ Adding plots to ITSTPC matching QC Sep 28, 2023

chiarazampolli force-pushed the updateITSTPCQC branch 4 times, most recently from c3d8cda to c9df921 Compare September 29, 2023 11:44

chiarazampolli force-pushed the updateITSTPCQC branch 2 times, most recently from e45263a to 9c079a2 Compare September 29, 2023 14:01

chiarazampolli force-pushed the updateITSTPCQC branch from 9c079a2 to a31ff55 Compare October 2, 2023 10:28

Adding plots to ITSTPC matching QC

1c14f70

chiarazampolli force-pushed the updateITSTPCQC branch from a31ff55 to 1c14f70 Compare October 2, 2023 12:26

chiarazampolli mentioned this pull request Oct 4, 2023

Updating GLO json, following PR O2-11974, QC-2001 AliceO2Group/O2DPG#1257

Merged

shahor02 approved these changes Oct 5, 2023

View reviewed changes

shahor02 merged commit 7c20c6b into AliceO2Group:dev Oct 5, 2023
13 of 15 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding plots to ITSTPC matching QC #11974

Adding plots to ITSTPC matching QC #11974

chiarazampolli commented Sep 27, 2023 •

edited

Loading

alibuild commented Sep 27, 2023

chiarazampolli commented Sep 28, 2023

chiarazampolli commented Sep 28, 2023

shahor02 commented Sep 28, 2023

chiarazampolli commented Sep 28, 2023

chiarazampolli commented Sep 28, 2023

shahor02 commented Sep 28, 2023

chiarazampolli commented Sep 29, 2023

alibuild commented Sep 29, 2023

alibuild commented Sep 29, 2023

alibuild commented Sep 29, 2023 •

edited

Loading

chiarazampolli commented Sep 29, 2023

shahor02 commented Sep 29, 2023

alibuild commented Sep 29, 2023 •

edited

Loading

chiarazampolli commented Oct 1, 2023

shahor02 commented Oct 1, 2023

chiarazampolli commented Oct 2, 2023

alibuild commented Oct 2, 2023 •

edited

Loading

alibuild commented Oct 2, 2023 •

edited

Loading

chiarazampolli commented Oct 2, 2023

martenole commented Oct 3, 2023

chiarazampolli commented Oct 3, 2023

shahor02 commented Oct 3, 2023

chiarazampolli commented Oct 3, 2023

chiarazampolli commented Oct 3, 2023

chiarazampolli commented Oct 3, 2023

chiarazampolli commented Oct 4, 2023 •

edited

Loading

shahor02 commented Oct 4, 2023

chiarazampolli commented Oct 4, 2023 •

edited

Loading

shahor02 commented Oct 4, 2023

chiarazampolli commented Oct 4, 2023

chiarazampolli commented Oct 5, 2023

shahor02 commented Oct 5, 2023

chiarazampolli commented Oct 5, 2023

Adding plots to ITSTPC matching QC #11974

Adding plots to ITSTPC matching QC #11974

Conversation

chiarazampolli commented Sep 27, 2023 • edited Loading

alibuild commented Sep 27, 2023

chiarazampolli commented Sep 28, 2023

chiarazampolli commented Sep 28, 2023

shahor02 commented Sep 28, 2023

chiarazampolli commented Sep 28, 2023

chiarazampolli commented Sep 28, 2023

shahor02 commented Sep 28, 2023

chiarazampolli commented Sep 29, 2023

alibuild commented Sep 29, 2023

alibuild commented Sep 29, 2023

alibuild commented Sep 29, 2023 • edited Loading

chiarazampolli commented Sep 29, 2023

shahor02 commented Sep 29, 2023

alibuild commented Sep 29, 2023 • edited Loading

chiarazampolli commented Oct 1, 2023

shahor02 commented Oct 1, 2023

chiarazampolli commented Oct 2, 2023

alibuild commented Oct 2, 2023 • edited Loading

alibuild commented Oct 2, 2023 • edited Loading

chiarazampolli commented Oct 2, 2023

martenole commented Oct 3, 2023

chiarazampolli commented Oct 3, 2023

shahor02 commented Oct 3, 2023

chiarazampolli commented Oct 3, 2023

chiarazampolli commented Oct 3, 2023

chiarazampolli commented Oct 3, 2023

chiarazampolli commented Oct 4, 2023 • edited Loading

shahor02 commented Oct 4, 2023

chiarazampolli commented Oct 4, 2023 • edited Loading

shahor02 commented Oct 4, 2023

chiarazampolli commented Oct 4, 2023

chiarazampolli commented Oct 5, 2023

shahor02 commented Oct 5, 2023

chiarazampolli commented Oct 5, 2023

chiarazampolli commented Sep 27, 2023 •

edited

Loading

alibuild commented Sep 29, 2023 •

edited

Loading

alibuild commented Sep 29, 2023 •

edited

Loading

alibuild commented Oct 2, 2023 •

edited

Loading

alibuild commented Oct 2, 2023 •

edited

Loading

chiarazampolli commented Oct 4, 2023 •

edited

Loading

chiarazampolli commented Oct 4, 2023 •

edited

Loading