Action hangs indefinitely if oidc-beacon malfunctions

[jku]

At some point after Dec 28 the sigstore-conformance action runs have started to not complete.

This is visible in both sigstore-python and sigstore-java PR workflow runs but will also happen in the selftest workflow here (I tried by rerunning a workflow that succeeded two weeks ago: it now hangs).

I have not identified a reason for this. I've looked at logs and the GitHub environment seems unchanged, as does Python. The only change I can see is that installing requirements.txt now leads to installing attrs 23.2.0 instead of 23.1.0 -- but pinning the older version does not change the situation so seems unrelated.

sigstore/sigstore-python#857

EDIT: this is because sigstore-conformance/extremely-dangerous-public-oidc-beacon was malfunctioning. Let's try to modify the action so that it gives some indication of what is going wrong in this case

[jku]

strace says that after collecting the tests (timestamp 2024-01-03T13:19:03.4878640Z in the raw log)

• pytest creates temporary copies of the assets (this finishes around 2024-01-03T13:19:03.5330507Z).
• then it ends up in a loop where every 30 secs it reads some networking files (like reads ca certs) and makes a connection to a github IP... then starts over
• I think it never starts an actual test

🤷

https://github.com/jku/sigstore-conformance/actions/runs/7397783000/job/20125543111 (this is slightly modified in addition to strace: only a single test gets collected here)

[jku]

then it ends up in a loop

My guess: this is the while loop in identity_token() in test/conftest.py

[steiza]

This was probably caused by the OIDC beacon job which stopped running 5 days ago: https://github.com/sigstore-conformance/extremely-dangerous-public-oidc-beacon/actions

[jku]

oh yeah, that makes sense. I've enabled the trigger action again.

Let's leave this open so we can add some sort of error that allows root-causing this failure

[woodruffw]

The "scheduled actions stop running after 60 days of inactivity" feature strikes again 🙂

[woodruffw]

Oh, maybe spoke too soon. I don't see the normal warning that appears when a scheduled workflow is disabled.

[jku]

I don't see the normal warning that appears when a scheduled workflow is disabled.

I re-enabled it 7 minutes ago.

[woodruffw]

For reference, tracker from when this last happened: sigstore-conformance/extremely-dangerous-public-oidc-beacon#4