Container image verification failed

[iamantil]

When verifying the signed container image with notation I am getting the error

notation verify $IMAGE
Warning: Always verify the artifact using digest(@sha256:...) rather than a tag(:v1) because resolved digest may not point to the same signed artifact, as tags are mutable.
Error: signature verification failed for all the signatures associated with $registry/$REPO@sha256:value

I am able to sign the container image successfully, I have also added the root certificate into the notation trusted store but still getting the error.

Any guidance would be appreciated.

[yizha1]

Thanks @iamantil for reporting the issue. I transferred your issue to the notation repo as it is related to Notation CLI.

Could you try to use the additional flag -v for notation verify command which will generate verbose logs.

[iamantil]

Here is the output

notation verify $IMAGE -v
USFO UsUSg the referrers tag schema
USFO Reference latest resolved to manifest descriptor: {MediaType:application/vnd.docker.distribution.manifest.v2+json Digest:sha256:5f3e23e8c86adb32b3b9de979d841f0f261a23a036937d31c41ad69aaa02d764 Size:1779 URLs:[] Annotations:map[] Data:[] Platform: ArtifactType:}
WarnUSg: Always verify the artifact usUSg digest(@sha256:...) rather than a tag(:latest) because resolved digest may not poUSt to the same signed artifact, as tags are mutable.
USFO CheckUSg whether signature verification should be skipped or not
USFO Trust policy configuration: &{Name:$STORE_NAME RegistryScopes:[contaUSerimagesignUSg.azurecr.io/code-signed-images] SignatureVerification:{VerificationLevel:strict Override:map[]} TrustStores:[ca:$STORE_NAME] TrustedIdentities:[x509.subject: CN=$CN,O=$O $STORE_NAME,L=SLN,ST=LN,C=$C]}
USFO Check over. Trust policy is not configured to skip signature verification
USFO ProcessUSg signature with manifest mediaType: application/vnd.oci.image.manifest.v1+json and digest: sha256:01812a5583335a842846b0f5599012f9f040f3eae7dfea238bddbf8dcd4f4002
USFO Trust policy configuration: &{Name:$STORE_NAME RegistryScopes:[contaUSerimagesignUSg.azurecr.io/code-signed-images] SignatureVerification:{VerificationLevel:strict Override:map[]} TrustStores:[ca:$STORE_NAME] TrustedIdentities:[x509.subject: CN=$CN,O=$O $STORE_NAME,L=SLN,ST=LN,C=$C]}
ERRO authenticity validation failed. Failure reason: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
WARN Signature sha256:01812a5583335a842846b0f5599012f9f040f3eae7dfea238bddbf8dcd4f4002 failed verification with error: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
USFO ProcessUSg signature with manifest mediaType: application/vnd.oci.image.manifest.v1+json and digest: sha256:79f8c9f883b1c9dcb5d52a1b06e06c808729ff9abd2fa0547f17bce74bfd03f7
USFO Trust policy configuration: &{Name:$STORE_NAME RegistryScopes:[contaUSerimagesignUSg.azurecr.io/code-signed-images] SignatureVerification:{VerificationLevel:strict Override:map[]} TrustStores:[ca:$STORE_NAME] TrustedIdentities:[x509.subject: CN=$CN,O=$O $STORE_NAME,L=SLN,ST=LN,C=$C]}
ERRO authenticity validation failed. Failure reason: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
WARN Signature sha256:79f8c9f883b1c9dcb5d52a1b06e06c808729ff9abd2fa0547f17bce74bfd03f7 failed verification with error: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
USFO ProcessUSg signature with manifest mediaType: application/vnd.oci.image.manifest.v1+json and digest: sha256:4331fa251d07dc00f9316e3b42bf8db33db2aa57f654893bd9a659e2735d4d37
USFO Trust policy configuration: &{Name:$STORE_NAME RegistryScopes:[contaUSerimagesignUSg.azurecr.io/code-signed-images] SignatureVerification:{VerificationLevel:strict Override:map[]} TrustStores:[ca:$STORE_NAME] TrustedIdentities:[x509.subject: CN=$CN,O=$O $STORE_NAME,L=SLN,ST=LN,C=$C]}
ERRO authenticity validation failed. Failure reason: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
WARN Signature sha256:4331fa251d07dc00f9316e3b42bf8db33db2aa57f654893bd9a659e2735d4d37 failed verification with error: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
USFO ProcessUSg signature with manifest mediaType: application/vnd.oci.image.manifest.v1+json and digest: sha256:7ae0827aba5dc0bdc478a0570bf87d24afee32003140a4dbefb81392b5439d03
USFO Trust policy configuration: &{Name:$STORE_NAME RegistryScopes:[contaUSerimagesignUSg.azurecr.io/code-signed-images] SignatureVerification:{VerificationLevel:strict Override:map[]} TrustStores:[ca:$STORE_NAME] TrustedIdentities:[x509.subject: CN=$CN,O=$O $STORE_NAME,L=SLN,ST=LN,C=$C]}
ERRO authenticity validation failed. Failure reason: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
WARN Signature sha256:7ae0827aba5dc0bdc478a0570bf87d24afee32003140a4dbefb81392b5439d03 failed verification with error: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
USFO ProcessUSg signature with manifest mediaType: application/vnd.oci.image.manifest.v1+json and digest: sha256:a7e8c180af1a6a031f1df071b49d394ca3eca91fbb814a8862514a5af327f3c4
USFO Trust policy configuration: &{Name:$STORE_NAME RegistryScopes:[contaUSerimagesignUSg.azurecr.io/code-signed-images] SignatureVerification:{VerificationLevel:strict Override:map[]} TrustStores:[ca:$STORE_NAME] TrustedIdentities:[x509.subject: CN=$CN,O=$O $STORE_NAME,L=SLN,ST=LN,C=$C]}
ERRO authenticity validation failed. Failure reason: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
WARN Signature sha256:a7e8c180af1a6a031f1df071b49d394ca3eca91fbb814a8862514a5af327f3c4 failed verification with error: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
USFO ProcessUSg signature with manifest mediaType: application/vnd.oci.image.manifest.v1+json and digest: sha256:696a2d74d1452dd84e3f531408eec12256d0645334ef4dbee266af3a1974b5b1
USFO Trust policy configuration: &{Name:$STORE_NAME RegistryScopes:[contaUSerimagesignUSg.azurecr.io/code-signed-images] SignatureVerification:{VerificationLevel:strict Override:map[]} TrustStores:[ca:$STORE_NAME] TrustedIdentities:[x509.subject: CN=$CN,O=$O $STORE_NAME,L=SLN,ST=LN,C=$C]}
ERRO authenticity validation failed. Failure reason: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
WARN Signature sha256:696a2d74d1452dd84e3f531408eec12256d0645334ef4dbee266af3a1974b5b1 failed verification with error: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
USFO ProcessUSg signature with manifest mediaType: application/vnd.oci.image.manifest.v1+json and digest: sha256:2eb10d483b86ba8c7d7b2e801c41b4b4c1cd35e502758e90c592e84f51819cf4
USFO Trust policy configuration: &{Name:$STORE_NAME RegistryScopes:[contaUSerimagesignUSg.azurecr.io/code-signed-images] SignatureVerification:{VerificationLevel:strict Override:map[]} TrustStores:[ca:$STORE_NAME] TrustedIdentities:[x509.subject: CN=$CN,O=$O $STORE_NAME,L=SLN,ST=LN,C=$C]}
ERRO authenticity validation failed. Failure reason: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
WARN Signature sha256:2eb10d483b86ba8c7d7b2e801c41b4b4c1cd35e502758e90c592e84f51819cf4 failed verification with error: error while parsUSg the certificate subject from the digital signature. error : "unsupported distUSguished name (DN) "CN=AzureContaUSerImageSignUSg,OU=CPMO,O=AzureContaUSerImageSignUSg,L=LN,ST=LN,C=US,1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d": notation does not support x509.subject identities contaUSUSg "=#""
Error: signature verification failed for all the signatures associated with contaUSerimagesignUSg.azurecr.io/code-signed-images@sha256:5f3e23e8c86adb32b3b9de979d841f0f261a23a036937d31c41ad69aaa02d764

[yizha1]

@iamantil Hello, currently Notation does not support the sign # after =. Would you mind sharing your scenario on how your subject DN is generated, especially this string: 1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d?

There are two options to mitigate this issue:

• Option 1: update 1.2.840.113549.1.9.1=#0c1d7072617368616e742e616e74696c40676c6f62616c7369676e2e636f6d to the format that Notation supports.
• Option 2: For testing purposes, you can use * for trustedIdentities, but this is not secure only for testing purposes.

Addtionally,
If you're using the notation policy import command to import the policy before running notation verify, and the policy JSON file includes =#, it should result in a failure. This leads me to believe you might have manually copied the policy file into the notation config directory. To avoid issues at the verification stage, I recommend using the notation policy import command, which can help identify problems earlier.