Index conflicts generated by logs

[pierluigilenoci]

The pod produces the payload.code field with two different types of data:

String:

"payload": {
      "message": "Secrets Manager can't find the specified secret value for staging label: AWSCURRENT",
      "code": "ResourceNotFoundException",
      "time": "2021-07-31T21:59:32.112Z",
      "requestId": "3bb8bc86-709e-444d-a723-40e4efd25af8",
      "statusCode": 400,
      "retryable": false,
      "retryDelay": 6.982864466638317
    },

Long:

"payload": {
      "code": 422,
      "statusCode": 422
    },

If this log is sent to an ES cluster it creates an index conflict.

Can you fix this behavior?

[pierluigilenoci]

@Flydiverny could you please take a look?

[pierluigilenoci]

@stephenthedev @Flydiverny could you please take a look?

[Flydiverny]

Do you have any related error message for when the two occur and/or log level for them? Not sure where the second variant originates from:

"payload": {
      "code": 422,
      "statusCode": 422
    },

Are both with the message failure while polling the secret <namesapce>/<es-name>?

[pierluigilenoci]

At this moment I have no 422 errors present but 500 errors, I hope it goes well anyway.

{"level":"error","message_time":"2021-07-23T23:00:06.151Z","pid":18,"hostname":"kubernetes-external-secrets-5b5667b5c4-m2xdp","payload":{"code":500,"statusCode":500},"msg":"failure while polling the secret [REDACTED]/[REDACTED]"}

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[pierluigilenoci]

@vladlosev @adutchak-x @davesteinberg @Flydiverny could you please take a look?

[vladlosev]

I am not a project maintainer so take my words with a grain of salt but it looks like it will be possible to do what you need. There are six occurrences of error logging in the code:

MacBook-Pro-4:kubernetes-external-secrets vlad$ grep -rnI --exclude='*/node_modules/*' '[.]error(' .
./bin/daemon.js:46:    logger.error('CRD installation check failed, statusCode: %s', err.statusCode)
./lib/backends/secrets-manager-backend.js:70:    this._logger.error(`Unexpected data from Secrets Manager secret ${key}`)
./lib/poller.js:138:      this._logger.error(err, `failure while polling the secret ${this._namespace}/${this._name}`)
./lib/poller.js:237:        this._logger.error(err, `failure while updating status for externalsecret ${this._namespace}/${this._name}`)
./lib/poller.js:388:      this._logger.error(err, `status check went boom for ${this._namespace}/${this._name}`)
./lib/external-secret.js:88:    logger.error(err, 'Watcher for namespace %s crashed', loggedNamespaceName)
MacBook-Pro-4:kubernetes-external-secrets vlad$

I should be possible to amend them all to stringify err.code if it's present. Do you want to submit a PR?

Of course, it's also possible to work around the issue by modifying your Logstash filters to do the same for pasyload.code, with the filter looking something like this:

    filter {
      mutate {
        convert => {
          "[response][code]" => "string"
        }
      }
    }

[pierluigilenoci]

@vladlosev before investing time in a PR i would like to know from @Flydiverny if it makes sense to do it for this.

[Flydiverny]

I'll happily review a PR when time permits but I don't think I'll have time to dig into this myself.
Please also see #864

[pierluigilenoci]

@Flydiverny I tagged you for exactly this reason.
Does it make sense to make these changes when the software has been "deprecated" anyway?

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[pierluigilenoci]

@Flydiverny could you please take a look?

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[pierluigilenoci]

@Flydiverny obviously this is deprecated and therefore it makes no sense to make changes.
But how is the new software from this point of view?

[Flydiverny]

@Flydiverny obviously this is deprecated and therefore it makes no sense to make changes. But how is the new software from this point of view?

I actually don't know, in the end I haven't been much involved in ESO after the initial CRD discussions :)