unable to return specific versions of a secret path in kv v2 in template, seems to only return latest version #1350
Comments
caleyg
changed the title
|
Hey @caleyg, I think I've figured out what is going on here. Consul-template caches the results it gets (from vault or consul) so that it doesn't have to re-fetch all of them each time one of them changes (when it needs to re-render the template). The problem is that the cache is keyed off the path of the secret, without the version. So the cache gets initialized with whatever version the first request is for and the rest then just return that value. The fix here is obviously to include the version info in the key for the cache. I'll have to see if it is that simple in practice though. I'm looking into it. |
|
Thank you so much for investigating and your time @eikenb |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Consul Template version
consul-template v0.24.1 (58aa6c6)
Vault version
vault 0.10.4
Configuration
{"consul": {"ssl": [{"ca_cert": "/etc/ssl/certs/ca-certificates.crt"}, {"verify": true}]}, "vault": [{"ssl": [{"ca_cert": "/etc/ssl/certs/ca-certificates.crt"}, {"verify": true}]}, {"renew_token": true}], "exec": {"env": [{"custom": "PATH=$PATH:$PWD"}]}, "template": [{"source": "/etc/some_other_file.yml.ctmpl"}, {"destination": "/etc/some_other_file.yml"}, {"error_on_missing_key": true}, {"command_timeout": "300s"}], "log_level": "debug"} {"consul": {"ssl": [{"ca_cert": "/etc/ssl/certs/ca-certificates.crt"}, {"verify": true}]}, "vault": [{"ssl": [{"ca_cert": "/etc/ssl/certs/ca-certificates.crt"}, {"verify": true}]}, {"renew_token": true}], "exec": {"env": [{"custom": "PATH=$PATH:$PWD"}]}, "template": [{"source": "/app/index.html.ctmpl"}, {"destination": "/app/index.html"}, {"error_on_missing_key": true}, {"command_timeout": "300s"}], "log_level": "debug"}Raw Template
{{ $service_name := `foo` }} {{ $stack_name := `bar` }} {{ $vault_path := ( print "secret_v2/" $stack_name "/" $service_name ) }} secret4_latest: {{ with secret ( print $vault_path "/secret4" ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret4_v1: {{ with secret ( print $vault_path "/secret4?version=" 1 ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret4_v2: {{ with secret ( print $vault_path "/secret4?version=" 2 ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret4_v3: {{ with secret ( print $vault_path "/secret4?version=" 3 ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret4_v4: {{ with secret ( print $vault_path "/secret4?version=" 4 ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret5_latest: {{ with secret ( print $vault_path "/secret5" ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret5_v1: {{ with secret ( print $vault_path "/secret5" "?version=1" ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret5_v2: {{ with secret ( print $vault_path "/secret5" "?version=2" ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret5_v3: {{ with secret ( print $vault_path "/secret5" "?version=3" ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }} secret5_v4: {{ with secret ( print $vault_path "/secret5" "?version=4" ) }} {{ range $k, $v := .Data.data }} {{ $v }} {{ end }}{{ end }}Rendered template
Actual versioned vault secrets
// latest version of secrets (zero) curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret5 {"request_id":"e6d6e182-5c0e-11ea-8b5d-6771cd7e39d5","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret5key":"somesecret5value_version3"},"metadata":{"created_time":"2020-03-01T21:30:18.255164372Z","deletion_time":"","destroyed":false,"version":4}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret4 {"request_id":"85d3a420-74ac-41de-357b-ea4a6306a690","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret4key":"somesecret4value_version3"},"metadata":{"created_time":"2020-03-01T21:24:30.534224223Z","deletion_time":"","destroyed":false,"version":4}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret5?version=0 {"request_id":"0de681f1-eb2f-5b8f-cc93-ee19ee2ba246","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret5key":"somesecret5value_version3"},"metadata":{"created_time":"2020-03-01T21:30:18.255164372Z","deletion_time":"","destroyed":false,"version":4}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret4?version=0 {"request_id":"f7a6afc4-5c0e-11ea-9664-a7309391da25","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret4key":"somesecret4value_version3"},"metadata":{"created_time":"2020-03-01T21:24:30.534224223Z","deletion_time":"","destroyed":false,"version":4}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret4?version=0 {"request_id":"25392fde-45b5-27b7-37fa-f32fbb137771","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret4key":"somesecret4value_version3"},"metadata":{"created_time":"2020-03-01T21:24:30.534224223Z","deletion_time":"","destroyed":false,"version":4}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret4?version=1 {"request_id":"57e6aedc-9406-07a4-e7bc-8fd396f8f823","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret4key":"somesecret4value_version1"},"metadata":{"created_time":"2020-03-01T21:23:50.76118051Z","deletion_time":"","destroyed":false,"version":1}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret4?version=3 {"request_id":"2994b83c-d742-807a-aa3e-0d5d88ba6f60","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret4key":"somesecret4value_version2"},"metadata":{"created_time":"2020-03-01T21:24:17.868740798Z","deletion_time":"","destroyed":false,"version":3}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret4?version=4 {"request_id":"8ef9e5d9-d724-e650-03a8-8477a17f15c4","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret4key":"somesecret4value_version3"},"metadata":{"created_time":"2020-03-01T21:24:30.534224223Z","deletion_time":"","destroyed":false,"version":4}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret5?version=0 {"request_id":"36673da2-1b0d-510e-1333-36a088637dda","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret5key":"somesecret5value_version3"},"metadata":{"created_time":"2020-03-01T21:30:18.255164372Z","deletion_time":"","destroyed":false,"version":4}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret5?version=1 {"request_id":"5b014412-22be-b8f6-23eb-f245b8e6ab2e","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret5key":"somesecret5value_version1"},"metadata":{"created_time":"2020-03-01T21:28:11.184234432Z","deletion_time":"","destroyed":false,"version":1}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/secret5?version=2 {"request_id":"8fe0e3f7-3918-57a4-5a72-1e5293a7d418","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret5key":"somesecret5value_version1"},"metadata":{"created_time":"2020-03-01T21:29:33.398767611Z","deletion_time":"","destroyed":false,"version":2}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret5?version=3 {"request_id":"6ba62ce5-e788-4c9d-85f2-2f13341979f9","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret5key":"somesecret5value_version2"},"metadata":{"created_time":"2020-03-01T21:29:51.385471519Z","deletion_time":"","destroyed":false,"version":3}},"wrap_info":null,"warnings":null,"auth":null} curl --header "X-Vault-Token: *** " https://vault/v1/secret_v2/data/secret5?version=4 {"request_id":"7e675c49-4a00-c268-ca91-631061a8e763","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"somesecret5key":"somesecret5value_version3"},"metadata":{"created_time":"2020-03-01T21:30:18.255164372Z","deletion_time":"","destroyed":false,"version":4}},"wrap_info":null,"warnings":null,"auth":null}Vault backends
Command
Debug output
Expected behavior
I expect to be able to target a specific version of a secret at any point in my template for the same secret path in the same template and return that value.
Actual behavior
Instead it seems the template gets pinned on a specific version of a secret. I have tried this in multiple templates using different versions, and it always version 0 that seems to get returned...
Steps to reproduce
The text was updated successfully, but these errors were encountered: