Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added bucket index flag, and enable bucket index by default. #254

Merged
merged 5 commits into from
Jan 20, 2021
Merged

Added bucket index flag, and enable bucket index by default. #254

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5b02d5d
Added option to enable bucket index.
pstibrany Jan 20, 2021
177cb75
Enable bucket index by default.
pstibrany Jan 20, 2021
872314e
CHANGELOG.md
pstibrany Jan 20, 2021
42c2d25
Update comments.
pstibrany Jan 20, 2021
6b08542
Change default.
pstibrany Jan 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- Cortex / Queries: added bucket index load operations and latency (available only when bucket index is enabled)
- Alerts: added "CortexBucketIndexNotUpdated" (bucket index only) and "CortexTenantHasPartialBlocks"
* [ENHANCEMENT] The name of the overrides configmap is now customisable via `$._config.overrides_configmap`. #244
* [ENHANCEMENT] Added flag to control usage of bucket-index, and enable it by default when using blocks. #254
* [BUGFIX] Honor configured `per_instance_label` in all panels. #239
* [BUGFIX] `CortexRequestLatency` alert now ignores long-running requests on query-scheduler. #242
* [BUGFIX] Honor configured `job_names` in the "Memory (go heap inuse)" panel. #247
Expand Down
23 changes: 20 additions & 3 deletions cortex/tsdb.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@

// Allow to fine tune compactor.
cortex_compactor_max_concurrency: 1,
// While this is the default value, we want to pass the same to the -blocks-storage.bucket-store.sync-interval
cortex_compactor_cleanup_interval: '15m',

// Enable use of bucket index by querier, ruler and store-gateway.
// Bucket index is generated by compactor from Cortex 1.7, there is no flag required to enable this on compactor.
cortex_bucket_index_enabled: true,
pstibrany marked this conversation as resolved.
Show resolved Hide resolved
},

blocks_chunks_caching_config::
Expand Down Expand Up @@ -59,8 +65,15 @@
'blocks-storage.bucket-store.metadata-cache.memcached.max-get-multi-batch-size': '100',
} else {},

querier_args+:: $.blocks_metadata_caching_config,
ruler_args+:: $.blocks_metadata_caching_config,
bucket_index_config:: if $._config.cortex_bucket_index_enabled then {
'blocks-storage.bucket-store.bucket-index.enabled': true,

// Bucket index is updated by compactor on each cleanup cycle.
'blocks-storage.bucket-store.sync-interval': $._config.cortex_compactor_cleanup_interval,
} else {},

querier_args+:: $.blocks_metadata_caching_config + $.bucket_index_config,
ruler_args+:: $.blocks_metadata_caching_config + $.bucket_index_config,

// The ingesters should persist TSDB blocks and WAL on a persistent
// volume in order to be crash resilient.
Expand Down Expand Up @@ -134,6 +147,7 @@
'compactor.data-dir': '/data',
'compactor.compaction-interval': '30m',
'compactor.compaction-concurrency': $._config.cortex_compactor_max_concurrency,
'compactor.cleanup-interval': $._config.cortex_compactor_cleanup_interval,

// Enable sharding.
'compactor.sharding-enabled': true,
Expand Down Expand Up @@ -189,7 +203,10 @@
// Persist ring tokens so that when the store-gateway will be restarted
// it will pick the same tokens
'store-gateway.sharding-ring.tokens-file-path': '/data/tokens',
} + $.blocks_chunks_caching_config + $.blocks_metadata_caching_config,
} +
$.blocks_chunks_caching_config +
$.blocks_metadata_caching_config +
$.bucket_index_config,

store_gateway_ports:: $.util.defaultPorts,

Expand Down