Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

creds from env, 1sec wait_timeout #6

Merged
merged 2 commits into from
Jun 21, 2023
Merged

creds from env, 1sec wait_timeout #6

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
747bd46
creds from env, 1sec wait_timeout
slorello89 Jun 16, 2023
3ac04a8
Update README.md
slorello89 Jun 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,17 +93,21 @@ tritonserver --cache-config redis,host=redis-host --cache-config redis,port=6379

| Configuration Option | Required | Description | Default |
|----------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------|---------|
| host | Yes | The hostname or IP address of the server where Redis is running. | N/A |
| port | Yes | The port number to connect to on the server. | N/A |
| host | Yes | The hostname or IP address of the server where Redis is running. | N/A |
| port | Yes | The port number to connect to on the server. | N/A |
| user | No | The username to use for authentication of the ACLs to the Redis Server | default |
| password | No | The password to Redis. | N/A |
| db | No | The db number to user. NOTE - use of the db number is considered an anti-pattern in Redis, so it is advised that you do not use this option | 0 |
| connect_timeout | No | The maximum time, in milliseconds to wait for a connection to be established to Redis. 0 means wait forever | 0 |
| socket_timeout | No | The maximum time, in milliseconds the client will wait for a response from Redis. 0 means wait forever | 0 |
| pool_size | No | The number pooled connections to Redis the client will maintain. | 1 |
| wait_timeout | No | The maximum time, in milliseconds to wait for a connection from the pool. | 100 |
| password | No | The password to Redis. | N/A |
| db | No | The db number to user. NOTE - use of the db number is considered an anti-pattern in Redis, so it is advised that you do not use this option | 0 |
| connect_timeout | No | The maximum time, in milliseconds to wait for a connection to be established to Redis. 0 means wait forever | 0 |
| socket_timeout | No | The maximum time, in milliseconds the client will wait for a response from Redis. 0 means wait forever | 0 |
| pool_size | No | The number pooled connections to Redis the client will maintain. | 1 |
| wait_timeout | No | The maximum time, in milliseconds to wait for a connection from the pool. | 1000 |


### Optional Environment Variables for Credentials

Optionally you may configure your `user`/`password` via environment variables. The corresponding `user` environment variable is `TRITONCACHE_REDIS_USERNAME` whereas the corresponding password environment variable is `TRITONCACHE_REDIS_PASSWORD`.
slorello89 marked this conversation as resolved.
Show resolved Hide resolved

## Monitoring and Observability

There are many ways to go about monitoring what's going on in Redis. One popular mode is to export metrics data from Redis to Prometheus, and use Grafana to observe them.
Expand Down
16 changes: 15 additions & 1 deletion src/redis_cache.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,15 @@ setOption(
}
}

void
setOptionFromEnv(const char* envVarName, std::string& option)
{
const char* envVarValue = std::getenv(envVarName);
if (envVarValue != nullptr) {
option = envVarValue;
}
}

TRITONSERVER_Error*
handleError(
const std::string& message, const std::string& key,
Expand Down Expand Up @@ -118,6 +127,11 @@ RedisCache::Create(
sw::redis::ConnectionOptions options;
sw::redis::ConnectionPoolOptions poolOptions;

// try pulling user/password from environment fist
// override if present in the config
setOptionFromEnv(USERNAME_ENV_VAR_NAME, options.user);
slorello89 marked this conversation as resolved.
Show resolved Hide resolved
setOptionFromEnv(PASSWORD_ENV_VAR_NAME, options.password);

setOption("host", options.host, document);
setOption("port", options.port, document);
setOption("user", options.user, document);
Expand All @@ -128,7 +142,7 @@ RedisCache::Create(
setOption("pool_size", poolOptions.size, document);
setOption("wait_timeout", poolOptions.wait_timeout, document);
if (!document.HasMember("wait_timeout")) {
poolOptions.wait_timeout = std::chrono::milliseconds(100);
poolOptions.wait_timeout = std::chrono::milliseconds(1000);
}

try {
Expand Down
4 changes: 4 additions & 0 deletions src/redis_cache.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ struct CacheEntry {
// the buffer back to Triton
constexpr uint32_t FIELDS_PER_BUFFER = 4;

constexpr const char* PASSWORD_ENV_VAR_NAME = "TRITONCACHE_REDIS_PASSWORD";
constexpr const char* USERNAME_ENV_VAR_NAME = "TRITONCACHE_REDIS_USERNAME";


#define RETURN_IF_ERROR(X) \
do { \
TRITONSERVER_Error* err__(X); \
Expand Down