multi-DB Part 1: create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json #2182

dzhangalibaba · 2018-10-23T00:46:56Z

this is the first step to moving different databases tables into different database instances
in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json
we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file.
if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today.
The configuration is as below, later we can add more db related information if needed:
{
...
"DATABASE": {
"redis-db-01" : {
"port" : "6380",
"database": ["APPL_DB", "STATE_DB"]
},
"redis-db-02" : {
"port" : "6381",
"database":["ASIC_DB"]
},
}
...
}
The detail description is at design doc at Design of creating user defined multiple database instances SONiC#271
The main idea is : when database.sh started, we check the configuration and generate corresponding scripts.
rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local
Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well.
based on those conf, we create databases instances as required.
at last, we ping_pong check database are up and continue

 Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com

jipanyang · 2018-10-23T01:07:26Z

Once the solution is agreed upon, "Configuration DB schema" section of https://github.com/Azure/sonic-swss/blob/master/doc/swss-schema.md should be updated with redis db instance schema.

qiluo-msft · 2018-10-23T02:07:32Z

May I ask a well formatted design doc before reviewing the implementation? We don't want to waste your time on implementation when we still have concerns on the design. #Closed

dzhangalibaba · 2018-10-23T17:41:42Z

sure. I'll prepare some slides to introduce the ideas. We can share it on the weekly meeting.

May I ask a well formatted design doc before reviewing the implementation? We don't want to waste your time on implementation when we still have concerns on the design.

#Resolved

dzhangalibaba · 2018-10-28T06:15:20Z

The Design DOC is here

sonic-net/SONiC#271

sure. I'll prepare some slides to introduce the ideas. We can share it on the weekly meeting.

May I ask a well formatted design doc before reviewing the implementation? We don't want to waste your time on implementation when we still have concerns on the design.

#Resolved

qiluo-msft · 2018-12-06T23:04:28Z

dockers/docker-database/Dockerfile.j2

@@ -35,6 +35,15 @@ RUN sed -ri 's/^(save .*$)/# \1/g;                       \
             s/^client-output-buffer-limit pubsub [0-9]+mb [0-9]+mb [0-9]+/client-output-buffer-limit pubsub 0 0 0/ \
            ' /etc/redis/redis.conf

-COPY ["supervisord.conf", "/etc/supervisor/conf.d/"]
-
-ENTRYPOINT ["/usr/bin/supervisord"]


supervisord [](start = 22, length = 11)

supervisord [](start = 22, length = 11)

We prefer keep supervisord as entrypoint. You may refer other docker containers for example with a start.sh. #Closed

if the database_config file is static as you said, then we can use supervisord as Entrypoint, and we can generate supervisord.conf during compiling/building

Keeping supervisord as entrypoint is a good practice. Even with database_config.json, it shall be possilble to generate redis-server.sh from redis-server.sh.j2 within db-docker-init.sh(dockers/docker-database/docker_init.sh), and let redis-server.sh handle the dynamic db instances case?

[program:db-docker-init.sh] command=/usr/bin/db-docker-init.sh priority=2 autostart=true autorestart=false stdout_logfile=syslog stderr_logfile=syslog [program:redis-server.sh] command=/usr/bin/redis-server.sh priority=3 autostart=false autorestart=false stdout_logfile=syslog stderr_logfile=syslog

dockers/docker-database/Dockerfile.j2

dockers/docker-database/docker_init.sh

files/build_templates/docker_image_ctl.j2

files/image_config/database/internal_ping_pong_database.sh.j2

dockers/docker-database/create_all_redis_config.sh.j2

files/build_templates/database.service.j2

qiluo-msft

As comments

dzhangalibaba · 2018-12-07T03:13:42Z

@qiluo-msft Thanks, will take a look #Resolved

nikos-github · 2018-12-13T22:48:04Z

I haven't followed the entire thread but I will read through it. In the meantime a comment on the config part that caught my eye - I feel that the database config should be under /etc/sonic/database or under /etc/sonic/db. Also I would suggest against a rigid/static config db file. #Resolved

qiluo-msft · 2018-12-13T22:53:06Z

From switch operator's point of view, this is not a config and it is an implementation detail and can be static in an image. Since it is a "config" for redis partitioning, it will be weird that we store it in a redis db.

In reply to: 447149304 [](ancestors = 447149304)

dockers/docker-database/create_single_redis_config.sh

dzhangalibaba · 2018-12-20T08:04:56Z

@qiluo-msft re-commit the codes, please have a look when you are available.

qiluo-msft · 2018-12-27T19:42:19Z

@dzhangalibaba Could you keep old commits and add new iteration as incremental commits. The review context are all lost and it is difficult to check your improvement. #Closed

dzhangalibaba · 2019-01-03T19:14:21Z

Sorry Qi, I used git push -force on my local branch and the old commit is overwritten. I thought the comment on this PR is enough and didn't keep the old commit. I'll keep them in future commit. Basically, the main changes is :
1 . database entry point is still supervisord
2. change the SED syntax easily to read and clean
3. make database_config.json static and is known while buidling/compiling
4. then database_config.json will always exist
5. use python to loop on the json file instead of rendering template

The new changes in the commit is more clear to follow. Could you take a look and comment. Thanks.

Best,
Dong

@dzhangalibaba Could you keep old commits and add new iteration as incremental commits. The review context are all lost and it is difficult to check your improvement.

jipanyang · 2019-01-13T23:35:47Z

dockers/docker-database/Dockerfile.j2

@@ -35,6 +35,15 @@ RUN sed -ri 's/^(save .*$)/# \1/g;                       \
             s/^client-output-buffer-limit pubsub [0-9]+mb [0-9]+mb [0-9]+/client-output-buffer-limit pubsub 0 0 0/ \
            ' /etc/redis/redis.conf

-COPY ["supervisord.conf", "/etc/supervisor/conf.d/"]
-
-ENTRYPOINT ["/usr/bin/supervisord"]


Keeping supervisord as entrypoint is a good practice. Even with database_config.json, it shall be possilble to generate redis-server.sh from redis-server.sh.j2 within db-docker-init.sh(dockers/docker-database/docker_init.sh), and let redis-server.sh handle the dynamic db instances case?

[program:db-docker-init.sh] command=/usr/bin/db-docker-init.sh priority=2 autostart=true autorestart=false stdout_logfile=syslog stderr_logfile=syslog [program:redis-server.sh] command=/usr/bin/redis-server.sh priority=3 autostart=false autorestart=false stdout_logfile=syslog stderr_logfile=syslog

files/image_config/database/database_config.json

dzhangalibaba · 2019-02-14T06:33:54Z

redesign and recode this PR after the discussion with SONiC team

dockers/docker-database/Dockerfile.j2

dockers/docker-database/start.sh

dockers/docker-database/supervisord_database.conf.j2

dockers/docker-database/create_single_redis_config.sh

files/build_templates/docker_image_ctl.j2

files/build_templates/sonic_debian_extension.j2

files/build_templates/docker_image_ctl.j2

…ined config

lguohan · 2019-08-15T09:18:42Z

thanks for the work. only one comment is left.

addressed review

…tabase-init.sh

dzhangalibaba · 2019-08-16T01:27:07Z

thanks for the work. only one comment is left.

modified as required

lguohan · 2019-08-28T15:51:17Z

retest this please

lguohan · 2019-08-28T16:24:30Z

retest this please

…base_config.json (sonic-net#2182) this is the first step to moving different databases tables into different database instances in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file. if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today. The configuration is as below, later we can add more db related information if needed: { ... "DATABASE": { "redis-db-01" : { "port" : "6380", "database": ["APPL_DB", "STATE_DB"] }, "redis-db-02" : { "port" : "6381", "database":["ASIC_DB"] }, } ... } The detail description is at design doc at sonic-net/SONiC#271 The main idea is : when database.sh started, we check the configuration and generate corresponding scripts. rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well. based on those conf, we create databases instances as required. at last, we ping_pong check database are up and continue  Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com

dzhangalibaba mentioned this pull request Oct 24, 2018

Design of creating user defined multiple database instances sonic-net/SONiC#271

Merged

stcheng added the Enhancement ➕ label Oct 24, 2018

dzhangalibaba force-pushed the createDBinst branch from 6c1c048 to 66f8945 Compare December 5, 2018 03:09

dzhangalibaba changed the title ~~create Redis DB instances based on CONF in config_db.json~~ create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json Dec 5, 2018

qiluo-msft reviewed Dec 6, 2018

View reviewed changes

dockers/docker-database/Dockerfile.j2 Outdated Show resolved Hide resolved

qiluo-msft reviewed Dec 6, 2018

View reviewed changes

dockers/docker-database/docker_init.sh Outdated Show resolved Hide resolved

qiluo-msft reviewed Dec 6, 2018

View reviewed changes

files/build_templates/docker_image_ctl.j2 Outdated Show resolved Hide resolved

qiluo-msft reviewed Dec 6, 2018

View reviewed changes

files/image_config/database/internal_ping_pong_database.sh.j2 Outdated Show resolved Hide resolved

qiluo-msft reviewed Dec 6, 2018

View reviewed changes

dockers/docker-database/create_all_redis_config.sh.j2 Outdated Show resolved Hide resolved

qiluo-msft reviewed Dec 6, 2018

View reviewed changes

files/build_templates/database.service.j2 Show resolved Hide resolved

qiluo-msft previously requested changes Dec 6, 2018

View reviewed changes

qiluo-msft reviewed Dec 14, 2018

View reviewed changes

dockers/docker-database/create_single_redis_config.sh Outdated Show resolved Hide resolved

dzhangalibaba force-pushed the createDBinst branch from 460ea33 to e7ad9d8 Compare December 20, 2018 02:02

jipanyang reviewed Jan 13, 2019

View reviewed changes

dzhangalibaba force-pushed the createDBinst branch from 707df1f to a7e5a0e Compare February 14, 2019 02:05

qiluo-msft reviewed Apr 2, 2019

View reviewed changes

dockers/docker-database/Dockerfile.j2 Outdated Show resolved Hide resolved

qiluo-msft reviewed Apr 2, 2019

View reviewed changes

dockers/docker-database/Dockerfile.j2 Outdated Show resolved Hide resolved

qiluo-msft reviewed Apr 2, 2019

View reviewed changes

dockers/docker-database/start.sh Outdated Show resolved Hide resolved