Configure a backup destination for ClickHouse

[bnaecker]

I've often wanted to collect the latest ClickHouse data from Dogfood. I've been doing that by manually taking a ZFS snapshot; creating a tarball; and copying it out to my own files. However, ClickHouse actually supports in-band backups, that take a consistent view of the database or any subset of it. It would be very nice to let ClickHouse do this work for us. That requires that we configure a backup destination, a set of paths that the system is configured to use. That can only be specified in the XML configuration file. I don't see a CLI flag for it. We don't ship that at all for single-node ClickHouse, but we do in the replica configuration XML files. We should set that parameter, to allow backups.

This may also be a good mechanism to collect ClickHouse data for support bundles.

[jclulow]

What format does it use for writing all the backup data out?

[bnaecker]

It looks like it creates a zip, but I’m not sure of the contents. One can choose particular pieces of the database to backup, such as tables or views, so I think it creates some subset of the directory tree it normally operates on. One can also choose a range of compression options, and do incremental backups. This may not be the best approach. For example a ZFS snapshot may be preferable. But it is built in and convenient, and gives a consistent view of the database state.