forked from torrust/torrust-index
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
refactor: add scaffolding for database migration command
- Loading branch information
1 parent
5d6dec0
commit 7513df0
Showing
11 changed files
with
661 additions
and
21 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
/.env | ||
/config.toml | ||
/data.db* | ||
/data_v2.db* | ||
/target | ||
/uploads/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
# DB migration | ||
|
||
With the console command `cargo run --bin db_migrate` you can migrate data from `v1.0.0` to `v2.0.0`. This migration includes: | ||
|
||
- Changing the DB schema. | ||
- Transferring the torrent files in the dir `uploads` to the database. | ||
|
||
## SQLite3 | ||
|
||
TODO | ||
|
||
## MySQL8 | ||
|
||
Please, | ||
|
||
> WARNING: MySQL migration is not implemented yet. We also provide docker infrastructure to run mysql during implementation of a migration tool. | ||
and also: | ||
|
||
> WARNING: We are not using a persisted volume. If you remove the volume used by the container you lose the database data. | ||
Run the docker container and connect using the console client: | ||
|
||
```s | ||
./db_migrate/docker/start_mysql.sh | ||
./db_migrate/docker/mysql_client.sh | ||
``` | ||
|
||
Once you are connected to the client you can create databases with: | ||
|
||
```s | ||
create database torrust_v1; | ||
create database torrust_v2; | ||
``` | ||
|
||
After creating databases you should see something like this: | ||
|
||
```s | ||
mysql> show databases; | ||
+--------------------+ | ||
| Database | | ||
+--------------------+ | ||
| information_schema | | ||
| mysql | | ||
| performance_schema | | ||
| sys | | ||
| torrust_v1 | | ||
| torrust_v2 | | ||
+--------------------+ | ||
6 rows in set (0.001 sec) | ||
``` | ||
|
||
How to connect from outside the container: | ||
|
||
```s | ||
mysql -h127.0.0.1 -uroot -pdb-root-password | ||
``` | ||
|
||
## Create DB for backend `v2.0.0` | ||
|
||
You need to create an empty new database for v2.0.0. | ||
|
||
You need to change the configuration in `config.toml` file to use MySQL: | ||
|
||
```yml | ||
[database] | ||
connect_url = "mysql://root:db-root-password@127.0.0.1/torrust_v2" | ||
``` | ||
|
||
After running the backend with `cargo run` you should see the tables created by migrations: | ||
|
||
```s | ||
mysql> show tables; | ||
+-------------------------------+ | ||
| Tables_in_torrust_v2 | | ||
+-------------------------------+ | ||
| _sqlx_migrations | | ||
| torrust_categories | | ||
| torrust_torrent_announce_urls | | ||
| torrust_torrent_files | | ||
| torrust_torrent_info | | ||
| torrust_torrent_tracker_stats | | ||
| torrust_torrents | | ||
| torrust_tracker_keys | | ||
| torrust_user_authentication | | ||
| torrust_user_bans | | ||
| torrust_user_invitation_uses | | ||
| torrust_user_invitations | | ||
| torrust_user_profiles | | ||
| torrust_user_public_keys | | ||
| torrust_users | | ||
+-------------------------------+ | ||
15 rows in set (0.001 sec) | ||
``` | ||
|
||
### Create DB for backend `v1.0.0` | ||
|
||
The `db_migrate` command is going to import data from version `v1.0.0` (database and `uploads` folder) into the new empty database for `v2.0.0`. | ||
|
||
You can import data into the source database for testing with the `mysql` DB client or docker. | ||
|
||
Using `mysql` client: | ||
|
||
```s | ||
mysql -h127.0.0.1 -uroot -pdb-root-password torrust_v1 < ./db_migrate/db_schemas/db_migrations_v1_for_mysql_8.sql | ||
``` | ||
|
||
Using dockerized `mysql` client: | ||
|
||
```s | ||
docker exec -i torrust-index-backend-mysql mysql torrust_v1 -uroot -pdb-root-password < ./db_migrate/db_schemas/db_migrations_v1_for_mysql_8.sql | ||
``` | ||
|
||
### Commands | ||
|
||
Connect to `mysql` client: | ||
|
||
```s | ||
mysql -h127.0.0.1 -uroot -pdb-root-password torrust_v1 | ||
``` | ||
|
||
Connect to dockerized `mysql` client: | ||
|
||
```s | ||
docker exec -it torrust-index-backend-mysql mysql torrust_v1 -uroot -pdb-root-password | ||
``` | ||
|
||
Backup DB: | ||
|
||
```s | ||
mysqldump -h127.0.0.1 torrust_v1 -uroot -pdb-root-password > ./db_migrate/db_schemas/v1_schema_dump.sql | ||
mysqldump -h127.0.0.1 torrust_v2 -uroot -pdb-root-password > ./db_migrate/db_schemas/v2_schema_dump.sql | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,152 @@ | ||
# 20220721205537_torrust_users.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_users ( | ||
user_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
date_registered DATETIME NOT NULL, | ||
administrator BOOLEAN NOT NULL DEFAULT FALSE | ||
) | ||
|
||
# 20220721210530_torrust_user_authentication.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_user_authentication ( | ||
user_id INTEGER NOT NULL PRIMARY KEY, | ||
password_hash TEXT NOT NULL, | ||
FOREIGN KEY(user_id) REFERENCES torrust_users(user_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220727213942_torrust_user_profiles.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_user_profiles ( | ||
user_id INTEGER NOT NULL PRIMARY KEY, | ||
username VARCHAR(24) NOT NULL UNIQUE, | ||
email VARCHAR(320) UNIQUE, | ||
email_verified BOOL NOT NULL DEFAULT FALSE, | ||
bio TEXT, | ||
avatar TEXT, | ||
FOREIGN KEY(user_id) REFERENCES torrust_users(user_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220727222313_torrust_tracker_keys.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_tracker_keys ( | ||
tracker_key_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
user_id INTEGER NOT NULL, | ||
tracker_key CHAR(32) NOT NULL, | ||
date_expiry BIGINT NOT NULL, | ||
FOREIGN KEY(user_id) REFERENCES torrust_users(user_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220730102607_torrust_user_public_keys.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_user_public_keys ( | ||
public_key_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
user_id INTEGER NOT NULL, | ||
public_key CHAR(32) UNIQUE NOT NULL, | ||
date_registered DATETIME NOT NULL, | ||
date_expiry DATETIME NOT NULL, | ||
revoked BOOLEAN NOT NULL DEFAULT FALSE, | ||
FOREIGN KEY(user_id) REFERENCES torrust_users(user_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220730104552_torrust_user_invitations.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_user_invitations ( | ||
invitation_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
user_id INTEGER NOT NULL, | ||
public_key CHAR(32) NOT NULL, | ||
signed_digest CHAR(32) NOT NULL, | ||
date_begin DATETIME NOT NULL, | ||
date_expiry DATETIME NOT NULL, | ||
max_uses INTEGER NOT NULL, | ||
personal_message VARCHAR(512), | ||
FOREIGN KEY(user_id) REFERENCES torrust_users(user_id) ON DELETE CASCADE, | ||
FOREIGN KEY(public_key) REFERENCES torrust_user_public_keys(public_key) ON DELETE CASCADE | ||
) | ||
|
||
# 20220730105501_torrust_user_invitation_uses.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_user_invitation_uses ( | ||
invitation_use_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
invitation_id INTEGER NOT NULL, | ||
registered_user_id INTEGER NOT NULL, | ||
date_used DATETIME NOT NULL, | ||
FOREIGN KEY(invitation_id) REFERENCES torrust_user_invitations(invitation_id) ON DELETE CASCADE, | ||
FOREIGN KEY(registered_user_id) REFERENCES torrust_users(user_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220801201435_torrust_user_bans.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_user_bans ( | ||
ban_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
user_id INTEGER NOT NULL, | ||
reason TEXT NOT NULL, | ||
date_expiry DATETIME NOT NULL, | ||
FOREIGN KEY(user_id) REFERENCES torrust_users(user_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220802161524_torrust_categories.sql | ||
|
||
CREATE TABLE torrust_categories ( | ||
category_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
name VARCHAR(64) NOT NULL UNIQUE | ||
); | ||
|
||
INSERT INTO torrust_categories (name) VALUES ('movies'), ('tv shows'), ('games'), ('music'), ('software'); | ||
|
||
# 20220810192613_torrust_torrents.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_torrents ( | ||
torrent_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
uploader_id INTEGER NOT NULL, | ||
category_id INTEGER NOT NULL, | ||
info_hash CHAR(40) UNIQUE NOT NULL, | ||
size BIGINT NOT NULL, | ||
name TEXT NOT NULL, | ||
pieces LONGTEXT NOT NULL, | ||
piece_length BIGINT NOT NULL, | ||
private BOOLEAN NULL DEFAULT NULL, | ||
root_hash BOOLEAN NOT NULL DEFAULT FALSE, | ||
date_uploaded DATETIME NOT NULL, | ||
FOREIGN KEY(uploader_id) REFERENCES torrust_users(user_id) ON DELETE CASCADE, | ||
FOREIGN KEY(category_id) REFERENCES torrust_categories(category_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220810201538_torrust_torrent_files.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_torrent_files ( | ||
file_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
torrent_id INTEGER NOT NULL, | ||
md5sum TEXT NULL DEFAULT NULL, | ||
length BIGINT NOT NULL, | ||
path TEXT DEFAULT NULL, | ||
FOREIGN KEY(torrent_id) REFERENCES torrust_torrents(torrent_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220810201609_torrust_torrent_announce_urls.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_torrent_announce_urls ( | ||
announce_url_id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT, | ||
torrent_id INTEGER NOT NULL, | ||
tracker_url VARCHAR(256) NOT NULL, | ||
FOREIGN KEY(torrent_id) REFERENCES torrust_torrents(torrent_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220812181520_torrust_torrent_info.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_torrent_info ( | ||
torrent_id INTEGER NOT NULL PRIMARY KEY, | ||
title VARCHAR(256) UNIQUE NOT NULL, | ||
description TEXT DEFAULT NULL, | ||
FOREIGN KEY(torrent_id) REFERENCES torrust_torrents(torrent_id) ON DELETE CASCADE | ||
) | ||
|
||
# 20220812184806_torrust_torrent_tracker_stats.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_torrent_tracker_stats ( | ||
torrent_id INTEGER NOT NULL PRIMARY KEY, | ||
tracker_url VARCHAR(256) NOT NULL, | ||
seeders INTEGER NOT NULL DEFAULT 0, | ||
leechers INTEGER NOT NULL DEFAULT 0, | ||
FOREIGN KEY(torrent_id) REFERENCES torrust_torrents(torrent_id) ON DELETE CASCADE, | ||
UNIQUE(torrent_id, tracker_url) | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
# 20210831113004_torrust_users.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_users ( | ||
user_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, | ||
username VARCHAR(32) NOT NULL UNIQUE, | ||
email VARCHAR(100) NOT NULL UNIQUE, | ||
email_verified BOOLEAN NOT NULL DEFAULT FALSE, | ||
password TEXT NOT NULL | ||
); | ||
|
||
# 20210904135524_torrust_tracker_keys.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_tracker_keys ( | ||
key_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, | ||
user_id INTEGER, | ||
key VARCHAR(32) NOT NULL, | ||
valid_until INT(10) NOT NULL, | ||
FOREIGN KEY(user_id) REFERENCES torrust_users(user_id) | ||
); | ||
|
||
# 20210905160623_torrust_categories.sql | ||
|
||
CREATE TABLE torrust_categories ( | ||
category_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, | ||
name VARCHAR(64) NOT NULL UNIQUE | ||
); | ||
|
||
INSERT INTO torrust_categories (name) VALUES | ||
('movies'), ('tv shows'), ('games'), ('music'), ('software'); | ||
|
||
# 20210907083424_torrust_torrent_files.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_torrent_files ( | ||
file_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, | ||
torrent_id INTEGER NOT NULL, | ||
number INTEGER NOT NULL, | ||
path VARCHAR(255) NOT NULL, | ||
length INTEGER NOT NULL, | ||
FOREIGN KEY(torrent_id) REFERENCES torrust_torrents(torrent_id) | ||
); | ||
|
||
# 20211208143338_torrust_users.sql | ||
|
||
ALTER TABLE torrust_users; | ||
ADD COLUMN administrator BOOLEAN NOT NULL DEFAULT FALSE; | ||
|
||
# 20220308083424_torrust_torrents.sql | ||
|
||
CREATE TABLE IF NOT EXISTS torrust_torrents ( | ||
torrent_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, | ||
uploader VARCHAR(32) NOT NULL, | ||
info_hash VARCHAR(20) UNIQUE NOT NULL, | ||
title VARCHAR(256) UNIQUE NOT NULL, | ||
category_id INTEGER NOT NULL, | ||
description TEXT, | ||
upload_date INT(10) NOT NULL, | ||
file_size BIGINT NOT NULL, | ||
seeders INTEGER NOT NULL, | ||
leechers INTEGER NOT NULL, | ||
FOREIGN KEY(uploader) REFERENCES torrust_users(username) ON DELETE CASCADE, | ||
FOREIGN KEY(category_id) REFERENCES torrust_categories(category_id) ON DELETE CASCADE | ||
); | ||
|
||
# 20220308170028_torrust_categories.sql | ||
|
||
ALTER TABLE torrust_categories | ||
ADD COLUMN icon VARCHAR(32); | ||
|
Oops, something went wrong.