Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

CI: Fix the export-data job to run under poetry #12418

Merged
merged 3 commits into from
Apr 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 9 additions & 14 deletions .ci/scripts/test_export_data_command.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,24 @@

# Test for the export-data admin command against sqlite and postgres

# Expects Synapse to have been already installed with `poetry install --extras postgres`.
# Expects `poetry` to be available on the `PATH`.

set -xe
cd "$(dirname "$0")/../.."

echo "--- Install dependencies"

# Install dependencies for this test.
pip install psycopg2

# Install Synapse itself. This won't update any libraries.
pip install -e .

echo "--- Generate the signing key"

# Generate the server's signing key.
python -m synapse.app.homeserver --generate-keys -c .ci/sqlite-config.yaml
poetry run synapse_homeserver --generate-keys -c .ci/sqlite-config.yaml

echo "--- Prepare test database"

# Make sure the SQLite3 database is using the latest schema and has no pending background update.
update_synapse_database --database-config .ci/sqlite-config.yaml --run-background-updates
poetry run update_synapse_database --database-config .ci/sqlite-config.yaml --run-background-updates

# Run the export-data command on the sqlite test database
python -m synapse.app.admin_cmd -c .ci/sqlite-config.yaml export-data @anon-20191002_181700-832:localhost:8800 \
poetry run python -m synapse.app.admin_cmd -c .ci/sqlite-config.yaml export-data @anon-20191002_181700-832:localhost:8800 \
--output-directory /tmp/export_data

# Test that the output directory exists and contains the rooms directory
Expand All @@ -37,14 +32,14 @@ else
fi

# Create the PostgreSQL database.
.ci/scripts/postgres_exec.py "CREATE DATABASE synapse"
poetry run .ci/scripts/postgres_exec.py "CREATE DATABASE synapse"

# Port the SQLite databse to postgres so we can check command works against postgres
echo "+++ Port SQLite3 databse to postgres"
synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml
poetry run synapse_port_db --sqlite-database .ci/test_db.db --postgres-config .ci/postgres-config.yaml

# Run the export-data command on postgres database
python -m synapse.app.admin_cmd -c .ci/postgres-config.yaml export-data @anon-20191002_181700-832:localhost:8800 \
poetry run python -m synapse.app.admin_cmd -c .ci/postgres-config.yaml export-data @anon-20191002_181700-832:localhost:8800 \
--output-directory /tmp/export_data2

# Test that the output directory exists and contains the rooms directory
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -270,9 +270,10 @@ jobs:
steps:
- uses: actions/checkout@v2
- run: sudo apt-get -qq install xmlsec1
- uses: actions/setup-python@v2
- uses: matrix-org/setup-python-poetry@v1
with:
python-version: "3.9"
python-version: ${{ matrix.python-version }}
extras: "postgres"
- run: .ci/scripts/test_export_data_command.sh

portdb:
Expand Down
1 change: 1 addition & 0 deletions changelog.d/12418.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Run the CI export-data script in the locked poetry environment.