One Integration test broken locally TestSnapshotHardDelete.test__postgres__snapshot_hard_delete

[kadero]

Describe the bug

Hello 👋,

I followed the CONTRIBUTING.md guide to run dbt in development, here the steps I did:

• use pyenv and virtualenv to manage my python env.
• Instal all python dependencies (pip install -r dev-requirements.txt -r editable-requirements.txt )
• init the database (make setup-db)
• launch the unit-tests ✔️
• launch the integration tests 🔴

004_simple_snapshot_test integration test failed:

_______________________________________________________________________ TestSnapshotHardDelete.test__postgres__snapshot_hard_delete _______________________________________________________________________

self = <test_simple_snapshot.TestSnapshotHardDelete testMethod=test__postgres__snapshot_hard_delete>

    @use_profile('postgres')
    def test__postgres__snapshot_hard_delete(self):
        self.run_sql_file('seed_pg.sql')
>       self._test_snapshot_hard_delete()

test/integration/004_simple_snapshot_test/test_simple_snapshot.py:838: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test/integration/004_simple_snapshot_test/test_simple_snapshot.py:867: in _test_snapshot_hard_delete
    self._snapshot_and_assert_invalidated()
test/integration/004_simple_snapshot_test/test_simple_snapshot.py:909: in _snapshot_and_assert_invalidated
    self.assertGreaterEqual(result[-1].astimezone(pytz.UTC), self._invalidated_snapshot_datetime)
E   AssertionError: datetime.datetime(2021, 9, 26, 6, 8, 10, 84187, tzinfo=<UTC>) not greater than or equal to datetime.datetime(2021, 9, 26, 8, 8, 9, 594456, tzinfo=<UTC>)

FYI, I launched the test at 9:26 UTC.
Seems like result[-1].astimezone(pytz.UTC) doesn't return the right UTC time.

---

I did few tests:
result[-1].replace(tzinfo=pytz.UTC) fix the issue home.

If we took a look to astimezone doc

Changed in version 3.6: The astimezone() method can now be called on naive instances that are presumed to represent system local time.

Steps To Reproduce

Not sure to know how to reproduce the BUG. Root cause if probably linked to my local environment 😕 .
Please note my environment variable TZ is equal to Europe/Paris .
Same for PG Timezone:

SHOW TIMEZONE;
Europe/Paris

• Install DBT in dev following the CONTRIBUTING.md guide
• To launch the test:

 python -m pytest -m profile_postgres test/integration/004_simple_snapshot_test/test_simple_snapshot.py::TestSnapshotHardDelete::test__postgres__snapshot_hard_delete

Expected behavior

Integration tests to pass.

System information

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• snowflake
• other (specify: ____________)

The output of dbt --version:

installed version: 0.21.0-rc1
   latest version: 0.20.2

Your version of dbt is ahead of the latest release!

Plugins:
  - postgres: 0.21.0rc1
  - snowflake: 0.21.0rc1
  - redshift: 0.21.0rc1
  - bigquery: 0.21.0rc1

The operating system you're using:
Ubuntu 20.04.2 LTS

The output of python --version:
Python 3.8.11

Additional context

I can contribute to fix this bug if needed 🙂 .

[jtcohen6]

Thanks for opening @kadero! As it happens, my timezone is also Europe/Paris, so I was able to reproduce this one pretty easily :)

The switch you recommended, from astimezone(pytz.UTC) to .replace(tzinfo=pytz.UTC), worked perfectly for me to get this test passing locally. The weird part is that, several months ago, I saw the same issue reported (#3297), and it was the same switch in reverse that did the trick: #3344.

So: I'm possible (even likely) that I was dead-wrong back in April/May, and I'd welcome a PR to make this change, and fix the test for folks in other parts of the world. I'd just want to make sure that we're not going to see this pop up again in an as-of-yet untested timezone.

[kadero]

@jtcohen6 I did a few other tests in multiple timezone:

# EDT
sudo timedatectl set-timezone America/New_York && timedatectl && python -m pytest -m profile_postgres test/integration/004_simple_snapshot_test/test_simple_snapshot.py::TestSnapshotHardDelete::test__postgres__snapshot_hard_delete

# PDT
sudo timedatectl set-timezone America/Los_Angeles && timedatectl && python -m pytest -m profile_postgres test/integration/004_simple_snapshot_test/test_simple_snapshot.py::TestSnapshotHardDelete::test__postgres__snapshot_hard_delete

# CEST
sudo timedatectl set-timezone Europe/Paris && timedatectl && python -m pytest -m profile_postgres test/integration/004_simple_snapshot_test/test_simple_snapshot.py::TestSnapshotHardDelete::test__postgres__snapshot_hard_delete

# NZDT
sudo timedatectl set-timezone Pacific/Auckland && timedatectl && python -m pytest -m profile_postgres test/integration/004_simple_snapshot_test/test_simple_snapshot.py::TestSnapshotHardDelete::test__postgres__snapshot_hard_delete

# IST
sudo timedatectl set-timezone Asia/Kolkata && timedatectl && python -m pytest -m profile_postgres test/integration/004_simple_snapshot_test/test_simple_snapshot.py::TestSnapshotHardDelete::test__postgres__snapshot_hard_delete

Without the fix (current develop), the test fail for CEST, NZDT and IST Timezone.
Pass for all timezone with the fix .replace(tzinfo=pytz.UTC).

Ok I will open a PR 👌