Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zoneinfo #623

Merged
merged 72 commits into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
ec3c8dc
Create timezone sub-package
niccokunzmann Jun 4, 2024
78d21ca
Start refactoring for zoneinfo: Move all pytz usage into timezone.pytz
niccokunzmann Jun 4, 2024
678feb9
Refactor timezone caching
niccokunzmann Jun 4, 2024
83e1ec6
Refactor out a pytz fix for rrule
niccokunzmann Jun 4, 2024
768741f
Refactor: Move timezone creation out of icalendar.cal
niccokunzmann Jun 4, 2024
077e7af
remove pytz in comments
niccokunzmann Jun 4, 2024
8b4d871
Move timezone id extraction
niccokunzmann Jun 4, 2024
8390c49
Refactor: remove pytz from prop.py
niccokunzmann Jun 4, 2024
f446169
remove pyts from test
niccokunzmann Jun 4, 2024
71acafe
refactor src/icalendar/tests/test_unit_cal.py
niccokunzmann Jun 4, 2024
caf2c7f
refactor src/icalendar/tests/test_period.py
niccokunzmann Jun 4, 2024
5ed7fca
Run first test with timezone awareness
niccokunzmann Jun 4, 2024
c7629e1
timezoned: +1 test
niccokunzmann Jun 4, 2024
8415bb0
Refactor tests to use pytest style
niccokunzmann Jun 4, 2024
3404241
refactor: move tests
niccokunzmann Jun 4, 2024
9af4a8d
refactor: move test
niccokunzmann Jun 4, 2024
6ee184c
move vBinary Tests
niccokunzmann Jun 4, 2024
1452fcf
move vBoolean test
niccokunzmann Jun 4, 2024
40dd7bb
create vBinary, vBoolean, vCalAddress tests
niccokunzmann Jun 4, 2024
7a1aeee
move vDDDTypes tests
niccokunzmann Jun 4, 2024
19d4a9d
move vDatetime tests
niccokunzmann Jun 4, 2024
01e9529
Move vPeriod tests
niccokunzmann Jun 4, 2024
e4debef
Make tests work
niccokunzmann Jun 4, 2024
50c583c
Test that timezone names are understood
niccokunzmann Jun 5, 2024
864a144
skip timezone name comparism for now
niccokunzmann Jun 5, 2024
e45ce94
Rename fix_pytz_rrule_until -> fix_rrule_until
niccokunzmann Jun 5, 2024
1c795ed
Add tzdata package to add-on timezones
niccokunzmann Jun 5, 2024
300e6d1
log changes
niccokunzmann Jun 5, 2024
22db734
Create provider interface and move function
niccokunzmann Jun 5, 2024
c0a13c3
Fix refactoring mistake
niccokunzmann Jun 5, 2024
6288697
Use default provider
niccokunzmann Jun 5, 2024
942490e
fix mistakes in tests
niccokunzmann Jun 5, 2024
e1b1127
Fix test dtstamp conversion to UTC
niccokunzmann Jun 5, 2024
0e11a23
Test all failing tests with pytz and zoneinfo
niccokunzmann Jun 6, 2024
b75be65
Parametrize tests to run with zoneinfo and pytz
niccokunzmann Jun 7, 2024
9035a22
Make timezones with / work
niccokunzmann Jun 7, 2024
2457f36
Make documentation build under Python 3.12
niccokunzmann Jun 4, 2024
c9f425b
Add methods to access examples faster
niccokunzmann Jun 7, 2024
9e89a02
Switch documentation to zoneinfo
niccokunzmann Jun 7, 2024
21211cc
Make a nicer reading
niccokunzmann Jun 7, 2024
b920876
move renaming backports.zoneinfo.ZoneInfo to zoneinfo.ZoneInfo into c…
niccokunzmann Jun 7, 2024
a7d8185
fix test run
niccokunzmann Jun 7, 2024
9bea7f7
Make tests run for pypy3, too
niccokunzmann Jun 7, 2024
864987c
include suggestion
niccokunzmann Jun 10, 2024
9bea8a0
Update link in README
niccokunzmann Jun 10, 2024
4eb94f5
address commit
niccokunzmann Jun 10, 2024
83eb74b
Add test to map to olson
niccokunzmann Jun 10, 2024
ee31374
improve docstrings
niccokunzmann Jun 10, 2024
86af756
Update src/icalendar/tests/conftest.py
niccokunzmann Jun 10, 2024
307a28b
Update README.rst
niccokunzmann Jun 10, 2024
a4c9ecd
Update src/icalendar/timezone/tzp.py
niccokunzmann Jun 10, 2024
c9f6c90
document tests
niccokunzmann Jun 10, 2024
e5785ec
Update src/icalendar/cal.py
niccokunzmann Jun 10, 2024
0d158c6
Update src/icalendar/timezone/tzp.py
niccokunzmann Jun 10, 2024
5146c08
Update src/icalendar/timezone/tzp.py
niccokunzmann Jun 10, 2024
5762ac3
Update src/icalendar/tests/test_timezoned.py
niccokunzmann Jun 10, 2024
0cef73b
Update README.rst
niccokunzmann Jun 11, 2024
9a86aa2
Update README.rst
niccokunzmann Jun 11, 2024
3e021a6
Update README.rst
niccokunzmann Jun 11, 2024
5fd11f3
Speed up tests with scoped cache
niccokunzmann Jun 12, 2024
b3fc6f8
Test the timezone offsets with pytz and zoneinfo
niccokunzmann Jun 12, 2024
abe94fb
use copyreg.pickle
niccokunzmann Jun 18, 2024
6478c12
Modify changelog
niccokunzmann Jun 18, 2024
a799e4c
Use pathlib instead of os.path
niccokunzmann Jun 18, 2024
4583615
Merge master
niccokunzmann Jun 19, 2024
96b4e76
Modify doctests and add more examples
niccokunzmann Jun 19, 2024
fa4db23
Update README.rst
niccokunzmann Jun 19, 2024
5c4c11e
Update README.rst
niccokunzmann Jun 19, 2024
2ee6137
Update README.rst
niccokunzmann Jun 19, 2024
50f9765
Update README.rst
niccokunzmann Jun 19, 2024
99084fb
Update README.rst
niccokunzmann Jun 19, 2024
d0fd617
Merge branch 'master' into zoneinfo
niccokunzmann Jun 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,45 @@
Changelog
=========


6.0.0 (unreleased)
------------------

Minor changes:

- Test that all code works with both ``pytz`` and ``zoneinfo``.

Breaking changes:

- Use ``zoneinfo`` for ``icalendar`` objects created from strings,
see `Issue #609 <https://github.com/collective/icalendar/issues/609>`_.

This is an tested extension of the functionality, not a restriction:
If you create ``icalendar`` objects with ``pytz`` timezones in your code,
``icalendar`` will continue to work in the same way.
Your code is not affected.

``zoneinfo`` will be used for those **objects that** ``icalendar``
**creates itself**.
This happens for example when parsing an ``.ics`` file, strings or bytes with
``from_ical()``.

If you rely on ``icalendar`` providing timezones from ``pytz``, you can add
one line to your code to get the behavior of versions below 6:

.. code:: Python

import icalendar
icalendar.use_pytz()

New features:

- ...

Bug fixes:

- ...

5.0.13 (unreleased)
-------------------

Expand Down
130 changes: 121 additions & 9 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,23 +45,98 @@ files.
Quick Guide
niccokunzmann marked this conversation as resolved.
Show resolved Hide resolved
-----------

``icalendar`` enables you to **create**, **inspect** and **modify**
calendaring information with Python.

To **install** the package, run::

pip install icalendar


Inspect Files
~~~~~~~~~~~~~

You can open an ``.ics`` file and see all the events::

>>> import icalendar
>>> path_to_ics_file = "src/icalendar/tests/calendars/example.ics"
>>> with open(path_to_ics_file) as f:
>>> from pathlib import Path
>>> ics_path = Path("src/icalendar/tests/calendars/example.ics")
>>> with ics_path.open() as f:
... calendar = icalendar.Calendar.from_ical(f.read())
>>> for event in calendar.walk('VEVENT'):
... print(event.get("SUMMARY"))
New Year's Day
Orthodox Christmas
International Women's Day

Using this package, you can also create calendars from scratch or edit existing ones.
Modify Content
~~~~~~~~~~~~~~

Such a calendar can then be edited and saved again.

.. code:: python

>>> calendar["X-WR-CALNAME"] = "My Modified Calendar" # modify
>>> print(calendar.to_ical()[:129]) # save modification
BEGIN:VCALENDAR
VERSION:2.0
PRODID:collective/icalendar
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:My Modified Calendar


Create Events, TODOs, Journals, Alarms, ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``icalendar`` supports the creation and parsing of all kinds of objects
in the standard.
niccokunzmann marked this conversation as resolved.
Show resolved Hide resolved

.. code:: python

>>> icalendar.Event() # events
VEVENT({})
>>> icalendar.FreeBusy() # free/busy times
VFREEBUSY({})
>>> icalendar.Todo() # Todo list entries
VTODO({})
>>> icalendar.Alarm() # Alarms e.g. for events
VALARM({})
>>> icalendar.Journal() # Journal entries
VJOURNAL({})


Have a look at `more examples
<https://icalendar.readthedocs.io/en/latest/usage.html>`_.

Use Timezones of your choice
niccokunzmann marked this conversation as resolved.
Show resolved Hide resolved
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

With ``icalendar``, you can localize your events to take place in different
timezones.
``zoneinfo``, ``dateutil.tz`` and ``pytz`` are compatible with ``icalendar``.
This example creates an event that uses all of the timezone implementations
with the same result:

.. code:: python

>>> import pytz, zoneinfo, dateutil.tz # timezone libraries
>>> import datetime, icalendar
>>> e = icalendar.Event()
>>> tz = dateutil.tz.tzstr("Europe/London")
>>> e["X-DT-DATEUTIL"] = icalendar.vDatetime(datetime.datetime(2024, 6, 19, 10, 1, tzinfo=tz))
>>> tz = pytz.timezone("Europe/London")
>>> e["X-DT-USE-PYTZ"] = icalendar.vDatetime(datetime.datetime(2024, 6, 19, 10, 1, tzinfo=tz))
>>> tz = zoneinfo.ZoneInfo("Europe/London")
>>> e["X-DT-ZONEINFO"] = icalendar.vDatetime(datetime.datetime(2024, 6, 19, 10, 1, tzinfo=tz))
>>> print(e.to_ical()) # the libraries yield the same result
BEGIN:VEVENT
X-DT-DATEUTIL;TZID=Europe/London:20240619T100100
X-DT-USE-PYTZ;TZID=Europe/London:20240619T100100
X-DT-ZONEINFO;TZID=Europe/London:20240619T100100
END:VEVENT



Versions and Compatibility
--------------------------
Expand All @@ -70,14 +145,51 @@ Versions and Compatibility
long-term compatibility with projects conflicts partially with providing and using the features that
the latest Python versions bring.

Since we pour more `effort into maintaining and developing icalendar <https://github.com/collective/icalendar/discussions/360>`__,
we split the project into two:
Volunteers pour `effort into maintaining and developing icalendar
<https://github.com/collective/icalendar/discussions/360>`__.
niccokunzmann marked this conversation as resolved.
Show resolved Hide resolved
Below, you can find an overview of the versions and how we maintain them.

Version 6
~~~~~~~~~

Version 6 of ``icalendar`` switches the timezone implementation to ``zoneinfo``.
This only affects you if you parse ``icalendar`` objects with ``from_ical()``.
The functionality is extended and is tested since 6.0.0 with both timezone
implementations: ``pytz`` and ``zoneinfo``.
niccokunzmann marked this conversation as resolved.
Show resolved Hide resolved

By default and since 6.0.0, ``zoneinfo`` timezones are created.

.. code:: python

>>> dt = icalendar.Calendar.example("timezoned").walk("VEVENT")[0]["DTSTART"].dt
>>> dt.tzinfo
ZoneInfo(key='Europe/Vienna')

If you would like to continue to receive ``pytz`` timezones in as parse results,
you can receive all the latest updates, and switch back to version 5.x behavior:

.. code:: python

>>> icalendar.use_pytz()
>>> dt = icalendar.Calendar.example("timezoned").walk("VEVENT")[0]["DTSTART"].dt
>>> dt.tzinfo
<DstTzInfo 'Europe/Vienna' CET+1:00:00 STD>

Version 6 is on `branch master <https://github.com/collective/icalendar/>`_ with compatibility to Python versions ``3.7+`` and ``PyPy3``.
We expect the ``master`` branch with versions ``6+`` to receive the latest updates and features.

Version 5
~~~~~~~~~

Version 5 uses only the ``pytz`` timezone implementation, and not ``zoneinfo``.
No updates will be released for this.
Please use version 6 and switch to use ``zoneinfo`` as documented above.

- `Branch 4.x <https://github.com/collective/icalendar/tree/4.x>`__ with maximum compatibility to Python versions ``2.7`` and ``3.4+``, ``PyPy2`` and ``PyPy3``.
- `Branch master <https://github.com/collective/icalendar/>`__ with the compatibility to Python versions ``3.7+`` and ``PyPy3``.
Version 4
~~~~~~~~~

We expect the ``master`` branch with versions ``5+`` receive the latest updates and features,
and the ``4.x`` branch the subset of security and bug fixes only.
Version 4 is on `branch 4.x <https://github.com/collective/icalendar/tree/4.x>`_ with maximum compatibility with Python versions ``2.7`` and ``3.4+``, ``PyPy2`` and ``PyPy3``.
The ``4.x`` branch only receives security and bug fixes if someone makes the effort.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this also mentioned for version 5?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at these:

We do not really make a release for 4.x, nobody invests time. For 5.x there will not really be a future release either because you can add one line to the code and then use 6.x. So, the claims will be dropped to support older versions but this is not part of this PR. The PR is for zoneinfo compatibility, also not for the next release. We can create an issue for the next release and what should be done, I think.

We recommend migrating to later Python versions and also providing feedback if you depend on the ``4.x`` features.

Related projects
Expand Down
5 changes: 2 additions & 3 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -210,9 +210,8 @@ Python type::
>>> vDatetime.from_ical('20050404T080000')
datetime.datetime(2005, 4, 4, 8, 0)

>>> dt = vDatetime.from_ical('20050404T080000Z')
>>> repr(dt)[:62]
'datetime.datetime(2005, 4, 4, 8, 0, tzinfo=<UTC>)'
>>> vDatetime.from_ical('20050404T080000Z')
datetime.datetime(2005, 4, 4, 8, 0, tzinfo=ZoneInfo(key='UTC'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting here, with the ZoneInfo change, I'm asking myself if old events stored with datetimes based on pytz will break, when the pytz libarary isn't available anymore.
I‌ will test this and put the results here.

No problem. The icalendar library isn't responsible to provide pytz to the consuming project.
So this is not an issue.


You can also choose to use the decoded() method, which will return a decoded
value directly::
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
# install requirements depending on python version
# see https://www.python.org/dev/peps/pep-0508/#environment-markers
'backports.zoneinfo; python_version == "3.7" or python_version == "3.8"',
'tzdata'
]


Expand Down
3 changes: 3 additions & 0 deletions src/icalendar/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,6 @@
q_split,
q_join,
)

# Switching the timezone provider
from icalendar.timezone import use_pytz, use_zoneinfo
91 changes: 60 additions & 31 deletions src/icalendar/cal.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

These are the defined components.
"""
from __future__ import annotations
from datetime import datetime, timedelta
from icalendar.caselessdict import CaselessDict
from icalendar.parser import Contentline
Expand All @@ -13,12 +14,23 @@
from icalendar.parser_tools import DEFAULT_ENCODING
from icalendar.prop import TypesFactory
from icalendar.prop import vText, vDDDLists
from icalendar.timezone_cache import _timezone_cache

import pytz
from icalendar.timezone import tzp
from typing import Tuple, List
import dateutil.rrule, dateutil.tz
from pytz.tzinfo import DstTzInfo
import os


def get_example(component_directory: str, example_name: str) -> bytes:
"""Return an example and raise an error if it is absent."""
here = os.path.dirname(__file__)
examples = os.path.join(here, "tests", component_directory)
if not example_name.endswith(".ics"):
example_name = example_name + ".ics"
example_file = os.path.join(examples, example_name)
if not os.path.isfile(example_file):
raise ValueError(f"Example {example_name} for {component_directory} not found. You can use one of {', '.join(os.listdir(examples))}")
with open(example_file, "rb") as f:
return f.read()


######################################
Expand Down Expand Up @@ -178,11 +190,7 @@ def add(self, name, value, parameters=None, encode=1):
if isinstance(value, datetime) and\
name.lower() in ('dtstamp', 'created', 'last-modified'):
# RFC expects UTC for those... force value conversion.
if getattr(value, 'tzinfo', False) and value.tzinfo is not None:
value = value.astimezone(pytz.utc)
else:
# assume UTC for naive datetime instances
value = pytz.utc.localize(value)
value = tzp.localize_utc(value)

# encode value
if encode and isinstance(value, list) \
Expand Down Expand Up @@ -367,11 +375,8 @@ def from_ical(cls, st, multiple=False):
comps.append(component)
else:
stack[-1].add_component(component)
if vals == 'VTIMEZONE' and \
'TZID' in component and \
component['TZID'] not in pytz.all_timezones and \
component['TZID'] not in _timezone_cache:
_timezone_cache[component['TZID']] = component.to_tz()
if vals == 'VTIMEZONE' and 'TZID' in component:
tzp.cache_timezone_component(component)
# we are adding properties to the current top of the stack
else:
factory = types_factory.for_property(name)
Expand Down Expand Up @@ -501,6 +506,12 @@ class Event(Component):
)
ignore_exceptions = True

@classmethod
def example(cls, name) -> Event:
niccokunzmann marked this conversation as resolved.
Show resolved Hide resolved
"""Return the calendar example with the given name."""
return cls.from_ical(get_example("events", name))



class Todo(Component):

Expand Down Expand Up @@ -553,6 +564,11 @@ class Timezone(Component):
required = ('TZID',) # it also requires one of components DAYLIGHT and STANDARD
singletons = ('TZID', 'LAST-MODIFIED', 'TZURL',)

@classmethod
def example(cls, name) -> Calendar:
"""Return the calendar example with the given name."""
return cls.from_ical(get_example("timezones", name))

@staticmethod
def _extract_offsets(component, tzname):
"""extract offsets and transition times from a VTIMEZONE component
Expand Down Expand Up @@ -580,12 +596,9 @@ def _extract_offsets(component, tzname):

rrulestr = component['RRULE'].to_ical().decode('utf-8')
rrule = dateutil.rrule.rrulestr(rrulestr, dtstart=rrstart)
if not {'UNTIL', 'COUNT'}.intersection(component['RRULE'].keys()):
# pytz.timezones don't know any transition dates after 2038
# either
rrule._until = datetime(2038, 12, 31, tzinfo=pytz.UTC)
tzp.fix_rrule_until(rrule, component['RRULE'])

# constructing the pytz-timezone requires UTC transition times.
# constructing the timezone requires UTC transition times.
# here we construct local times without tzinfo, the offset to UTC
# gets subtracted in to_tz().
transtimes = [dt.replace (tzinfo=None) for dt in rrule]
Expand Down Expand Up @@ -622,13 +635,31 @@ def _make_unique_tzname(tzname, tznames):
tznames.add(tzname)
return tzname

def to_tz(self):
"""convert this VTIMEZONE component to a pytz.timezone object
def to_tz(self, tzp=tzp):
"""convert this VTIMEZONE component to a timezone object
"""
return tzp.create_timezone(self)

@property
def tz_name(self) -> str:
"""Return the name of the timezone component.

Please note that the names of the timezone are different from this name
and may change with winter/summer time.
"""
try:
zone = str(self['TZID'])
return str(self['TZID'])
except UnicodeEncodeError:
zone = self['TZID'].encode('ascii', 'replace')
return self['TZID'].encode('ascii', 'replace')

def get_transitions(self) -> Tuple[List[datetime], List[Tuple[timedelta, timedelta, str]]]:
"""Return a tuple of (transition_times, transition_info)

- transition_times = [datetime, ...]
- transition_info = [(TZOFFSETTO, dts_offset, tzname)]

"""
zone = self.tz_name
transitions = []
dst = {}
tznames = set()
Expand Down Expand Up @@ -684,14 +715,7 @@ def to_tz(self):
break
assert dst_offset is not False
transition_info.append((osto, dst_offset, name))

cls = type(zone, (DstTzInfo,), {
'zone': zone,
'_utc_transition_times': transition_times,
'_transition_info': transition_info
})

return cls()
return transition_times, transition_info


class TimezoneStandard(Component):
Expand Down Expand Up @@ -729,6 +753,11 @@ class Calendar(Component):
required = ('PRODID', 'VERSION', )
singletons = ('PRODID', 'VERSION', 'CALSCALE', 'METHOD')

@classmethod
def example(cls, name) -> Calendar:
"""Return the calendar example with the given name."""
return cls.from_ical(get_example("calendars", name))

# These are read only singleton, so one instance is enough for the module
types_factory = TypesFactory()
component_factory = ComponentFactory()
Loading