Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Figure out why parsedatetime==2.5 breaks collate tests #721

Closed
shaycrk opened this issue Dec 12, 2019 · 4 comments
Closed

Figure out why parsedatetime==2.5 breaks collate tests #721

shaycrk opened this issue Dec 12, 2019 · 4 comments
Assignees
Labels
blocked Waiting for an update in an external library

Comments

@shaycrk
Copy link
Contributor

shaycrk commented Dec 12, 2019

Currently pegged parsedatetime (a csvkit/agate dependency) to version 2.4 to avoid the error, but ideally could remove this specific dependency if we tracked down the underlying issue. For a stacktrace with the bug, see this TravisCI run (pasted below, too) while collecting test cases.

collected 385 items / 1 errors                                                 
==================================== ERRORS ====================================
_________ ERROR collecting src/tests/collate_tests/test_integration.py _________
src/tests/collate_tests/test_integration.py:20: in <module>
    cache_initialized_db=True, on_initialized=initialize_db.handler
.tox/py3/lib/python3.6/site-packages/testing/common/database.py:41: in __init__
    init_handler(self.cache)
src/tests/collate_tests/initialize_db.py:14: in handler
    load_data(connection)
src/tests/collate_tests/initialize_db.py:37: in load_data
    check=True,
/opt/python/3.6.7/lib/python3.6/subprocess.py:418: in run
    output=stdout, stderr=stderr)
E   subprocess.CalledProcessError: Command '['csvsql', '-v', '--no-constraints', '--tables', 'food_inspections', '--insert', '--db', 'postgresql://postgres@127.0.0.1:58119/test', '/home/travis/build/dssg/triage/src/tests/collate_tests/food_inspections_subset.csv']' returned non-zero exit status 1.
------------------------------- Captured stderr --------------------------------
Traceback (most recent call last):
  File "/home/travis/build/dssg/triage/.tox/py3/bin/csvsql", line 8, in <module>
    sys.exit(launch_new_instance())
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/csvkit/utilities/csvsql.py", line 221, in launch_new_instance
    utility.run()
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/csvkit/cli.py", line 118, in run
    self.main()
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/csvkit/utilities/csvsql.py", line 115, in main
    self._failsafe_main()
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/csvkit/utilities/csvsql.py", line 150, in _failsafe_main
    **self.reader_kwargs
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/agate/table/from_csv.py", line 88, in from_csv
    return Table(rows, column_names, column_types, row_names=row_names)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/agate/table/__init__.py", line 109, in __init__
    self._column_types = column_types.run(rows, self._column_names)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/agate/type_tester.py", line 110, in run
    if len(row) > i and not column_type.test(row[i]):
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/agate/data_types/base.py", line 29, in test
    self.cast(d)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/agate/data_types/date.py", line 76, in cast
    (value, ctx, _, _, matched_text), = self.parser.nlp(d, sourceTime=ZERO_DT)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/parsedatetime/__init__.py", line 2233, in nlp
    version)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/parsedatetime/__init__.py", line 1844, in parse
    retS, retTime, matched = parseMeth(s, sourceTime)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/parsedatetime/__init__.py", line 1408, in _partialParseQUnits
    sourceTime = self._evalQUnits(parseStr, sourceTime)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/parsedatetime/__init__.py", line 1100, in _evalQUnits
    sourceTime = self._buildTime(sourceTime, quantity, modifier, units)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/parsedatetime/__init__.py", line 389, in _buildTime
    ctx.updateAccuracy(realunit)
  File "/home/travis/build/dssg/triage/.tox/py3/lib/python3.6/site-packages/parsedatetime/context.py", line 137, in updateAccuracy
    acc = self._ACCURACY_REVERSE_MAPPING[acc]
KeyError: 's'
@thcrock
Copy link
Contributor

thcrock commented Dec 12, 2019

The comments here look relevant bear/parsedatetime#233

@nanounanue nanounanue self-assigned this May 28, 2020
@nanounanue nanounanue added the blocked Waiting for an update in an external library label May 28, 2020
@thcrock
Copy link
Contributor

thcrock commented Jun 3, 2020

@nanounanue @shaycrk They released parsedatetime 2.6 which should fix this issue.

The latest parsedatetime actually got bundled in #753 so this issue should be closed if that PR gets merged.

@nanounanue
Copy link
Contributor

@shaycrk I am checking this in several related branches. I will take care, don't worry

@nanounanue
Copy link
Contributor

Closed in a8a8388

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Waiting for an update in an external library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants