Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error processing .nm4 and .nmea files #93

Open
tsuzzy opened this issue Sep 13, 2024 · 2 comments
Open

Error processing .nm4 and .nmea files #93

tsuzzy opened this issue Sep 13, 2024 · 2 comments
Assignees
Labels
bug Something isn't working duplicate This issue or pull request already exists

Comments

@tsuzzy
Copy link
Collaborator

tsuzzy commented Sep 13, 2024

When using the function aisdb.database.decode_msgs() to decode and process .nm4 and .nmea files, the following message is prompted:

error processing path/to/filename.nm4, skipping checksum...

This results in an empty table being created in the SQLite database.

The issue can be reproduced with the following code,:

import aisdb

filepath = ['./sample_data.nm4']

with aisdb.DBConn(dbpath = 'sample.db') as dbconn:
    aisdb.decode_msgs(filepaths=filepath, dbconn=dbconn, source='TESTING')

Content in sample_data.nm4 is:

\s:41925,c:1635731906,t:1635731966*63\!AIVDM,1,1,,,11mg=5@000<Oe=Cbv:E@08L`00Ra,0*41
\s:41925,c:1635731906,t:1635731966*63\!AIVDM,1,1,,,15Utl40P0:<2TeiUEfmbk?vd08=T,0*78

Expected the file should be processed and data should be populated in the database.

Actual output:

generating file checksums...
checking file dates...
creating tables and dropping table indexes...
Memory: 21.92GB remaining.  CPUs: 12.  Average file size: 0.00MB  Spawning 4 workers
processing ./sample_data.nm4
saving checksums...
error processing ./sample_data.nm4, skipping checksum...
cleaning temporary data...

As observed, the message indicates a failure in processing the file, leading to the checksum being skipped and an empty table in the database.
Running the test script test_002_decode.py produces a similar output. However, the test passes because this failure does not raise any errors.

Environment:

  • OS: Ubuntu 22.04.4 LTS
  • Python version: 3.10.12
  • AISdb version: 1.7.3
@tsuzzy tsuzzy added the bug Something isn't working label Sep 13, 2024
@gabrielspadon
Copy link
Collaborator

Related to #78 , please address this issue as well.

@gabrielspadon gabrielspadon added the duplicate This issue or pull request already exists label Sep 16, 2024
@gabrielspadon
Copy link
Collaborator

Also related to #85 , please address this issue as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

2 participants