Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add a new database driver for Amazon DynamoDB #22077

Merged
merged 7 commits into from
Nov 15, 2022

Conversation

passren
Copy link
Contributor

@passren passren commented Nov 9, 2022

SUMMARY

There is no database driver for Amazon DynamoDB so far. I developed a Python DB API 2.0 (PEP 249) client for Amazon DynamoDB and did some work to be compatible with Superset. My Project is here. The latest version is 0.4.2 which have already supported Superset connector.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

image

TESTING INSTRUCTIONS

Add the unit test for db_engine_spec of pydynamodb.
For the setup in the Superset, please refer to the doc in my project.
https://github.com/passren/PyDynamoDB/wiki/5.-Superset

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

Copy link
Contributor Author

@passren passren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new database driver for AWS DynamoDB

Copy link
Member

@nytai nytai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov
Copy link

codecov bot commented Nov 15, 2022

Codecov Report

Merging #22077 (12bdf14) into master (4496748) will decrease coverage by 1.46%.
The diff coverage is 70.58%.

@@            Coverage Diff             @@
##           master   #22077      +/-   ##
==========================================
- Coverage   67.12%   65.66%   -1.47%     
==========================================
  Files        1818     1819       +1     
  Lines       69639    69656      +17     
  Branches     7496     7496              
==========================================
- Hits        46747    45738    -1009     
- Misses      20954    21980    +1026     
  Partials     1938     1938              
Flag Coverage Δ
hive ?
mysql 78.40% <70.58%> (-0.01%) ⬇️
postgres 78.46% <70.58%> (-0.01%) ⬇️
presto ?
python 78.52% <70.58%> (-3.05%) ⬇️
sqlite ?
unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/db_engine_specs/dynamodb.py 70.58% <70.58%> (ø)
superset/tables/schemas.py 0.00% <0.00%> (-100.00%) ⬇️
superset/columns/schemas.py 0.00% <0.00%> (-100.00%) ⬇️
superset/db_engines/hive.py 0.00% <0.00%> (-85.19%) ⬇️
...set/advanced_data_type/plugins/internet_address.py 16.32% <0.00%> (-79.60%) ⬇️
superset/utils/pandas_postprocessing/boxplot.py 20.51% <0.00%> (-79.49%) ⬇️
superset/charts/post_processing.py 11.76% <0.00%> (-77.95%) ⬇️
...perset/advanced_data_type/plugins/internet_port.py 18.75% <0.00%> (-77.09%) ⬇️
superset/utils/pandas_postprocessing/rolling.py 21.87% <0.00%> (-68.75%) ⬇️
...perset/utils/pandas_postprocessing/contribution.py 34.61% <0.00%> (-65.39%) ⬇️
... and 60 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@nytai nytai merged commit b32de3d into apache:master Nov 15, 2022
@villebro
Copy link
Member

Thanks for this work @passren, awesome work! It would be nice if you could follow up by adding an entry to the docs to make it more discoverable for other users.

@villebro
Copy link
Member

Thanks for this work @passren, awesome work! It would be nice if you could follow up by adding an entry to the docs to make it more discoverable for other users.

Whoops, I didn't notice you already had a PR open for this, thanks! #22145 (updated the PR description to cross reference this PR)

@passren
Copy link
Contributor Author

passren commented Nov 18, 2022

Thanks for this work @passren, awesome work! It would be nice if you could follow up by adding an entry to the docs to make it more discoverable for other users.

Whoops, I didn't notice you already had a PR open for this, thanks! #22145 (updated the PR description to cross reference this PR)

@villebro, my bad. I forgot to update the docs with this PR. So created the new PR to do that. Thanks for your caring. :D

@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.1.0 and removed 🚢 2.1.3 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L 🚢 2.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants