Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use MethodName.File when value ends with .py #5295

Merged
merged 8 commits into from
Jul 29, 2022

Conversation

leoebfolsom
Copy link
Contributor

@leoebfolsom leoebfolsom commented May 24, 2022

Resolves #5289

Description

This allows the user to write a command such as dbt run -m my_fancy_stats_model.py. Currently, a model ending in .py (acknowledging such models don't yet exist) cannot include the file extension in the dbt run -m command.

My approach here was to create a new fixture called table_model_py. This allowed me to not touch table_model but still achieve the desired functionality. I thought this was a reasonable first step, in the spirit of "optimize later."

However, I think a cleaner approach would be either:
a) Updating table_model to allow for both .sql and .py extensions; or
b) Rename table_model to be table_model_sql, and maintain it alongside table_model_py.

I'm guessing a would be preferred (more scalable, less repeated code and less code in general), but I didn't want to build something like that out without getting input from others.

This is my first (attempted) contribution to dbt-core, so I may be doing things wrong--comments/guidance are welcome and desired! Thank you.

Checklist

@leoebfolsom leoebfolsom requested review from a team and iknox-fa May 24, 2022 04:57
@cla-bot cla-bot bot added the cla:yes label May 24, 2022
@leoebfolsom leoebfolsom requested a review from a team as a code owner May 26, 2022 23:54
@ChenyuLInx
Copy link
Contributor

@leoebfolsom the changes and test look great to me!

We are going to hold off on merging this until the python model is put out into the main branch.

@@ -80,7 +80,7 @@ def __post_init__(self):
def default_method(cls, value: str) -> MethodName:
if _probably_path(value):
return MethodName.Path
elif value.lower().endswith(".sql"):
elif value.lower().endswith((".sql",".py")):
Copy link
Contributor

@jtcohen6 jtcohen6 Jun 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Not to expand the scope of this well-defined and narrowly targeted PR, but: should we also have ".csv" here?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this for dbt seed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer to @jtcohen6 and @ChenyuLInx on this decision. I'd have to review the dependencies / code in more detail to understand whether including .csv would require additional changes, and I'm sure you have a better sense for that. If it's simply a matter of adding the .csv to this one line to slightly expand this PR, I definitely don't object! ty

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#5578 for this comment

Copy link
Contributor

@iknox-fa iknox-fa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me.

@jtcohen6
Copy link
Contributor

I think we're finally ready for this one :) @ChenyuLInx

@ChenyuLInx
Copy link
Contributor

I think we're finally ready for this one :) @ChenyuLInx

Haha I was looking for this today then saw your comment

@ChenyuLInx ChenyuLInx closed this Jul 28, 2022
@ChenyuLInx ChenyuLInx reopened this Jul 28, 2022
@@ -21,6 +22,7 @@ def models(self):
return {
"view_model.sql": base_view_sql,
"table_model.sql": base_table_sql,
"table_model.py": base_table_py,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we probably don't want to add this to the base adapter tests as not all adapters support running this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just the unittest above looks great to me

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All right, and it looks like you made that change, so you don't need anything from me?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just saw your other comment--looks good to me! Thank you!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I just went ahead and did it thinking that it would be great for this to be included in the Beta release coming up soon!

Thank you so much for contributing this and the super quick response!!!

@ChenyuLInx
Copy link
Contributor

@leoebfolsom we are back since PR for python model just merged. I am going to just remove the changes in integration test and just push this in. Let me know if there's anything else you would want us to update!

@ChenyuLInx ChenyuLInx closed this Jul 28, 2022
@ChenyuLInx ChenyuLInx reopened this Jul 28, 2022
@ChenyuLInx
Copy link
Contributor

Close and reopen to rerun synk check

@ChenyuLInx ChenyuLInx merged commit 5153023 into dbt-labs:main Jul 29, 2022
@leoebfolsom leoebfolsom deleted the leoebfolsom_file_selector_py branch July 29, 2022 15:25
agoblet pushed a commit to BigDataRepublic/dbt-core that referenced this pull request Sep 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-680] [Feature] Support for .py files when using file selectors
4 participants