Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make default test-paths and analysis-paths configs consistent #2659

Closed
clrcrl opened this issue Jul 29, 2020 · 5 comments · Fixed by #4007
Closed

Make default test-paths and analysis-paths configs consistent #2659

clrcrl opened this issue Jul 29, 2020 · 5 comments · Fixed by #4007
Assignees
Labels
1.0.0 Issues related to the 1.0.0 release of dbt bug Something isn't working

Comments

@clrcrl
Copy link
Contributor

clrcrl commented Jul 29, 2020

Describe the bug

Currently, the default values for various <resource>-paths are:

source-paths: ['models']
macro-paths:['macros']
data-paths: ['data']
test-paths: ['test']
analysis-paths:  []

(source)

The defaults for test-paths and analysis-paths:

  • Are not consistent with the other defaults (i.e. ['test'] instead of ['tests'], and [] instead of ['analyses'])
  • Differ from the values set by dbt init which are ['tests'] and ['analysis'] respectively (source),
Config What I expect the default to be The actual default The value set by dbt init
test-paths ['tests'] ['test'] ['tests']
analysis-paths ['analyses'] [] ['analysis'] (singular)

We should:

  • Update the default values to be ['tests'] and ['analyses']
    - Create a compatibility shim of some sort:
    - Jake: "in the default case, we can check if any *.sql files exist inside tests, and if not fall back to looking in test (and emit a deprecation warning). This will be annoying for a hopefully tiny subset of the population who have a directory named tests with sql files in it." We don't want to do this
  • Update the starter-project to set ['analyses'] for new projects going forward
@clrcrl clrcrl added bug Something isn't working triage labels Jul 29, 2020
@jtcohen6 jtcohen6 added 1.0.0 Issues related to the 1.0.0 release of dbt and removed triage labels Jul 29, 2020
@jtcohen6 jtcohen6 mentioned this issue Jun 16, 2021
11 tasks
@gshank
Copy link
Contributor

gshank commented Jun 24, 2021

How do we feel about the config for models being named 'source-paths'? That's a bit confusing too since we have "models" and "sources" and "models" go in the "source-paths"...

@gshank
Copy link
Contributor

gshank commented Jun 24, 2021

I'm wondering whether checking a directory for SQL files is the way we want to go in a world in which file contents are not acquired by looking at a filesystem. The "frontend" processor has to know which files to send and therefore would need to do those kinds of checks. So there's a bit of a chicken-and-egg problem (something we'll have to deal with in a storage adapter world anyway).

@jtcohen6
Copy link
Contributor

How do we feel about the config for models being named 'source-paths'? That's a bit confusing too since we have "models" and "sources" and "models" go in the "source-paths"...

@gshank You're absolutely right! See #1607, which is also on the v1.0 list. If you want to talk about that one at the same time as this issue, or consolidate the two into one effort, I think that would make a lot of sense.

I'm wondering whether checking a directory for SQL files is the way we want to go in a world in which file contents are not acquired by looking at a filesystem.

This feels like a valuable discussion! We've talked in the past about using Jinja blocks to demarcate what code represents what thing, e.g. #184. That was prominently featured on our v1.0 list, way back when. It's not something I'm convinced we need or want to do, but I think it raises fair questions for a world where dbt doesn't have to be so tightly coupled with a file system.

@leahwicz
Copy link
Contributor

leahwicz commented Oct 4, 2021

@jtcohen6 I am assigning this to @emmyoop to take a look at. Should the exit criteria be the 2 "should" points from above?

  • Update the default values to be ['tests'] and ['analyses']
  • Update the starter-project to set ['analyses'] for new projects going forward

@jtcohen6
Copy link
Contributor

jtcohen6 commented Oct 5, 2021

@leahwicz Yes, that sounds exactly right to me! Because the init starter project sets explicit values for analysis-paths and test-paths, I don't believe these changes should be breaking for 99% of projects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.0.0 Issues related to the 1.0.0 release of dbt bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants