Optional devDependencies?

[morethanreal]

I would like a way to specify optional devDependencies, ie. devDependencies that doesn't cause an error if they're not found.

[mgol]

Ping? This would be very useful; for the same reason we need optionalDependencies for production deps, we need them for development.

[othiym23]

This is not a terrible idea, but I need to think about how we might add something like this without it getting unwieldy and confusing. Also I don't really like optionalDevDependencies. ;)

[rlidwka]

Also I don't really like optionalDevDependencies

Maybe something like this would look better:

{
  "dependencies": {
    "foo": {
      "version": ">= 1.2.3",
      "optional": true,
      "dev": true
    }
  }
}

[mgol]

Also I don't really like optionalDevDependencies

The name may look a little weird but it's a direct equivalent of optionalDependencies so any other name would most likely be quite confusing.

[othiym23]

@rlidwka Having a firm grasp of what the version distribution of the CLI is for npm users, I am confident that if we made that change, the entire npm community would be inadequate to handle the resulting deluge of support tickets. That is a huge breaking change and isn't going to happen.

@mzgol It is a direct equivalent, and that's part of why I'm uncomfortable with this whole concept. There are too many property names including Dependency in package.json, and I'm pretty sure that that is a source of considerable confusion for npm users. This may be a case of a feature that is necessary for a small subset of users (which is why I think it may be a justified addition) and unobtrusive for everybody else, but I'm still uneasy about adding yet more complication to that aspect of npm.

[iarna]

@rlidwka To clarify the breakingness of the change: This'd be easy to implement in a backward compatible fashion but impossible to do in a forward compatible fashion. As such, users of older versions of npm wouldn't be able to install these modules. Given how slow people are to upgrade npm, forward compatibility is actually more troublesome than backward compatibility.

[rlidwka]

To clarify the breakingness of the change: This'd be easy to implement in a backward compatible fashion but impossible to do in a forward compatible fashion.

Add an ability to do this first, then in a couple of years when people are all using npm that supports it, newer syntax will gradually replace older one.

In fact I am thinking about forward compatibility in the future. Now we have "package": "version" mapping in the dependencies hash, which is impossible to extend in a forward-compatible way. But we might want to add more data about each dependency in the future (url? registry? shasum? gpg key?), so allowing each dependency spec to be an object would be a big step to make npm more future-proof. I think if npm accepted hashes instead of version strings, npm-shrinkwrap.json probably wouldn't have even existed, all this information could be just written there.

[Qard]

I need this. I have a test dependency that will fail to install on particular versions of node/io.js, that'll cause TravisCI to report failures. :(

[focusaurus]

So honestly I think the software lifecycle modeled by npm's current feature set (dev and prod), while adequate for many use cases (especially small modules), is frequently inadequate for applications. My bikeshed opinion here would be consider a rethink that acknowledges use cases such as:

• there are packages I need at build/CI time but not in the final build artifact (think uglify, CSS preprocessors, browserify, bower, etc)
  • And sometimes I just want to do a build, not run tests, so devDependencies is not what I want as that combines my test packages in a suboptimal way
  • usually these are necessary for local development as well
• there are packages I need at automated test time but not in the final build (mocha, phantomjs, code coverage, linters, etc). These may not be strictly necessary to do a build, though.
• there are packages I only optionally need for local development (watchify, node-inspector, these types of developer tools that typically are unnecessary at CI time, and may be unwanted on a per-developer choice)

So another way to think of this is some of my applications have lifecycles that include development, optional development, test, build, and run (production).

Thus my overall thought would be allow more flexible grouping/categorizing of deps so I can express this. My first thought would be tagging them as below but clearly a lot of thought/iteration would be required.

{
  "dependencies": {
    "mocha": {
      "version": "2.4.0",
      "tags": ["test"]
    },
    "browserify": {
      "version": "9.1.0",
      "tags": ["build"]
    },
    "express": {
      "version": "4.1.0",
      "tags": ["dev", "test", "production"]
    }
  }
}

The cli would need to accommodate this something like npm install --tags=dev,test and npm prune --tags=test.

Just food for thought. Given the implications, I'm not holding my breath for any traction on this.

[othiym23]

This has been moved to the npm roadmap, which we're using instead of the confusing next-* labels now.

[marijnh]

(Just to add a data point: I use fuse-bindings-dependent package in my dev workflow, which is likely to not install properly on non-Linux platforms. Right now, I'm just not listing it in my package.json at all, since I don't want to burden all users with failure messages for unneeded packages during install. As long as we have the current schema for dependencies, optionalDevDependencies seems an obvious and harmless feature to support.)

[othiym23]

The CLI team just had a fairly extensive discussion that covered both the idea of reworking how dependencies is laid out, and also the cost of integrating a simple optionalDevDependencies field to sit alongside optionalDependencies, and came to the following conclusions:

• It would be nice to integrate the various kinds of (optional,dev,bundled,peer,whatever)Dependencies into a single data structure, and there are some technical strategies that might make it possible without completely wrecking the ecosystem, but that’s a project of scale and scope greater than the team can reasonably hope to tackle within the medium (6-12 month) future. On its own, it’s a change of similar or greater complexity to npm@3, and in practice that took nearly a year of solid development to get stable enough for general use. It’s something that everybody wants, but the need isn’t strong enough yet to justify prioritizing it over our other outstanding work, and this use case is definitely not enough to justify it on its own.
• There’s enough incidental complexity to adding something even as simple as optionalDevDependencies on its own that we’re not going to get to that either within the next year. Those of you who have use cases for it have done an excellent job of describing how and why it would be useful to you, but you’re all power users, and are describing what I believe is an edge case of an edge case for the community at large. The changes would need to be made to read-package-tree, read-package-json, and normalize-package-data (in addition to the CLI itself), and would introduce yet more complicated conditional logic to what is already a pretty hairy chunk of the installer (look at how optionalDependencies are implemented for an example of how this gets unpleasantly complicated in practice – very few people would expect that to be implemented how it is, and the implementation is kind of dictated by the way it work). This isn’t to say that it’s not a valid use case, just that it’s also an encumbrance (in the form of complexity) that the CLI can’t really afford at this point.

So, I’m going to close this issue because this isn’t something the CLI team intends to implement. That said, I think that eventually some version of npm should really get around to unifying the various kinds of dependencies, and also everybody’s input on this issue was very valuable. Thanks to all for their time.