-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Built-in YAML support #4910
Comments
You would have to represent these advanced components in the Nix language somehow; perhaps: { type = "yaml-advanced-component"; value = "..."; } With such a representation, nothing should stop you from converting that to proper YAML in a derivation. There's some concern about keeping Nix itself easy to bootstrap, so adding a yaml library dependency comes at a cost. For TOML we only have a parsing function, so we don't need IFD to read those files into Nix values. The same might apply for YAML, but that's not the use case you're describing. |
I marked this as stale due to inactivity. → More info |
I still care about this feature. |
I would be very happy to have a
|
I'm happy to take ownership of this. I'll get started on Wednesday next week. |
How is the status on this @ners ? We just had another use case popping up at nix-community/dream2nix#234 |
Adding a YAML parser effectively makes that specific YAML parser behavior part of the Nix language, because the Nix language should be reproducible. Parsing YAML through IFD is actually a good thing for reproducibility, because it pins the implementation of the YAML parser to a specific version, preventing any repro bugs that will be caused by fixes in the YAML parser, switching to a different YAML library, YAML major version upgrades, etc. |
Another approach to solve the I think IFD is not a good option for parsing. See #1491 (comment) Aside from the mentioned issues, the problem with IFD is that it significantly impacts UX in a bad way. For example, if the package name is part of the yaml, and therefore read through IFD, simply listing your packages will become an expensive operation. |
👀 I wouldn't think of Nix as a project that prefers to sacrifice correctness and reliability for a little performance improvement.
Not taking the package name from IFD doesn't seem too hard.
It does complicate things a bit. Usually this is not a problem at all. When it is, remote builds can be used to solve it. If we don't consider that to be sufficient, we could add explicit cross-evaluation support to Nixpkgs. Similar to Note that this is only for building. Listing packages and package metadata can be done without IFD. |
I suppose a middle ground is to require the |
My interest comes from building compatibility layers between nix and existing package managers. Reading other serialization formats like JSON, TOML, YAML is crucial for this as other package managers store data usually in these formats. YAML not being supported is a pain point. I have tried IFD, and I made the experiences that it complicates things a lot. Sure, one can try to avoid reading the package name via IFD and instead use some regex. Doing the same with package metadata becomes a lot harder already. Also, requiring users to setup remote builders in order to evaluate nix expressions, I think, is not the kind of UX we should aim for.
Sounds interesting. Maybe I misunderstand, but does this means that there would be a
I like that idea. There might be an overhead of maintaining more than one implementation at the same time, but old versions could be deprecated after a while. |
I don't want to diminish the importance of the ability to process YAML documents using the evaluator. I want us to make an informed decision and hopefully not sacrifice other important features like reproducibility.
Being able to build age-old packages on a current Nix installation is not something I think we should sacrifice after all these years.
Just like with cross, the builder is responsible for creating an output that is appropriate for No more dangerous than cross.
RFC 92 implementation has not been completed yet, so this may be a premature judgement.
I agree regexes are problematic. What else do you need for |
Some automatic nix packaging approaches benefit from not having to write (or generate) nix expressions. This on-the-fly translation increases eval complexity and might not be the first choice for repos like nixpkgs, but it allows using nix on existing software projects without intermediary steps, which, for example, is valuable for side-by-side usage with other package managers where otherwise, two sets of package expressions would have to be maintained. It also allows to use nix as a standard interface to inspect arbitrary packages of other ecosystems.
|
Actually, @arcnmx implemented a subset of YAML here: https://github.com/arcnmx/nixexprs/blob/f90f7c0a758f2142a448b9e59cc5d6f768b9275b/lib/from-yaml.nix Maybe it will prove sufficient to parse those yaml lock files. |
If there's one thing we learned from @grahamc talk at nixcon, it would be to implement |
I would be happy, if this could be discussed in #7340. At the moment I just added |
This will be useful for Dart / Flutter packaging in Nixpkgs. Currently we either IFD or convert the pubspec.lock file from yaml to json and use lib.importJSON. |
Is your feature request related to a problem? Please describe.
Currently in nixpkgs YAML is often rendered as JSON and
pkgs.formats.yaml
denoteswhich glances over the fact that JSON is only a subset of YAML.
Specifically we have no proper way to denote builtin and user-defined types, which are unquoted strings prefixed with two or one exclamation mark.
Describe the solution you'd like
A more complete mapping from Nix to YAML.
Describe alternatives you've considered
Rendering an attrset to JSON, then using remarshals
json2yaml
to convert it to YAML and then apply a regular expression to unquote strings that start with an exclamation mark.Home-assistant uses these to substitute secrets and do other includes at runtime.
Additional context
https://en.wikipedia.org/wiki/YAML#Advanced_components
The text was updated successfully, but these errors were encountered: