Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build improvements #93

Closed
wants to merge 13 commits into from
Closed

Build improvements #93

wants to merge 13 commits into from

Conversation

duijf
Copy link
Contributor

@duijf duijf commented Apr 2, 2020

Background

Internally, we found that the static binary for vaultenv based on integer-simple is quite a bit slower when creating TLS connections than we'd like. Most vaultenv users shouldn't notice this, but we start vaultenv quite frequently, so this overhead adds up.

We'd like to make it easy to have two binaries:

  • A static binary based on integer-simple that we can use for releases. This should be more than sufficient for most users.
  • A static binary based on GMP that users can build from source if they want super fast TLS handling. (We sadly cannot redistribute this binary because of the constraints of the LGPL).

High level implementation

I ran into a few problems with our build infrastructure, namely:

  1. We did not have a great way to reliably produce Debian packages yet.
  2. We can not the full-build-script attribute from nix/vaultenv-static.nix in a Nix build itself. It requires internet access.

Therefore, I took the following approach:

  • Use survey.nix from static-haskell-nix directly so our build does not require internet access due to stack2nix.
  • Introduce an abstraction to allow us to build Debian packages.
  • Hpack was giving me more trouble than it was worth so I went for a plain cabal file (with common stanzas to cut the boilerplate).
  • Write the Nix expression for vaultenv by hand instead of generating it from stack2nix.

Status

  • I can build a static binary based on libgmp.
  • The static binary based on integer-simple has some problems. I ran into most things mentioned in cabal2nix evaluation failures nh2/static-haskell-nix#73, and am out of time for now. Upstream has a lot of activity, so I'm waiting on that.
  • The CI config still needs to be updated to work release.nix.

As there are quite some sweeping changes in here already, I'm asking for a preliminary review here.

@duijf duijf requested a review from fatho April 2, 2020 11:07
Copy link
Member

@fatho fatho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes themselves look all good to me, but they have some wide-reaching consequences on other parts. The readme needs an update now (and other parts of the documentation possibly as well), and the integration tests still rely on stack at the moment.

I'm currently trying to build the GMP derivation locally to see whether everything works as expected, but that might still take a while as it appears to currently being building a MUSL-based GCC first...

default.nix Show resolved Hide resolved
vaultenv.cabal Outdated Show resolved Hide resolved
vaultenv.cabal Show resolved Hide resolved
@duijf
Copy link
Contributor Author

duijf commented Oct 6, 2021

This one didn't age too well. Closing this 😅

@duijf duijf closed this Oct 6, 2021
@fatho
Copy link
Member

fatho commented Oct 6, 2021

@duijf The nix derivation part served us well though, until we merged something like that in a fresh PR in #98, just without the static-binary mess.

@duijf
Copy link
Contributor Author

duijf commented Oct 6, 2021

Ah that's cool! Glad it ended up being useful in some way :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants