wasm artifact doesn't exist (anymore?)

[phadej]

Error: [GHCup-05841] Download failed: Process "curl" with arguments ["-fL", "-o",
                                                "/home/phadej/.ghcup/tmp/ghcup-0e8c66d3020a1a24/ghc-9.6.2.20230523-x86_64-linux-alpine3_12-cross_wasm32-wasi-release+fully_static.tar.xz.tmp",
                                                "https://gitlab.haskell.org/api/v4/projects/3223/jobs/1530829/artifacts/ghc-x86_64-linux-alpine3_12-cross_wasm32-wasi-release+fully_static.tar.xz"] failed with exit code 22.

[hasufell]

Yeah, gitlab is constantly busted and also broke all GHC nightlies. There's no hope.

It's not my responsibility.

@chreekat

[hasufell]

Ah, we have stable bindists here https://raw.githubusercontent.com/amesgen/ghc-wasm-bindists/main/meta.json

@amesgen

[hasufell]

Fixed

[hasufell]

Also updated bindist documentation: https://www.haskell.org/ghcup/guide/#cross-support

[chreekat]

Note that GHC HQ does not provide wasm bindists yet, as it is not considered stable enough yet. Does GHCup make that clear to users?

Or could GHC HQ make that more clear?

In other words, I do think it's GHCup's responsibility, because GHCup is of its own initiative augmenting GHC in a way that GHC can't support right now.

[hasufell]

Note that GHC HQ does not provide wasm bindists yet

That's not the point really. We provide all sorts of bindists not sanctioned by GHC HQ and we really don't care.

The point is that gitlab is configured to constantly purge artifacts and is an unreliable source for downloads.

[chreekat]

No, that's not the point. Although it's very true and I wish nobody outside of GHC devs relied on the nightly pipeline, even if it was totally reliable.

The point is that you broadcast a certain level of reliability and availability for GHC that is not based on reality and contradicts GHC's own statements. Then people get surprised when something breaks, reducing trust in both GHC and GHCup. This makes it harder for GHC to actually focus on the bugs it does have in the platforms it does claim to support. Nobody's perfect here; all software has bugs. But indicating that something does work, when we know it doesn't, is the problem.

[hasufell]

This is a false statement.

Cross bindists are marked as experimental. Please see the readme: https://github.com/haskell/ghcup-metadata#metadata-variants-distribution-channels

---

However, GHCup doesn't really care whether bindists are provided by upstream. We have our own QA.

[chreekat]

I disagree it is a false statement, but I guess this is the end of productive discussion on this issue. I will rest now.

[hasufell]

I disagree it is a false statement, but I guess this is the end of productive discussion on this issue. I will rest now.

Please elaborate if you make accusations that GHCup conveys incorrect expectations while these are clearly documented in the metadata readme and further described in the distribution policies.

[chreekat]

The fact that GHCup distributes a bindist of the wasm cross compiler at all means it is broadcasting a level of reliability that does not exist. Your first response on this issue puts the blame for this unreliability on GHC, when it was GHCup's own initiative that made a bindist available in the first place.

Your policy of providing bindists that upstream does not support is in conflict with your policy of maintaining a good relationship with upstream providers. How you handle that is up to you, but I don't think conjuring up reliability for unsupported platforms is making it any easier.

[chreekat]

For what it's worth, I think it's good that we disagree on which platforms to support. I am too conservative and would rather reduce availability and focus on making things perfect . An extreme version of that would be to drop support for everything except x86_64-linux and focus on getting truly reliable and stable releases running on just that. But that would be just as bad as claiming "GHC supports all platforms present and future, just use at your own risk". Better to have a balance somehow.

[chreekat]

I propose #152 and haskell/ghcup-hs#928 to highlight the experimental nature of the cross compilers.

[chreekat]

...and I admit I am totally off-topic by talking about the reliability of the wasm cross compiler itself, as opposed to the reliability of the download location for it or any other bindist. I totally agree artifacts from the nightly pipeline are not a thing anybody should rely on.

[hasufell]

I propose #152 and haskell/ghcup-hs#928 to highlight the experimental nature of the cross compilers.

all merged

[hasufell]

The fact that GHCup distributes a bindist of the wasm cross compiler at all means it is broadcasting a level of reliability

I don't think so. This was communicated as experimental from the start. We can make it more prominently "experimental". Pre-releases from upstream are also experimental and end users should be aware.

Your first response on this issue puts the blame for this unreliability on GHC, when it was GHCup's own initiative that made a bindist available in the first place.

Correct. Gitlab is unreliable and has been so for a long time, rendering nightlies absolutely broken.

Your policy of providing bindists that upstream does not support is in conflict with your policy of maintaining a good relationship with upstream providers.

Ok. I think I have to make this crystal clear: GHCup is a distribution channel and may stop relying on upstream bindists entirely. I do not care what upstream thinks of this. For people who do care, we provide ghcup-vanilla-X.Y.Z.yaml.

Upstream bindists have had tons of issues and half of them remain broken due to the DESTDIR bug. GHCup strives to be a quality distribution channel.

At the moment I have no appetite to work on bindist issues anymore, unless I get paid full-time for it. But we have no obligation to distribute only upstream bindists.

Your argument really makes no sense to me no matter how I look at it. Linux distributions also package GHC on their own. Where you get the impression from that we need to rely on GHC HQ to provide bindists escapes me.

[angerman]

@chreekat i don't think I agree with this assessment. Yes, marking them experimental is clearly a good idea. But I don't think GHCup should be bound to using upstream bindists or even be restricted to upstream bindists only.

Upstream should imho make sure the compiler builds reasonably well on all platforms.

Nix, Haskell.nix, chocolaty, brew, Ubuntu and other distributions may choose to build their own (parched) bindists from source. I don't think there is any reason to require anyone to use GHC HQ bindists.

IF we want to do packaging, let's do this, at the HF and have someone dedicated to providing system packages for all platforms. I don't think this is remotely feasible with the current set of employees and volunteers. Leaving it to community/distribution efforts where necessary seems appropriate.

[chreekat]

Sigh, yes. I do constantly forget that GHCup considers itself more like Debian than like rustup. I get confused by the name and the positioning of GHCup in the Haskell ecosystem. I guess this is the root of our misunderstanding.

Rustup does what I expect:

Rustup installs The Rust Programming Language from the official release channels...

(emphasis mine)

GHCup doesn't do what I expect, but I will try to remember and respect that fact.

[hasufell]

GHCup doesn't do what I expect, but I will try to remember and respect that fact.

Yeah, the official release channels don't really do what I'd expect either ;)

1. bindist issues are not fixed until the next release
2. platforms are silently dropped
3. configurations may change on a whim (e.g. alpine static vs dynamic), hard for consumers to detect at times
4. old bindists are not curated
5. bindist test suite is mostly broken
6. ...

Now, should GHC devs deal with all these things? I actually think NO.

But they should:

• make it feasible to run GHC test suite on the end-users system (that's where it matters, not in CI)
  • and have processes and mechanisms to get feedback for failing test suites (send report via cli that gets aggregated and analyzed somewhere)
• be aware that their build system is not just a dev tool for GHC hackers, but an interface for distributors and end users
• be very mindful about platform support
• stop relying on hermetically built bindists: make sure a manually compiled GHC works on all platforms and have sufficient mechanisms to detect when it wouldn't (through bindist configure, runtime checks, test suite, ...)
• have prereleases for all GHC versions, including minor releases
• better communicate everything that potentially impacts distributors