Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build: add openbsd/arm builder #25498

Closed
FiloSottile opened this issue May 22, 2018 · 21 comments
Closed

x/build: add openbsd/arm builder #25498

FiloSottile opened this issue May 22, 2018 · 21 comments
Labels
Builders x/build issues (builders, bots, dashboards) FeatureRequest FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. new-builder
Milestone
Unreleased

Comments

@FiloSottile
Copy link
Contributor

@4a6f656c used to maintain one but it doesn't seem to be running anymore.

Cloud arm32 is hard to find, so this might end up being a Raspberry Pi on my desk.
@gopherbot gopherbot added this to the Unreleased milestone May 22, 2018
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label May 22, 2018
@4a6f656c
Copy link
Contributor

I've resurrected the openbsd/arm builder and should be able to continue running it for the foreseeable future (or until another option is found).

@bcmills
Copy link
Contributor

bcmills commented May 6, 2019

I see that the builder is running, but it seems not to have enough memory to complete the tests reliably. Basically all of the runs are failing with fatal error: runtime: out of memory.

Is it possible to add more RAM to the builder, or perhaps configure it with some swap to deal with bigger tests?

Examples:
https://build.golang.org/log/17984e5b3a059c1882855a2c8774fdb7d8ecfad6
https://build.golang.org/log/5233126dc3a18b78a2fee33288f378485eeb66ac
https://build.golang.org/log/ce51b2eebed1b20dc6be028224d613fda8bcc216

@bcmills bcmills mentioned this issue Jun 28, 2019
Closed
@4a6f656c
Copy link
Contributor

@bcmills - the platform is limited to 1GB of data size and this was a regression from Go 1.12, which was fixed via issue #27739.

@bcmills
Copy link
Contributor

bcmills commented Aug 30, 2019

@4a6f656c, please disable the openbsd-arm builder until it can be converted to use cmd/buildlet.

The old-style builders are no longer supported (#21191 (comment)), and we can't easily fix configuration issues with them (such as #32838).

@bcmills bcmills mentioned this issue Sep 3, 2019
Open
@bcmills bcmills mentioned this issue Oct 4, 2019
Closed
@4a6f656c
Copy link
Contributor

4a6f656c commented Jan 4, 2020

@bcmills @dmitshur - a replacement openbsd-arm builder (host-openbsd-arm-joelsing) is running with cmd/buildlet, however is repeatedly failing in the coordinator (server side) with a writeSnapshot: failed to write snapshot to GCS timeout... I presume that my somewhat limited upstream bandwidth, combined with being a long way away (network wise), means that the time limit is insufficient to complete the upload. Could you please investigate?

@bcmills
Copy link
Contributor

bcmills commented Jan 6, 2020

(CC @toothrot, @cagedmantis)

@4a6f656c
Copy link
Contributor

@bcmills @toothrot @cagedmantis - I've turned the buildlet off since it has just been spinning its wheels for the last month. Please let me know when I should try again.

@4a6f656c 4a6f656c mentioned this issue Jan 31, 2020
Closed
@FiloSottile FiloSottile removed their assignment Feb 19, 2020
@4a6f656c 4a6f656c mentioned this issue Apr 13, 2020
Closed
@dmitshur
Copy link
Contributor

dmitshur commented Apr 13, 2020 •
edited
Loading

@4a6f656c Thanks for working on this and sorry about the delay. A few questions to better understand the writeSnapshot issue that's blocking you from proceeding.

Have you considered the SkipSnapshot configuration option for this builder? Do you know what the effect of using it on your builder would be?

The writeSnapshot timeout was increased from 1 minute to 5 minutes 3 years ago in CL 37468, with a comment noting that most operations should happen in well under a minute. Do you have information about the upstream network speed of your builder, so we can estimate if increasing that timeout slightly would be sufficient, or if we need to approach this differently.

@4a6f656c
Copy link
Contributor

@dmitshur - ah, I was not aware of SkipSnapshot - that does in fact look like it would solve the issue, not sure if sharded tests are an issue here (for openbsd/arm and openbsd/arm64 they would seem unlikely, not sure for linux/riscv64). The upstream network is ~1Mbps, the bigger issue is likely the RTT from .au to .us.

I'll raise a CL...

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/228238 mentions this issue: dashboard: enable SkipSnapshot for openbsd-arm-jsing builder

gopherbot pushed a commit to golang/build that referenced this issue Apr 14, 2020
@4a6f656c
dashboard: enable SkipSnapshot for openbsd-arm-jsing builder
9a1cfea 
The builder network is too slow/distant to upload snapshots successfully.

Updates golang/go#25498

Change-Id: Ida86eb1f84e34c9810227c98370cef980529796f
Reviewed-on: https://go-review.googlesource.com/c/build/+/228238
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@4a6f656c
Copy link
Contributor

@dmitshur - could you please let me know when that change is deployed and I'll try restarting the openbsd/arm builder.

@dmitshur
Copy link
Contributor

@4a6f656c Sure. It's deployed now.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/228597 mentions this issue: dashboard: reduce the amount of work for openbsd/arm

@4a6f656c
Copy link
Contributor

@dmitshur - thanks, we're now up and running, although I've just mailed another CL to reduce the amount of work the builder is trying to do.

gopherbot pushed a commit to golang/build that referenced this issue Apr 16, 2020
@4a6f656c
dashboard: reduce the amount of work for openbsd/arm
6a8b9e1 
Given the performance of this hardware, reduce the amount of work that
it is trying to do.

Updates golang/go#25498

Change-Id: I943ab2b01bd10c5217c097464d3904b0f4de2d0a
Reviewed-on: https://go-review.googlesource.com/c/build/+/228597
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@4a6f656c
Copy link
Contributor

@dmitshur - any chance of getting the last change deployed (openbsd/arm currently has 873 builds queued)?

@dmitshur
Copy link
Contributor

dmitshur commented Apr 20, 2020 •
edited
Loading

@4a6f656c Yes, it will be deployed at the nearest opportunity, likely today.

@4a6f656c
Copy link
Contributor

@dmitshur - thanks!

@andybons
Copy link
Member

@4a6f656c the openbsd-arm-jsing builder has been failing builds for a bit now. Can you take a look?

@4a6f656c
Copy link
Contributor

@andybons the builder was upgraded to OpenBSD 6.7 - builds resumed functioning after https://go-review.googlesource.com/c/go/+/234381 was landed. There are still a handful of intermittent failures which I need to look at - most of these are likely related to the fact that it is a slow non-MP platform.

@4a6f656c
Copy link
Contributor

@andybons @dmitshur - I presume this can now be closed?

@dmitshur
Copy link
Contributor

A builder for linux/arm has been added and it's live, so let's close this. Thank you Joel!

@dmitshur dmitshur added the NeedsFix The path to resolution is known, but the work has not been done. label Aug 20, 2020
@tklauser tklauser mentioned this issue Jan 14, 2021
Closed
gopherbot pushed a commit to golang/net that referenced this issue Jan 19, 2021
@tklauser
ipv6: skip known-failing tests on openbsd-{arm,arm64,mips64}-jsing bu…
5f4716e 
…ilders

The arm64 and mips64 builder run OpenBSD 6.8:

openbsd-arm64-jsing:
OpenBSD gopher-arm64.sing.id.au 6.8 GENERIC.MP#2 arm64

openbsd-mips64-jsing:
OS: OpenBSD er4.sing.id.au 6.8 GENERIC.MP#0 octeon

The openbsd-arm-jsing builder was offline but according to
golang/go#25498 (comment)
it's running at least OpenBSD 6.7. Assuming that this version is
affected as well (or the builder was upgraded to 6.8 in the meantime),
skip TestPacketConnReadWriteMulticastUDP on this builder as well.

For golang/go#11811.
Updates golang/go#42064.

Change-Id: Ia9919057bfc2a69b8884228508ae919ce9d624e9
Reviewed-on: https://go-review.googlesource.com/c/net/+/283752
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Joel Sing <joel@sing.id.au>
@golang golang locked and limited conversation to collaborators Aug 20, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Builders x/build issues (builders, bots, dashboards) FeatureRequest FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. new-builder
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
7 participants
@bradfitz @andybons @FiloSottile @dmitshur @bcmills @4a6f656c @gopherbot