[REQUEST] arm64 support #186

davidjb · 2023-11-16T08:11:49Z

What would you like to have implemented?

Builds of Chromium for both x86_64 and arm64.

I know this has been discussed a lot previously in #93 and #19 but with the general availability release of provided.al2023 for Lambda (https://aws.amazon.com/blogs/compute/introducing-the-amazon-linux-2023-runtime-for-aws-lambda/), this now seems feasible given previous discussions around blockers.

I wanted to open the discussion around adding this support as some changes would be needed to the build & release processes to handle both architectures. Mostly, I wanted to see what help I could be with that process, to document my findings so far, and contribute PRs/fixes for issues in the compilation process where possible.

Why would it be useful?

arm64-based AWS Lambda function invocation is notably cheaper than x86_64, but also having an arm64 version of Chromium allows for situations such as where emulation of x86_64 isn't possible or problematic on ARM CPUs (e.g. local invocation of functions via AWS SAM).

Other info

I'm currently running the Ansible playbook from this repo against both c6i.16xlarge and c6g.16xlarge instances for x86_64 and arm64 respectively. It's taken over an hour just to clone the Chromium source (at least 44GB of data), but I'll report the issues I'm encountering on either environment below.

Status so far

How/where is aws.tar.br built and updated?

x86_64 host

x64 build: successfully on AL2023 with current Ansible playbook
arm64 build: in progress
- Requires target_cpu=["x64", "arm"] to be added to .gclient configuration
- Requires gn to be set up like so gn gen out/Default --args="target_cpu=\"arm64\""
- autoninja -C out/Default chrome errors:
  - Requires export LDFLAGS="-z max-page-size=0x10000" due to Unable to build arm64 binaries (segfault during v8 bytecode_builtins_list_generator) envoyproxy/envoy#17100

ARM64/Gravitron host

Given the below and general notes at https://chromium.googlesource.com/chromium/src/+/main/docs/linux/chromium_arm.md, it appears that cross-compiling for ARM is the only option at present.

https://bugs.chromium.org/p/chromium/issues/detail?id=1468386

[ec2-user@host chromium]$ gclient sync --delete_unversioned_trees --revision 905e8bdd32d891451d94d1ec71682e989da2b0a1 --with_branch_head
Updating depot_tools...
'/srv/source/depot_tools/.cipd_bin/goma_ctl.py' was not found and may not be supported on this platform!
Syncing projects: 100% (157/157), done.

________ running 'cipd ensure -log-level error -root /srv/source/chromium -ensure-file /tmp/tmpxji4sdpz.ensure' in '.'
Errors:
  failed to resolve infra/rbe/client/linux-arm64@re_client_version:0.113.0.8b45b89-gomaip (line 8): no such package: infra/rbe/client/linux-arm64
Error: Command 'cipd ensure -log-level error -root /srv/source/chromium -ensure-file /tmp/tmpxji4sdpz.ensure' returned non-zero exit status 1
Errors:
  failed to resolve infra/rbe/client/linux-arm64@re_client_version:0.113.0.8b45b89-gomaip (line 8): no such package: infra/rbe/client/linux-arm64

https://groups.google.com/a/chromium.org/g/chromium-discuss/c/M7MPEeu4kFA

[ec2-user@host chromium]$ gclient runhooks
Updating depot_tools...
'/srv/source/depot_tools/.cipd_bin/goma_ctl.py' was not found and may not be supported on this platform!
Hook 'vpython3 -vpython-spec src/.vpython3 -vpython-tool install' took 26.87 secs
Running hooks:   3% ( 6/170) nacltools
________ running 'python3 src/build/download_nacl_toolchains.py --mode nacl_core_sdk sync --extract' in '/srv/source/chromium'
Traceback (most recent call last):
  File "/srv/source/chromium/src/build/download_nacl_toolchains.py", line 55, in <module>
    sys.exit(Main(sys.argv[1:]))
  File "/srv/source/chromium/src/build/download_nacl_toolchains.py", line 51, in Main
    return package_version.main(args)
  File "/srv/source/chromium/src/native_client/build/package_version/package_version.py", line 1290, in main
    arguments = ParseArgs(args)
  File "/srv/source/chromium/src/native_client/build/package_version/package_version.py", line 1118, in ParseArgs
    host_arch = pynacl.platform.GetArch3264()
  File "/srv/source/chromium/src/native_client/pynacl/platform.py", line 133, in GetArch3264
    assert machine in ARCH3264_DICT, "Unrecognized arch machine: %s" % machine
AssertionError: Unrecognized arch machine: aarch64
Error: Command 'python3 src/build/download_nacl_toolchains.py --mode nacl_core_sdk sync --extract' returned non-zero exit status 1 in /srv/source/chromium
Traceback (most recent call last):
  File "/srv/source/chromium/src/build/download_nacl_toolchains.py", line 55, in <module>
    sys.exit(Main(sys.argv[1:]))
  File "/srv/source/chromium/src/build/download_nacl_toolchains.py", line 51, in Main
    return package_version.main(args)
  File "/srv/source/chromium/src/native_client/build/package_version/package_version.py", line 1290, in main
    arguments = ParseArgs(args)
  File "/srv/source/chromium/src/native_client/build/package_version/package_version.py", line 1118, in ParseArgs
    host_arch = pynacl.platform.GetArch3264()
  File "/srv/source/chromium/src/native_client/pynacl/platform.py", line 133, in GetArch3264
    assert machine in ARCH3264_DICT, "Unrecognized arch machine: %s" % machine
AssertionError: Unrecognized arch machine: aarch64

The text was updated successfully, but these errors were encountered:

joergbaier · 2023-11-28T20:35:24Z

Didn't check what the github actions for this repo are doing, but this could also help in the future
https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/

Sparticuz · 2023-11-30T18:59:03Z

A few notes.

Github Actions costs too much, $15 per compile from what I've heard from others. AWS is much cheaper at a buck or two.
Yes, I believe that graviton support should be possible now.
- probably will need to be it's own package (@sparticuz/chromium-arm64 i guess?)
here is my previous attempt Chromium for arm64 chrome-aws-lambda#11 much of this can be pulled in
aws.tar.br is hand generated. I basically booted up a provided.al2 docker container and copied the files out of it. This should be generated, which should probably be it's own PR. With al2023 needing it's own files, aws.tar should probably be renamed to al2.tar
It will only support node20+

davidjb · 2023-12-01T00:47:23Z

Thanks for your insight, @Sparticuz. I'll aim to circle back around to looking at this -- I hadencountered issues during the compilation process (described above) and needed to push on with another way forward just for now. Cross-compiling for arm64 is the only option as I've discovered due to a lack of support in Chromium's build processes, but I also encountered issues during the cross-compile on c6i.16xlarge with AL2023.

For now I just wanted to list out the list of packages and libraries that Chromium needs at runtime at least (I'm using the chromium arm64 build from Playwright and that's working well on AL2023). I've got the following building a Layer via SAM, which is running the 'build' in a public.ecr.aws/sam/build-provided.al2023:latest-arm64 container:

dnf install -y \
  alsa-lib \
  at-spi2-atk \
  at-spi2-core \
  atk \
  avahi-libs \
  cairo \
  cups-libs \
  dbus-libs \
  expat \
  freetype \
  fontconfig \
  fribidi \
  google-noto-sans-vf-fonts \
  google-noto-serif-vf-fonts \
  graphite2 \
  gnutls \
  harfbuzz \
  libX11 \
  libXau \
  libXcomposite \
  libXcomposite \
  libXdamage \
  libXext \
  libXfixes \
  libXi \
  libXrandr \
  libXrender \
  libbrotli \
  libdatrie \
  libdrm \
  libgudev \
  libpng \
  libthai \
  libwayland-server \
  libxcb \
  libxkbcommon \
  mesa-libgbm \
  nettle \
  nspr \
  nss \
  nss-util \
  p11-kit-trust \
  pango \
  pixman \
  systemd-libs

mkdir -p "/tmp/lib"
cp -P /usr/lib64/{\
  libasound,\
  libatk-bridge-2.0,\
  libatspi,\
  libatk-1.0,\
  libavahi-client,\
  libavahi-common,\
  libcairo,\
  libcups,\
  libdbus-1,\
  libexpat,\
  libfreetype,\
  libfontconfig,\
  libfribidi,\
  libgraphite2,\
  libgnutls,\
  libharfbuzz,\
  libX11,\
  libXau,\
  libXcomposite,\
  libXdamage,\
  libXext,\
  libXfixes,\
  libXi,\
  libXrandr,\
  libXrender,\
  libbrotlicommon,\
  libbrotlidec,\
  libdatrie,\
  libdrm,\
  libudev,\
  libpng16,\
  libthai,\
  libwayland-server,\
  libxcb,\
  libxcb-render,\
  libxcb-shm,\
  libxkbcommon,\
  libgbm,\
  libnettle,\
  libhogweed,\
  libnspr4,\
  libplc4,\
  libplds4,\
  libnss3,\
  libsmime3,\
  libssl3,\
  libnssutil3,\
  libsoftokn3,\
  libfreeblpriv3,\
  libnssckbi,\
  libpango-1.0,\
  libpixman-1,\
  libsystemd\
  }.* "/tmp/lib"

set -e; \
  cd /tmp/lib; \
  tar -cf al2023.tar *; \
  brotli --best --force al2023.tar

tuke307 · 2024-01-23T17:30:08Z

any updates on this? ARM support would be very nice

almeidapaulooliveira · 2024-02-01T18:53:54Z

+1

luishgp · 2024-02-07T14:19:07Z

+1

anaet-aaron · 2024-02-19T23:50:01Z

+1

n-lam · 2024-03-17T11:24:56Z

+1

cjnoname · 2024-03-25T22:48:55Z

+1

async-lambda · 2024-04-10T02:38:46Z

+1

JonnyBurger · 2024-04-26T12:58:30Z

I seem to have managed to compile a binary for ARM64 AWS Lambda https://github.com/remotion-dev/chrome-build-instructions/blob/main/V2.md

Hope this is helpful, probably some more work is required to get it working with this project.

Sparticuz · 2024-04-26T15:08:07Z

I seem to have managed to compile a binary for ARM64 AWS Lambda https://github.com/remotion-dev/chrome-build-instructions/blob/main/V2.md

Hope this is helpful, probably some more work is required to get it working with this project.

Awesome! I am traveling the next few weeks, but will try to look at this asap!!

ostaplisovyj · 2024-06-08T05:35:30Z

Any updates on this issue? I would really appreciate having Chromium available at least for local sam invocations, without x86_64 emulation.

stephankaag · 2024-06-28T14:46:13Z

I was able to compile Chromium for AWS Lambda on the ARM64 architecture ~~(only Amazon Linux 2 for now, will build for Amazon Linux 2023 asap).~~

Amazon Linux 2 (NodeJS 16 & NodeJS 18):
Download Chromium 120 (For Playwright v1.40.1 , v1.40.0 & Puppeteer v21.8.0)
Download Chromium 121 (For Playwright v1.41.2 , v1.41.1 , v1.41.0 & Puppeteer v22.1.0 , v22.0.0 , v21.11.0 , v21.10.0 , v21.9.0)

Amazon Linux 2023 (NodeJS 20):
Download Chromium 120 (For Playwright v1.40.1 , v1.40.0 & Puppeteer v21.8.0)
Download Chromium 121 (For Playwright v1.41.2 , v1.41.1 , v1.41.0 & Puppeteer v22.1.0 , v22.0.0 , v21.11.0 , v21.10.0 , v21.9.0)

cjnoname · 2024-08-05T10:50:03Z

any updates on this? :)

davidizac · 2024-09-10T11:51:32Z

Any updates on this @Sparticuz ?

LorenzoRogai · 2024-09-10T15:14:22Z

Is there any another block now for building binaries in arm64? ARM64 GitHub runners are now in GA and they cost even less compared to x86

davidjb added the enhancement New feature or request label Nov 16, 2023

Sparticuz pinned this issue Nov 30, 2023

Sparticuz mentioned this issue Mar 4, 2024

[BUG] Failed to launch the browser process! /tmp/chromium: /tmp/chromium: cannot execute binary file #235

Closed

hexi-verifiet mentioned this issue Sep 26, 2024

Layer with Python runtime support shelfio/chrome-aws-lambda-layer#63

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[REQUEST] arm64 support #186

[REQUEST] arm64 support #186

davidjb commented Nov 16, 2023 •

edited

Loading

joergbaier commented Nov 28, 2023

Sparticuz commented Nov 30, 2023

davidjb commented Dec 1, 2023

tuke307 commented Jan 23, 2024

almeidapaulooliveira commented Feb 1, 2024

luishgp commented Feb 7, 2024

anaet-aaron commented Feb 19, 2024

n-lam commented Mar 17, 2024

cjnoname commented Mar 25, 2024

async-lambda commented Apr 10, 2024

JonnyBurger commented Apr 26, 2024

Sparticuz commented Apr 26, 2024

ostaplisovyj commented Jun 8, 2024

stephankaag commented Jun 28, 2024 •

edited

Loading

cjnoname commented Aug 5, 2024

davidizac commented Sep 10, 2024 •

edited

Loading

LorenzoRogai commented Sep 10, 2024 •

edited

Loading

[REQUEST] arm64 support #186

[REQUEST] arm64 support #186

Comments

davidjb commented Nov 16, 2023 • edited Loading

What would you like to have implemented?

Why would it be useful?

Other info

Status so far

x86_64 host

ARM64/Gravitron host

joergbaier commented Nov 28, 2023

Sparticuz commented Nov 30, 2023

davidjb commented Dec 1, 2023

tuke307 commented Jan 23, 2024

almeidapaulooliveira commented Feb 1, 2024

luishgp commented Feb 7, 2024

anaet-aaron commented Feb 19, 2024

n-lam commented Mar 17, 2024

cjnoname commented Mar 25, 2024

async-lambda commented Apr 10, 2024

JonnyBurger commented Apr 26, 2024

Sparticuz commented Apr 26, 2024

ostaplisovyj commented Jun 8, 2024

stephankaag commented Jun 28, 2024 • edited Loading

cjnoname commented Aug 5, 2024

davidizac commented Sep 10, 2024 • edited Loading

LorenzoRogai commented Sep 10, 2024 • edited Loading

davidjb commented Nov 16, 2023 •

edited

Loading

stephankaag commented Jun 28, 2024 •

edited

Loading

davidizac commented Sep 10, 2024 •

edited

Loading

LorenzoRogai commented Sep 10, 2024 •

edited

Loading