Docker cross-platform Elixir build for linux/amd64 on linux/arm64, Apple Silicon M1, M2, M3, etc.

Problem

When building an Elixir application docker image on an arm64 platform, the process crashes with a variety of errors:

• Segmentation fault
• Error code 139
• Error code 143
• Memory request of some enormous size

Explanation

The issue is related to the interaction between the Erlang JIT compiler introduced in OTP25, and QEMU virtualization. In short, QEMU does not detect a scenario where code gets replaced by JIT.

Solution

The Dockerfile should include this line before any mix operations:

ENV ERL_FLAGS="+JPperf true"

This has the effect of disabling the double-memory mapping of the JIT-generated code, which works around the QEMU's behavior and fixes the crash. This also includes the Linux perf support by providing the dump and map symbols in separate files under tmp/ directory: jit-*.dump and perf-*.map. The whole tmp/ dir can be omitted when creating the image by including it in .dockerignore file, so it does not affect the resulting image size.

Example

Dockerfile

...
FROM elixir:1.14.1-alpine as builder
ARG MIX_ENV
ENV MIX_ENV=$MIX_ENV

# Sets the ERL_FLAGS conditionally when the DOCKER_DEFAULT_PLATFORM is present.
ARG DOCKER_DEFAULT_PLATFORM
ENV ERL_FLAGS=${DOCKER_DEFAULT_PLATFORM:+'+JPperf true'}

RUN apk add --no-cache \
    gcc \
    git \
    make \
    musl-dev

RUN mix local.rebar --force && \
    mix local.hex --force
...

docker-compose.yml

...
  backend:
    image: "backend:${BUILD_VERSION}"
    build:
      context: .
      args:
        MIX_ENV: prod
        # Substituting with null in case of missing variable to silence the warning
        # in the on-demand scenario. Remove the susbtitution in a static setup.
        DOCKER_DEFAULT_PLATFORM: ${DOCKER_DEFAULT_PLATFORM-}
...

The DOCKER_DEFAULT_PLATFORM variable can be specified inline or exported in the shell. This way the image can be built as needed for running locally on arm64, or for deployment to a linux/amd64.

DOCKER_DEFAULT_PLATFORM="linux/amd64" docker compose build backend

Sources

1. Mix deps.get memory explosion when doing cross-platform Docker build
2. Erlang erl +JPperf docs
3. erts: Decouple use of single-mapped memory from perf support
4. Variable Substitution Handy Dandy Reference Table