Patching the Dynamic Linker/Loader of a Compiled Binary using patchelf

Given a foreign binary on a Nix system, you have to patch its dynamic linker/loader.

First you need to find out what linker/loader to use.

You can do this by going into your nix repl and loading your desired nixpkgs.

Then you run:

"${pkgs.glibc}/lib64/ld-linux-x86-64.so.2"

Which will return you a path that looks like this:

/nix/store/d54amiggq6bw23jw6mdsgamvs6v1g3bh-glibc-2.25-123/lib64/ld-linux-x86-64.so.2

You can then use:

patchelf --set-interpreter '/nix/store/d54amiggq6bw23jw6mdsgamvs6v1g3bh-glibc-2.25-123/lib64/ld-linux-x86-64.so.2' ./binarytobepatched

Or you can run it directly:

/nix/store/d54amiggq6bw23jw6mdsgamvs6v1g3bh-glibc-2.25-123/lib64/ld-linux-x86-64.so.2 ./binarytobepatched

Note that if you are doing this inside a Nix expression, you should be instead using:

$(cat $NIX_CC/nix-support/dynamic-linker)

Note that if you use ldd on NixOS on an unpatched binary it will probably show:

        /lib64/ld-linux-x86-64.so.2 => /nix/store/d54amiggq6bw23jw6mdsgamvs6v1g3bh-glibc-2.25-123/lib64/ld-linux-x86-64.so.2 (0x00007f04fcc29000)

That ld-linux-x86-64.so.2 path is derived from the Glibc that ldd itself was built with.

There is a program called ld-linux-x86-64.so.2. This is the "dynamic linker/loader". It is the interpreter of compiled binary programs. https://linux.die.net/man/8/ld-linux.so

# finds the 32 bit dynamic linker and loader
ls /nix/store/*glibc*/*/ld-linux.so.2
# finds the 64 bit dynamic linker and loader
ls /nix/store/*glibc*/*/ld-linux-x86-64.so.2

When you are in a nix-shell or a nix build environment, it's easier to do this:

cat $NIX_CC/nix-support/dynamic-linker

This program is not usually put on the PATH, but you running it directly will give you options that allow you to specialise the execution of ELF executables on Linux operating systems.

According to the FHS, the designated locations for the dynamic linker and loader are:

# for 32 bit
/lib/ld-linux.so.2
# for 64 bit
/lib64/ld-linux-x86-64.so.2

This is why for portable Linux binaries, you generally will want the interpreter path set to one of those paths depending on if you are producing 32 bit binary or 64 bit binary. In fact this is the case for the nodejs executables when downloaded directly off https://nodejs.org.

However not all distros follow this process. NixOS is one of them that does not have anything in /lib nor /lib64.

On NixOS, binaries are expected to have their interpreter path set directly to one of the paths within the /nix/store. Furthermore, binaries are expected to have their DT_RUNPATH also set to relevant library paths to the /nix/store.

This is why so-called portable binaries do not run on NixOS normally, give a weird error message when you execute them:

bash: ./result: No such file or directory

Portable binaries will generally not have the DT_RUNPATH set, because they would expect the operating system to have the correctly configured search paths either via LD_LIBRARY_PATH or the ld.so.cache or finally compiled "system search path" that is embedded into the dynamic linker & loader.

When you use ldd on a binary executable, ldd knows how to find the ld-linux.so so when it shows up, that won't affect the actual execution of the program.

Ok so the easiest way to run a non-NixOS executable in this case is to do these things:

$(cat $NIX_CC/nix-support/dynamic-linker) --library-path '<setsomepath>:<setsomepath>' ./binary

  postFixup = ''
    patchelf \
      --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" \
      --set-rpath "${lib.makeLibraryPath [ stdenv.cc.cc.lib glibc ]}" \
      $out/bin/binary
  '';

In our case, our postFixup would then do the opposite, rather than setting the interpreter and path to NixOS, it would be:

postFixup = ''
  patchelf --remove-rpath ./prebuilds/linux-x64/node.napi.node
'';

CMCDragonkai/patching_the_dynamic_linker_loader_of_a_compiled_binary_using_patchelf.md

Select an option

No results found

Select an option

No results found

Patching the Dynamic Linker/Loader of a Compiled Binary using patchelf

CMCDragonkai commented May 12, 2024

Uh oh!