Just moozing

Before you can check your notes, you must make them…

Raspberry pi – more fun

leave a comment »

I800px-rpi-logo-landscape-printn my last blog entry, I described how I mounted and changed files in a raspberry pi image and saved it again – all on my laptop. The real problems arose, when I wanted to actually run binaries and install programs. I have solved that now, and in this blog post I will describe how.

Last time I described the process of updating images. The modified version looks like this

  1. Download latest rasbian
  2. Extract
  3. Mount
  4. Chroot
  5. Run update script
  6. Unmount
  7. Zip the result

I have added the chroot step. This is the way we usually do things, when we e.g. have a USB stick or similar – we mount it, chroot, install stuff and unmount. This is insufficient since the architecture of the binaries on raspberry image is different than the architecture of the workstation, ie. ARM7 vs. amd64

I though I had to go through packer to do it, and I found this link. It turns out that packer is not needed and the project uses binfmt, which is a way of running binaries of one architecture on another.

This in turn led me to proot, which is just super cool and does all the stuff I need. It does chroot and binfmt all-in-one.

Using binfmt

Using Debian as an example:

apt-get install qemu binfmt-support qemu-user-static

Binfmt uses qemu behind the scenes to give a transparent execution of ARM binaries.

Mounting the raspberry image as previously described:

# fuse-ext2 -v part2.img mnt -o uid=0

# file /bin/hostname
/bin/hostname: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=aec4c0f94e43733828795f82729ad0fe4f578c4f, stripped

# mnt/bin/hostname
/lib/ld-linux-armhf.so.3: No such file or directory

The error is not that it is an arm binary – it is that it cannot find dynamically linked libraries – so it works.

The issues with the libraries may be solved by either 1) installing them on the host or 2) using chroot to change “top level” and have the dynamic linked libraries that was installed with the binary.

Solution 1) requires root privileges on the host and may work using multiarch. I have not tested this. It includes configurations and packages that I don’t want on my laptop.

Solution 2) also requires root privileges and that we manually mount ‘/proc’, ‘/sys’ and some of the system directories. It is both chroot and mount –bind that requires root permission. Proot bypasses this by relying “on ptrace, an unprivileged system-call available in every Linux kernel.”

Using proot

I ended up with a command that looked like this:

cp update_image.sh $MNTDIR/home/pi
proot -0 -R $MNTDIR -w / -q qemu-arm-static /bin/bash /home/pi/update_image.sh

Quick description of the paramters:

  • -0: become root inside the new root environment
  • -R $MNTDIR: user $MNTDIR as new root dir, and bind-mount relevant directories.
  • -w /: change to this directory on start. Do this or get warnings
  • -q qemu-arm-static: use this “filter” for translating all binaries inside
  • /bin/bash /home/pi/update_image.sh: the command to run inside

The reference manual is here.

The command above will run the update_image.sh script inside a chroot and exit when done – and in a completely transparent manner, the ARM binaries are run.


Putting it on gitlab.com

When the above is working it is fairly trivial to put it into gitlab.com and run it automatically on every push.

The repo is here. I suggest starting in the .gitlab.ci.yml file.

I have added caching to the CI build. It might be smart, or it might not be – I am unsure. It was added to reduce build time by reducing the amount of disk copying needed.



  • binfmt is worth knowing if you are a developer/tester that is making use of cross compilation, and wants to test the program on Intel architecture – it is often a lot simpler to get a lot of Intel architecture CPU processing power, than power in embedded systems.
  • proot is supercool and is in user-space only.
  • Next step is to see if I can use Ansible to configure the images. Maybe I can using connection=chroot.

Written by moozing

April 6, 2019 at 12:00

Posted in Tech

Tagged with , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: