Building for Raspberry Pi using distcc

Building locally on a Raspberry Pi can be slow. The most common way around this is to cross-compile, which is much quicker, but requires more setup.¹ There is also a third alternative, distcc, which distributes the compilation to remote computers in a transparent manner. Setting up distcc is somewhat simpler compared to cross-compiling, but we still gain speed compared to local build.

The distcc program can distribute builds across several machines on a network.² Today we will setup and compile CANlib on an old Raspberry Pi Model B running Rasbian 8.0 using distcc to distribute the compilation onto a Linux computer running Ubuntu 16.04.

First we need to set up and configure our Ubuntu host machine.

Using apt we install the required packages distcc and git.

The Raspberry Pi Foundation is providing a ready-to-use toolchain in their GitHub repository. Create a directory and fetch the toolchain from GitHub. We are only interested in the latest revision and not the whole history so we use the argument --depth 1 .

Now we have to make sure that when distcc calls our compiler, it should use our cross compiler. To do so we create symbolic links to the compilers.

We edit the distcc configurations file found in /etc/default/distcc.

1. Set STARTDISTCC to “true” in order to start the distcc daemon, distccd, at boot.
2. Change ALLOWEDNETS to allow the network or individual computers to connect. We just allow our Raspberry Pi to connect so we set its ip-address here.
3. Comment out the row containing ‘LISTENER=”127.0.0.1″‘, otherwise distcc will only listen on the loopback interface.
4. Set maximum allowed tasks using JOBS. You can get the number of available processors on your computer by using the nproc command. On our machine we wanted to limit the number of jobs to two.

The edited configuration file at /etc/default/distcc will now look like the following (comments omitted):

Now we need to make sure that distcc finds and uses our toolchain. We do this by including the path to the cross toolchain first in the PATH variable in /etc/init.d/distcc. Our user is named ‘ubu’ on the Ubuntu host so the line becomes:

Reload systemd, since we made changes to the init script, and start distccd.

We can now set up and configure our Raspberry Pi.

Using apt we first install the required packages. bc is required to be able to execute make prepare on the kernel.

Create symbolic links to distcc.

Setup the distcc environment variables. Note that you will have to do this at every login, unless you put them in /etc/profile.

1. Set the DISTCC_HOSTS environment variable to a space-delimited list of host machines. We just add our single host using 192.168.0.2, which is the IP-address of the host machine.
2. If you do not want to compile anything locally, set the environment variable “DISTCC_SKIP_LOCAL_RETRY”= to 1

How to find out what version of the Linux headers to use is described in the blog article Building CANlib on Raspberry Pi.

How to download and extract the kernel source is done in the same way as is described in the blog article Building CANlib on Raspberry Pi.

Now we can setup kernel configuration and build the headers in exactly the same way as is described in the blog article Building CANlib on Raspberry Pi. However, this time we will now use the distcc toolchain and send the compilation job to our Ubuntu host.

The two make commands now took about two minutes, only one minute less than building locally, but still about 30% reduction in time.

The host uses the following versions:

The client, a Raspberry Pi Model B is using the following versions:

1 For a description on how to build the kernel for a Raspberry Pi, click here.

2 distcc documentation can be found here.