DTrace, FreeBSD 9.0, and Erlang
While working on the DTrace probes for the Erlang R15 release (which I’ve blogged about earlier this month), I discovered that the build recipe for FreeBSD 9.0 was broken. Oh no, my favorite OS doesn’t work, sound the alarm!
FreeBSD has supported kernel-space DTrace probes for a while. However, support for user-space probes, USDT, has not arrived until FreeBSD 9.0RC1. ”RC1” means “Release Candidate #1”. As of this writing, RC2 is the newest release. The official release of FreeBSD 9.0 (to be called FreeBSD 9.0-RELEASE) will not be ready for at least another month or perhaps longer; see the 9.0 wiki release schedule for more details.
As I mentioned in the introduction to this article, USDT probes are a bit broken in FreeBSD 9.0RC2. I’ve added two different kinds of probes to the Erlang virtual machine:
1. Probes that are defined as part of the core virtual machine. These probes fire in response to events within the virtual machine, e.g. spawning a new process or a garbage collection event. These probes are defined in [erlang_dtrace.d](https://github.com/slfritchie/otp/blob/dtrace-review3/erts/emulator/beam/erlang_dtrace.d).
2. A probe that can be triggered directly by Erlang code, e.g., `dtrace:p(42, "Hello, world!").` This probe is defined in [dtrace_user.d](https://github.com/slfritchie/otp/blob/dtrace-review3/lib/dtrace/c_src/dtrace_user.d) and by the Erlang module [dtrace.erl](https://github.com/slfritchie/otp/blob/dtrace-review3/lib/dtrace/src/dtrace.erl).
The probe that can be fired directly from Erlang code must be compiled using position-independent code (PIC) and assembled into a shared library. The Erlang function dtrace:init() will load the shared library into the virtual machine and initialize the Erlang<->C glue necessary for an Erlang function to call a C function.
Both FreeBSD 9.0RC1 and FreeBSD 9.0RC2 cannot compile and link to PIC code. So all of the probes in erlang_dtrace.d will work correctly (because they’re not part of a shared library), but the probe in dtrace_user.d will not work correctly.
There is a problem report (PR) for FreeBSD that discusses this PIC problem: kern/159046: [dtrace] [patch] dtrace library is linked with a wrong flags on -CURRENT. I emailed an update to that ticket this afternoon, asking that Alex Samorukov’s fix be folded into the 9.0-RELEASE of FreeBSD. I have no idea if it will happen in time or not … but I have my fingers crossed.
In the meantime, I have a fix for those who are using one of the FreeBSD release candidates where the PIC bug is still present. You have a couple of options:
1. If you're familiar with building & installing FreeBSD kernels and FreeBSD user-space programs, then following the advice in the FreeBSD problem report (linked above) will work just fine.
2. Download PIC-compiled copies of the system libraries that are required for PIC-compiled trace code and install them manually.
If you choose option #2, then I have a step-by-step recipe for you:
* Download the 5.5 KByte file [http://www.snookles.com/scott/misc/freebsd-9.0rcX.dtrace-pic-libs.tar.gz](http://www.snookles.com/scott/misc/freebsd-9.0rcX.dtrace-pic-libs.tar.gz) to your FreeBSD 9.0RCx's `/tmp` directory.
* Run this command: `gunzip -c /tmp/freebsd-9.0rcX.dtrace-pic-libs.tar.gz | (cd / ; tar xvfp -)`
Good luck, and have fun with DTrace.
UPDATE 2011-Nov-29
It looks like the FreeBSD problem report has now been resolved, and the patch will be included in the 9.0-RELEASE distribution when it is released in the next month or three. Very good news!