FDT overlay is an extension to FDT format that lets user to modify base FDT run-time: add new nodes, add new properties to existing nodes or modify existing properties. It’s useful when you have base board and some extension units like cape/shield for Pi/BBB or loadable FPGA logic for Zynq. I will not go into details you can find internals described on Adafruit or Raspberry Pi websites.

When dealing with overlays there are two options where to handle them: loader or kernel. Managing overlays at kernel level gives more flexibility but requires more related logic, e.g. re-init pinmux after applying overlay, re-run newbus probe/attach. On the other hand loader-level support is quite straightforward and involves nothing but DTB modifications and it’s a natural first step to adding FDT overlays to FreeBSD.

Proposed solution is to add fdt_overlays variable that contains coma-separated list of dtbo files, e.g.: “bbb-no-hdmi.dtbo,bbb-4dcape-43.dtbo”. This variable can be defined either as a loader(8) variable or as a u-boot env variable. During the boot ubldr load base DTB and right before passing control to the kernel it would go through files, load them from /boot/dtb/ direсtory on root partition and apply to the base blob. Final DTB would be passed to kernel.

You can find patch and review comments to it on Differential site: D3180. It contains:
- Extension to dtc to generate dynamic symbols and fixup info.
- ubldr fdt_overlays support

As Warner Losh mentioned it’s not clear yet how to deal with dynamic symbols support patch. It’s not part of official dtc tree though it’s accepted by RPi and BBB communities.

Just quick (and overdue) update: HDMI for BBB is in HEAD as of r284534.

HDMI support for Beaglebone Black is stable now and supports reading EDID, you can get the path here. Before committing it I’d like to make interoperability between HDMI framer and FB/LCD drivers as generic as possible and for this I need at least one more system with working HDMI to find common patterns. For this purpose I picked up i.MX6-based Hummingboard and now try to get video output working on it. There is some minor progress but it seems before getting to HDMI/IPU I need to do some work on clock management part of the system. So it’s going to be some time before I see first pixels on my monitor.

I also got 4DCAPE-43, neat Beaglebone Black LCD cape by 4D Systems. Patch (download here) for it is pretty minimal: VT support in kernel config, panel info and pins configuration in dts, and one improvement in GPIO driver (setting default values for OUT GPIO pins). None of the other features except reset button work yet but getting GPIO keyboard working would be an interesting project by itself.

Here is the picture of cape in action:


Today for the first time I’ve got stable and correctly positioned output on HDMI monitor connected to BeagleBone Black. It involved fixing bug in AM335x LCDC controller, fixing bug in I2C controller, and a lot of experiments with register-pushing. Code requires major clean-up and is not ready for the tree yet. I’ll post patch when it’s in readable form.


Qt5 for FreeBSD/Pi

January 14, 2015 — Leave a comment

Build SD card image using crochet-freebsd with option VideoCore enabled. Mount either SD card itself of image to build host

mount /dev/mmcsd0s2a /pi

Checkout Qt5 sources and patch them

cd /src
git clone git://gitorious.org/qt/qt5.git qt5
cd qt5
git checkout 5.4.0
./init-repository --module-subset=$MODULES

fetch -q -o - http://people.freebsd.org/~gonzo/arm/rpi/qt5-freebsd-pi.diff | patch -p1

Configure, build and install Qt5 to SD card

./configure -platform unsupported/freebsd-clang -no-openssl -opengl es2 -device freebsd-rasp-pi-clang -device-option CROSS_COMPILE=/usr/armv6-freebsd/usr/bin/ -sysroot /pi/ -no-gcc-sysroot -opensource -confirm-license -optimized-qmake -release -prefix /usr/local/Qt5 -no-pch -nomake tests -nomake examples -plugin-sql-sqlite

gmake -j `sysctl -n hw.ncpu`
sudo gmake install

You need BSD-specific plugins to enable mouse and keyboard input in EGLFS mode

cd /src/
git clone https://github.com/gonzoua/qt5-bsd-input.git
cd qt5-bsd-input
sudo gmake install

Build application you’d like run and install it. I use one of the examples here

cd /src/qt5/qtbase/examples/opengl/cube
sudo gmake install

Unmount SD card, boot Pi, make sure vchiq is loaded

root@raspberry-pi:~ # kldload

Start application

root@raspberry-pi:~ # /usr/local/Qt5/examples/opengl/cube/cube -plugin bsdkeyboard -plugin bsdsysmouse

If you see something like this:

EGL Error : Could not create the egl surface: error = 0x3003

Or this:

QOpenGLFramebufferObject: Framebuffer incomplete attachment.

It means you need to increase GPU memory by setting gpu_mem in config.txt. Amount depends on framebuffer resolution. 128Mb works for me on 1920×1080 display.

bsdsysmouse plugin uses /dev/sysmouse by default, so you either should have moused running or specify actual mouse device, e.g.:

root@raspberry-pi:~ # cube -plugin bsdkeyboard -plugin bsdsysmouse:/dev/ums0

bsdkeyboard uses STDIN as input device, so if you’re trying to start app from serial console it should be something like this:

root@raspberry-pi:~ # cube -plugin bsdkeyboard -plugin bsdsysmouse < /dev/ttyv0

I’ve update ioquake binaries and pushed respective changes to github. New version has support for mouse and keyboard so you can actually play Quake3 on FreeBSD/Pi alas without sound. It should run out of the box on normal console. Make sure you have moused running or specify mouse device by setting Q_MOUSE_DEV environment variable, e.g.:

env Q_MOUSE_DEV=/dev/ums0 ioquake3.arm +set s_initsound 0

Audio on Raspberry Pi

January 9, 2015 — Leave a comment

With stable VCHIQ driver next obvious target was to add VCHIQ-based audio support. So let me introduce to you: vchiq_audio, first take. It’s part of vchiq-freebsd repo so if you use Crochet to build SD card image just enable option VideoCore in config file and module will be automatically included.

From shell run kldload vchiq_audio and you’re good to do. I believe that audio output is picked up automatically by VideoCore so if you have HDMI connected it’s probably going to be HDMI. I do not have device to confirm this. Adding knob to control audio output (auto, headphones, HDMI) is on my ToDo list.

Quality is not ideal though. From quick tests it seems to work fine on system with rootfs on NFS but there are audio drops on SD-based system while playing mp3 over NFS. I’m going to debug and stresstest it more thoroughly next week.

Short instruction on how to install mpg321 package on RPi:

env PACKAGESITE=http://chips.ysv.freebsd.org/packages/11armv6-11armv6/ SIGNATURE_TYPE=none pkg bootstrap

mkdir -p /usr/local/etc/pkg/repos
cd /usr/local/etc/pkg/repos
echo 'FreeBSD: { enabled: no }' > FreeBSD.conf

cat > chips.ysv.conf <<__EOF__
chips.ysv: {
  url: "http://chips.ysv.freebsd.org/packages/freebsd:11:armv6:32:el:eabi:softfp",
  mirror_type: "http",
  signature_type: "none",
  enabled: yes

pkg install mpg321

Update: support for keyboard/mouse has been added

After New Year I got back to hacking the VCHIQ stuff (thanks to adrian@ for prodding). Since last time I touched NetBSD folks got it merged to main tree, syncing with latest upstream code and fixing some stupid bugs in my codebase. So I partially merged things back, spent some time on fixing more bugs introduced by yours truly, merged userland bits from latest Broadcom’s bits (and fixing some bugs introduced by them). And as a result VCHIQ got stable enough to run ioquake3d on raspberry pi. Well, you can’t play it because there is no sound and no mouse support and keyboard support is severely crippled but you can navigate menus and watch demoes.

Here is short summary of how to get it running:

  • Get latest HEAD that includes r276794
  • Get latest crochet-freebsd
  • Create configuration file for RasspberryPi, make sure that it’s configured for 2Gb SD card and has VideoCore enabled. i.e. it contains:
    option ImageSize 1950mb # for 2 Gigabyte card
    option VideoCore
  • Build RPi image and flash to SD card
  • mount FreeBSD partition, e.g. mount /dev/mmcsd0s2a /mnt
  • Copy Quake3 PAK files to /baseq3 directory on SD card
  • Download http://people.freebsd.org/~gonzo/arm/ioquake3.arm.tar.gz and copy *.so files to /baseq3 and ioqake3.arm to /usr/bin on SD card
  • Unmount FreeBSD partition and mount boot partiotion, e.g. mount_msdosfs /dev/mmcsd0s1 /mnt
  • Edit config.txt and change gpu_mem value to 64
  • Unmount SD card and boot it on your Pi
  • Load vchiq module: kldload vchiq
  • Start Quake3: ioqake3.arm +set s_initsound 0

Keyboard support is really broken. TAB and ENTER works, so you can navigate menus. But that’s pretty much it.

ioquake3 codebase with my minor changes located here: https://github.com/gonzoua/quake3
I provide pre-compiled binaries because for some reason ioquake3 built with xdev tools crash in qsort (libc incompatibilities?) so I use make buildenv to build it.

And here is photo of demo in action (there are RaspberryPi and ZedBoard on it too, yay!)
Photo Jan 07

A week ago Adrian Chadd asked me to take a loot at FreeBSD/MIPS emulation. So last week I’ve been busy tidying up stuff in that department and looking up bits of information on various emulators. This morning I finally committed last changeset so now is the time to write up summary.


There are two widely used MIPS emulatoes that FreeBSD supports: QEMU and GXemul. Both of them support numerous MIPS devices but we’re interested in only two.

MALTA is more or less standard for MIPS emulation and supported by both emulators. QEMU supports 32 and 64 bit variants with both big and little-endian byte order. So four modes in total. Also for MALTA machine QEMU provides PCNet NIC emulation.

GXemul supports 32 and 64 bit modes of MALTA but only for little-endian byte order. Big-endian byte order is not supported due to incomplete PCI controller implementation. No NIC support for MALTA machine. Also Gxemul provides so-called oldtestmips emulation mode: generic implementation of abstract MIPS machine with very simplified NIC/disk devices. In theory it should be faster then actual hardware emulation but I haven’t got around to benchmarking yet. oldtestmips can be run in 32 and 64 bit mode, but only big-endian byte order is supported.

Disk images

I used raw disk images created by makefs(8) utility. The advantage of raw disk image is that they can be used with both emulators. qemu provides more options in this area but they’re out of scope of this article. I created four images for my tests: disk.img, disk64.img, diskel.img, and disk64el.img. 32-bit big-endian, 64-bit big-endian, 32-bit little-endian, 64-bit big-endian. The process looks somewhat like this script:

set -e

export TARGET=mips
export TARGET_ARCH=mips
export SRCCONF=/dev/null
export SRCROOT=/src/FreeBSD/head
export MAKEOBJDIRPREFIX=/src/FreeBSD/obj/head
export DESTDIR=/src/FreeBSD/tftproot/$TARGET_ARCH
make -C $SRCROOT buildworld

sudo -E mkdir -p /src/FreeBSD/tftproot/$TARGET_ARCH

# modify /etc/fstab and /etc/rc.conf in $DESTDIR

# Create 512Mb disk image with big-endian UFS
# For TARGET_ARCH set to mipsel or mips64el use "-B le" switch 
sudo -E makefs -M 538968064 -B be /src/FreeBSD/disk.img $DESTDIR

Change TARGET_ARCH and disk image name accordingly for mipsel/mip64/mips64el targets.


As of QEMU 1.6.0 there is one known problem with it: NIC emulation does not work in big-endian mode. See this patch for fix and details. Also, amount of memory limited to 128Mb. Use MALTA kernel config for 32-bit mode and MALTA64 for 64-bit mode.

Command lines for various modes/byte orders are:

qemu-system-mips -M malta -kernel /path/to/mips.mips/MALTA/.../kernel -hda /src/FreeBSD/disk.img -nographic
qemu-system-mips64 -M malta -kernel /path/to/mips.mips64/.../MALTA64/kernel -hda /src/FreeBSD/disk64.img -nographic
qemu-system-mipsel -M malta -kernel /path/to/mips.mipsel/.../MALTA/kernel -hda /src/FreeBSD/diskel.img -nographic
qemu-system-mips64el -M malta -kernel /path/to/mips.mips64el/.../MALTA64/kernel -hda /src/FreeBSD/disk64el.img -nographic


GXemul requires two patches for proper FreeBSD/MIPS emulation. First one implements rdhwr op for reg 29 required by TLS support. The second one fixes UDP packet checksum calculation and required for proper functioning of emulated DHCP server. Both of these patches have been committed upstream but there were no GXemul release after that so they’re not available as part of emulators/gxemul port.

As with QEMU kernel configs for MALTA are MALTA and MALTA64. And since only little-endian byte order is supported command lines for this emulation mode are:

gxemul -e malta -C 4Kc -d /home/gonzo/FreeBSD/diskel.img /path/to/mips.mipsel/.../MALTA/kernel
gxemul -e malta -d /home/gonzo/FreeBSD/disk64el.img /path/to/mips.mips64el/.../MALTA64/kernel

For oldtestmips emulation mode is other way around, only big-endian. And to make things more complicated it’s GXEMUL kernel config for 64-bit and GXEMUL32 for 32-bit. So here you are:

gxemul -M 256 -E oldtestmips -d /home/gonzo/FreeBSD/disk64.img /path/to/mips.mips64/.../GXEMUL/kernel
gxemul -M 256 -C 4Kc -E oldtestmips -d /home/gonzo/FreeBSD/disk.img /path/to/mips.mips/.../GXEMUL32/kernel

Maximum amount of memory set could be set by -M command-line switch for oldtestmips emulation is 256M. Otherwise physical memory would overlap with memory-mapped regions reserved by devices. There are some preliminary work to work around this limitation but no results yet.

QEMU support in FreeBSD/armv6 regressed since I tried it last time few months back. Changes in FreeBSD kernel and in QEMU itself revealed bugs that were masked by previous behaviour.

In FreeBSD it was r248467: the way memory/IO resources are activated on FDT bus has been changed and it triggered bug in versatile_pci.c

The other issue is more complex. It seems that PCI IRQ routing in QEMU was out of sync with real hardware. So after commit 66a96d7018b9cbabb73c9b87b62a37e4cc46580a IRQ numbers assigned to PCI devices by FreeBSD kernel by default were invalid. Authors of QEMU eventually added compatibility knob to fall back to previous logic. So if you’re using QEMU 1.5 or later add this option to your command line:

-global versatile_pci.broken-irq-mapping=1