Friday, 7 November 2014

Intel BayTrail - J1900

On of the main challengers to the current generation of ARM SOCs are the Intel BayTrail range. For me the interesting part of the family are E38XX (atom) and J1X00 (celeron) processors which boast  7-10W TDP. In this article I will cover the some initial performance metrics against the J1900 with the Intel Linux software stack. My test device was a low profile MX1900J industrial mini-itx board produced by BCM Advanced Research.

What's nice about the MX1900J :

1. Low profile with a heat sink that is approximately 15mm high.

2. On board DC power jack (12v) hence no need for a separate DC to DC converter board.

3. 4 x USB 3.0

4. Inclusion of LVDS and GPIO support.

5. Dual Ethernet NIC's

What's different about this board is the inclusion of a Display Port connector instead of HDMI along with VGA output. The BIOS has UEFI 64 and legacy support.

From an application developers view point there's quite a few advantages with the x86 platform. Firstly there is the vast amount of existing software that can run of the 'out of the box' or with minimal changes. Another is the shorter ramp up time between set up/configuration of the BSP/kernel/rootfs to actual application development. Lastly I would also argue that Intel do seem to devote a fair amount of resources to open source development therefore the underlying BSP have the potential to keep up with the latest trends (eg Chromium-ozone, Wayland, Tzien).

The J1900 GPU core supports Intel HD 4000 graphics and there are two linux graphics drivers available for the J1900. The lesser known of the two is the EMGD driver (Embedded Media and Graphics Driver) which are closed sourced binaries that are accessible through user space libraries eg libdrm, mesa and libva. The EMGD documentation targets these drivers for Fedora 18 against  Kernel 3.8 and xorg 13.1. Having previously used the EMGD drivers they can be ported to other Linux distros however problems may arise when upgrading or moving to newer  distro versions, where ABI breakages prevent this happening or cause stability issues. Intel prefer EMGD as they claim better 3D performance due to the Unified 3D (UFO) Driver.

The alternative to EMGD is the open source (Intel Linux Graphics) driver which can have better support for later kernels and hence usable on a later Linux distro. The downside may be a slight drop in overall performance and possibly stability. I chose to deploy the open source drivers against a very lightweight Ubuntu 14.04 image. The drivers provides OpenGL 3.3 and OpenGL ES 3.0.

The J1900 GPU core has 4 EU (Execution Units) combined with a maximum GPU frequency of 854Mhz. To given you have ideal of where the J1900 fits in the 'food chain' lets compare the FPS (frame per second) rates of running the WebGL Aquarium Demo on a imx6q, J1900 and an older 1037u celeron.

Number of Fish
1 50 100 500 1000
Platform Screen Resolution Frames Per Second
i.mx6q 1280x720 8 7 7 5 5
J1900 1680x1050 48 48 47 40 33
1037u 1680x1050 60 60 60 60 60

First lets be clear the above results are to be interpreted as a relative comparison. It should not be used as a primary marker for judging one platform to be superior to the other. Each platform has its merits based on a number of factors.  The i.mx6q as expected struggles (even at the lower resolution the rendering was not smooth) mainly due to the lower spec CPU/GPU core , an inefficient X driver and possibly some inefficient code paths in Chromium.  The older 1037u dual core celeron performs well due to the higher GPU frequency 1HGz and 6 EU, the trade off is a higher thermal output at 17W. What I couldn't easily account for was the drop off in the FPS rate at 1000 for the J1900. Below are the results from the BabylonJS train demo which is an intensive WebGL application supporting multiple camera angles, CAM_TRAIN being my favourite. What's interesting is that the FPS rate did not deviate when forcing Chromium to use EGL/GLES instead of OpenGL for the J1900.

Platform Screen Resolution Frames per second
J1900 1680x1050 13
1037u 1680x1050 24

Video playback is available through VAAPI plus libav with h.264/mpeg-2  hardware accelerated encoding/decoding. mplayer and gstreamer 1.0 support is readily available. CPU usage for decoding Big Buck Bunny at 1080p (H.264) was around 13% both in mplayer and using a simple gstreamer 1.0 pipeline. Decoding a 720p usb webcam at 30fps (YUY2) with the output encoded (H.264) to file and displayed to screen using a simple gstreamer pipeline resulted in 15% CPU usage.

Given the recent interest in HTML5 development for embedded platforms I  deployed a development build of Chromium (build 39). Chromium is fast becoming the web container of choice given the recent adoption of its engine in QT (QtWebEngine). HTML5test reported a healthy score of 512 out of 555 against Chromium.  A test HTML5 page with two embedded video files (playing concurrently) along a with a bunch of images (png) and static text ran without hiccup consuming 20% cpu. I briefly ran some demo HTML5 widgets from  zebra , webix and Kendo UI  again these ran smoothly. What should be possible with this platform is the ability to create a HTML5 GUI interface that could drive the rest of the application hosted on the same machine.

On the whole the results look very encouraging and the J1900 seems to offer a good trade off for a fan less solution with decent performance. Furthermore it should provide a relatively smooth route for application development. The main consideration is form factor and it is possible to find the J1900 in 3.5" SBC or Q7 form.

Saturday, 27 September 2014

i.MX6 Efficient font rendering and smoothing scrolling

Here is a short video demonstrating efficient font rendering and smoothing scrolling using OpenGL ES.

You may be forced to adopt this approach if your finding alternatives routes such as Qt5, HTML 5 or GTK+/Cario aren't giving you satisfactory results. The downside is that since you are starting from a low base (this a very low level implementation) it requires a far amount of development effort. On the positive side the code can be made to be as efficient as possible (to reduce power consumption/heat) and can be highly customised, in the above demo we control the screen from a remote PC.

Sunday, 27 July 2014

I.MX6 Developing with WebGL

On the new features in BSP 3.10.17 was support for WebGL. In a nutshell WebGL is javascript API supporting 3D and 2D graphics rendering, more about WebGL can be found here. WebGL is based on Open GL ES 2.0.

WebGL support opens up the possibility of developing and running graphical web based application on the i.mx6 within a browser. Furthermore there's also the possibility of the deploying a LAMP stack to serve the application from the i.mx6.  This opens up the possibility of developing simple games, kiosk applications, interaction user manuals/instructions and signage displays, etc..

To give you a taste of what is possible with the current WebGL implementation, I've put together a number of short videos that run existing WebGL demos/applications. These were run on lightweight Debian rootfs with Chromium browser (under X) on an i.mx6q board as part of a prototyping exercise. Chromium has been tweaked to maximize performance and the screen resolution was 720p (1280x720. Beware that not all WebGL application will run, some will fail because the Vivante libraries currently lack the 'Float Textures' extensions, others because the GPU is not powerful enough to give a decent FPS rate. Apologies for the quality of the videos. I'd advise that a i.mx6 quad processor is used to run WebGL as the examples consume 25-35% CPU under load.

The first video which also sums up what can be done on the i.mx6 with WebGL is the 'Lego build Chrome' demo. This application allows the creation of a Lego structure.

three.js is a javascript library which makes WebGL easier, here are some WebGL/CSS 3D examples which play nicely.

babylon.js is a 3D engine based on WebGL and javascript. The 'Create Your Own Shader' demo allows online editing of shaders.

CopperCube is an editor for creating 3D application, this is the 'Backyard Demo'.

Finally the "undulating-monkey" from aerotwist.

Saturday, 7 June 2014

I.MX6 Debian Jessie (GPU/VPU) 3.10.17_GA

Given that we a finally have a GA release of the 3.10.17 kernel and BSP from Freescale (although there also is a 3.10.31 alpha release in the pipleline). Here is an upgraded the Debian Jessie rootfs. You can download it from here and my previous 2 posts (here and here) cover installation and configurations step. Furthermore you require a kernel based on this release with the appropriate dts file for your i.mx6 device. This is a barebone rootfs with gpu/vpu support  and as previously stated I use this rootfs mainly for development. I haven't observed any major changes since the 3.10.17 beta release.

Friday, 11 April 2014

I.MX6 - gstreamer-imx and usb webcam support

Following on from my previous post about gstreamer-imx, this blog covers hooking a usb webcam using the gstreamer-imx plugins. It starts with creating simple pipeline for screen output, next is a pipeline for time lapse video recording. We then progress to streaming to VLC and finally implementing a simple rtsp server. Below is video demonstrating a simple RTSP server running on a AR6MXQ streaming to VLC. The webcam is pointing at the screen and streams the output (720p @10fps). We launch a VLC client window which is located in the lower right side of the same screen (unfortunately this causes an echo effect). As you will see any activity on the screen is streamed to the VLC client window (lower right) albeit with a delay.

The webcam in question was a microsoft HD-3000 which in theory is capable of 720p at 30fps. As with most webcams (which don't encode H264) it outputs YUYV (YUY2) and more importantly MJPEG. On linux you can easily determine what your webcam capabilities are by launching  v4l2-ctl, eg:

v4l2-ctl --list-formats-ext

This post provides a guide to what is possible with a gstreamer-imx plugins and the examples provided should not be treated as production ready. I'm assuming the examples will work on other usb webcams and or potentially other v4l2 devices. However the examples may require alterations depending on your webcam and it's capabilities, therefore background reading on gstreamer is recommended.

Testing was conducted on debian jessie and the target device was a AR6MXQ board provided by BCM Advanced Research. The examples should run on most imx6 devices as there is no device specific dependencies.

While testing with the HD-3000, the webcam  occasionally stop sending a stream if I configured the wrong resolution. The workaround was to unplug the device or reset the usb port  (replace '1-1.3' with the correct usb host and port id for your webcam) eg :

echo 1-1.3 > /sys/bus/usb/drivers/usb/unbind
echo 1-1.3 > /sys/bus/usb/drivers/usb/bind

Output to screen

Now back to the webcam, first step was getting output displayed to the screen. This requires reading the v4l2 source and converting the input so that is compatible with imxeglvivsink. Initial testing revealed decoding MPEG from the webcam performed significantly better than decoding YUY2. Another issue encountered (thanks to Carlos) was imxvpudec outputting Y42B something imeglvivsink currently can't cope with hence the inclusion of imxipuvideotransform element. So here is the final pipeline (webcam input is MPEG 720p at 30fps) :

gst-launch-1.0 v4l2src ! 'image/jpeg,width=1280,height=720,framerate=30/1' ! im
xvpudec ! imxipuvideotransform ! imxeglvivsink sync=false

CPU usage for the above was around 10%, while for YUY2 (720p at 10fps) it rises to 25% with this pipeline:

gst-launch-1.0 v4l2src ! 'video/x-raw,width=1280,height=720,framerate=10/1' ! imxipuvideotransform ! imxeglvivsink sync=false

Simple time lapsed video recording

Now lets implement a simple timelapsed video recorder that outputs to file and screen. I limited to the input MPEG stream to 10fps, reduced the bitrate  and encoded as h264 to reduce the output file size. Additional CPU load occurs due to the inclusion of the clockoverlay element, without the element it is difficult to know when the recording was taken.  Without this clockoverlay element CPU load is < 10%.

gst-launch-1.0 v4l2src ! 'image/jpeg,width=1280,height=720,framerate=10/1' ! imx
vpudec ! imxipuvideotransform ! clockoverlay time-format="%Y/%m/%d %H:%M:%S" ! tee name=splitter ! queue ! imxvpuenc_h264 bitrate=1024 ! filesink location=test .mp4 splitter. ! queue ! imxeglvivsink sync=false

The above pipeline generates approximately 400Mb per hour so probably not practical for production use.

Streaming to VLC

To enable streaming to VLC we need to create a 'sdp' file, this instruction VLC to act as a rtp server, below are the contents of the file:

m=video 5000 RTP/AVP 96
c=IN IP4
a=rtpmap:96 H264/90000

Save the contents to file eg 'imx6.sdp' and launch VLC,  because there is no h/w acceleration for VLC on the imx6, VLC was running on a PC :

vlc --no-audio imx6.sdp

On the imx6, we are a rtp client submitting h264 payloads to the VLC server (note the inclusion of the host/ip address of the PC) :

gst-launch-1.0 -v v4l2src ! 'image/jpeg,width=1280,height=720,framerate=10/1' !
imxvpudec ! imxipuvideotransform ! imxvpuenc_h264 bitrate=1024 ! rtph264pay ! udpsink host=<host/ip of PC> port=5000

Simple RTSP server

And lastly, lets try running an rtp server. Fortunately there is an additional gstreamer plugin (gst-rtsp-server) with rtp support that includes an example test server. The downside is that it needs to be built from sources. You will need to checkout the 1.1.90 tag and build (similar to gstreamer-imx). Once built we can launch the example test server and pass it a pipeline similar to that used when outputting to screen (you need to export the library path so that is found) eg:

export ./gst-rtsp-server/gst/rtsp-server/.libs
cd ./gst-rtsp-server/examples/.libs/

./test-launch '(v4l2src ! 'image/jpeg,width=1280,height=720,framerate=10/1' !imxvpudec ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay name=pay0 pt=96 )' 

The test server listens on port 8554 therefore the rtsp URL is

rtsp://<host/ip of imx6>:8554/test

You can test by launching VLC and opening a 'Network Stream' to the URL. While streaming the CPU load on the imx6 hovered around 60%. Given this is example code it should be possible to optimise the pipeline/code to bring down this figure.

Thursday, 10 April 2014

I.MX6 - gstreamer-imx

gstreamer-imx is set of gstreamer 1.0 imx6 video plugins principally developed by Carlos Giani a.k.a dv_ (great work!). Once stable these plugins "hopefully" will supersede the Freescale BSP gstreamers 0.10 plugins which are showing their age given that gstreamer 0.10 is longer maintained.

The main driver for this work was to get a usb webcam streaming via these plugins this will be covered in the next post. In meantime below are build instructions for deploying the latest code to the debian rootfs if you fancy experimenting.  I'm finding the debian jessie build useful to jump start prototype development on the imx6 mainly due to the availability of prebuilt packages.

Here is a short video demonstrating the use of color key along with the imxipusink to hide/view video within XFCE. The video only appears for the color key 'black' hence it is visible in the terminal windows and menu. The test device was a AR6MXQ board provide by BCM Advanced Research. As a designer/provider of industrial motherboards BCM have done a good job with their first feature rich ARM development board.

The current debian rootfs already includes an older build of the gstreamer-imx plugins. Upgrading these to a newer release is fairly trivial, you can build the latest sources natively on debian as follows:

1. Remove the existing plugins

 rm /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/ 
 rm /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/ 
 rm /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/ 

 rm /usr/lib/ 
 rm /usr/lib/ 
 rm /usr/lib/ 

2. Build lastest sources

 git clone git://  
 cd gstreamer-imx  
 ./waf configure --prefix=usr --kernel-headers=/usr/include  
 ./waf install  

3. Deploy plugins

 cp usr/lib/gstreamer-1.0/ /usr/lib/arm-linux-gnueabihf/gstreamer-1.0  
 cp usr/lib/gstreamer-1.0/ /usr/lib/arm-linux-gnueabihf/gstreamer-1.0  
 cp usr/lib/gstreamer-1.0/ /usr/lib/arm-linux-gnueabihf/gstreamer-1.0  
 cp usr/lib/gstreamer-1.0/ /usr/lib/arm-linux-gnueabihf/gstreamer-1.0  
 cp usr/lib/ /usr/lib  
 cd /usr/lib  
 ln -s  
 ln -s  

4. Reboot

5. Verify the plugins are present

 root@debian-imx6:~# gst-inspect-1.0 | grep imx  
 imxipu: imxipuvideotransform: Freescale IPU video transform element  
 imxipu: imxipusink: Freescale IPU video sink  
 imxvpu: imxvpudec: Freescale VPU video decoder  
 imxvpu: imxvpuenc_h263: Freescale VPU h.263 video encoder  
 imxvpu: imxvpuenc_h264: Freescale VPU h.264 video encoder  
 imxvpu: imxvpuenc_mpeg4: Freescale VPU MPEG-4 video encoder  
 imxvpu: imxvpuenc_mjpeg: Freescale VPU motion JPEG video encoder  
 imxeglvivsink: imxeglvivsink: Freescale EGL video sink  
 imxv4l2src: imxv4l2src: V4L2 CSI Video Source  

6. Test video playback (under X11)

 gst-launch-1.0 playbin uri=file://<video file> video-sink=imxeglvivsink  

Unfortunately the elements/sinks aren't documented so you may need to refer to the source code to determine features or properties. There is also a new plugin for CSI camera sources (imxv4l2src) provided by  Philip Craig.

Monday, 24 March 2014

I.MX6 Debian Jessie (GPU/VPU) 3.10.17_beta

Following on from the previous jessie rootfs, this is an updated version with the 3.10.17_beta BSP libraries deployed. It should work on most i.mx6 devices providing you have a valid uboot and kernel. Furthermore you can also use it as a test bed for Wayland/Weston development as per my last post.

One of the primary changes in the beta BSP is the inclusion of xrandr support within the X driver. Unfortunately the driver is partially functioning, for example I couldn't get it change resolution without restarting the X server. Another change is the Vivante driver are updated to 4.6.9p13 (see).

As with the previous rootfs:

1. Download and extract the tar file (md5 e1024db60c74df661e884a28eff104f6) onto your boot media
2. Deploy a compatible uboot on your boot media
3. Deploy 3.10.17 beta kernel (or a kernel with the 4.6.9p13 patches merged) on your boot media
4. If required enable serial console support in /etc/inittab (see)
5. Setup networking as required.

By default the system boots to console mode, refer to the previous post on how to verify the rootfs is functioning correctly.

I've included some scripts to allow switching between the fb, x11 and wayland vivante libraries, these are located in /root and self explanatory.


If you choose to use x11 by default XFCE (lightdm) is disabled. To launch it:

service lightdm start

If you have no screen output (assuming HDMI) then (because of the xrandr feature) you will to have to manually configure the screen resolution in xorg.conf. To do this un-comment the screen SubSection and change to the correct resolution.

    SubSection     "Display"
        Modes      "U:1280x720p-60"

If can't find the correct resolution then check the last output from /var/log/xorg.0.log. The current the resolution chosen by the X driver will be shown by the following statements:

[    38.607] (II) VIVANTE(0): Using user preference for initial modes
[    38.608] (II) VIVANTE(0): Output DISP3 BG using initial mode U:1280x720p-60
[    38.768] (II) VIVANTE(0): imxDisplayPreInit: virtual set 1280 x 720, display

The log file may also list all the possible modes available, it does this by reading EDID (assuming HDMI) which seems to be a hit or miss affair depending on your TV/monitor. For example you may see something similar to this: 

[    35.223] (II) VIVANTE(0): Modeline "U:720x576p-50"x0.0   27.00  720 732 796 864  576 581 586 625 -hsync -vsync -csync (31.2 kHz
[    35.224] (II) VIVANTE(0): Modeline "U:1920x1080p-60"x0.0  148.50  1920 2008 2052 2200  1080 1084 1089 1125 +hsync +vsync -csync
(67.5 kHz e)

Replace the Modes value with correct one from xorg.0.log and restart lightdm. For example:

Modes      "U:1920x1080p-60"

Monday, 3 March 2014

I.MX6 Wayland/Weston (3.10.17_beta)

Among the many items on my to do list was to determine how complete the Wayland support was in latest 3.10.17_beta BSP release. A complete description of Wayland and Weston can be found here.

In short, Weston is a reference implementation of a Wayland compositor. In order to run Weston we need to build Wayland and Weston, as a starting point I found some outdated instructions here. From these instructions and after further investigation it became clear that the freescale Weston implementation relies on a framebuffer backend (compositor-fbdev.c). The wayland libraries seem to be very similar to their framebuffer counterparts and support double and triple buffering.There are 2 possible configurations for the framebuffer backend:
  1. A custom render (GAL2D) which use the GC320 GPU + fbdev. There's a couple of patches that implement the render (gal2d-renderer.c)
  2. Use the built-in gl_render which uses EGL/GLES for composition.
I chose to use the gl_render so that the EGL sample(s) could be run. To run weston the following packages need to built (instructions for native compilation are below).

  1. wayland 
  2. ibxkbcommon 
  3. pixman
  4. cairo
  5. weston
The packages were built for deployment on the debian jessi rootfs which was upgraded to the 3.10.17_beta BSP along a 3.10.17 kernel. Wayland/Weston were built from the master branch which currently are at version 1.4. Here is a short video demonstrating a number of Weston examples running on a UDOO (this was the most convenient device at hand to test with).

Although the video demonstrates a number of Weston examples running well, I encountered high CPU when invoking the  'weston-smoke' example which tests SHM buffer sharing. Furthermore I encountered a few lockups what's unclear is whether these are a result of using the master branches of the deployed packages or a problem within fsl wayland libraries. Given the 3.10.17 BSP is in beta it should be treated as so.

Building and deploying Wayland/Weston

Ensure you have deployed the 3.10.17 Vivante gpu headers and libaries,  ensure symbolic links point to '-wl' libraries eg:

/usr/lib/ ->
/usr/lib/ ->
/usr/lib/ ->
/usr/lib/ ->

You may need to pull in addition packages to successfully complete compilation. Building cairo/pixman may require pulling in X11 packages. As mentioned these steps were completed on my debian jessi rootfs and should be transferable to other distros.

Build Wayland

export WLD=/usr
git clone git://
cd wayland
./ --prefix=$WLD --disable-documentation
make && make install
cd ..

Build libxkbcommon

git clone git://
cd libxkbcommon
make && make install 

cd ..

Build cairo

git clone git://
cd cairo
./ glesv2_CFLAGS="-DLINUX=1 -DEGL_API_FB -DEGL_API_WL" --enable-glesv2 --disable-xcb
make && make install
cd ..

Build pixman

git clone git://
cd pixman
make && make install
cd .

Build Weston

git clone git://
cd weston

We need patch compositor-fbdev.c so that a EGL handle is retrieved from the framebuffer so that gl_render can use it. See my patch

Now export these variables and build

export LDFLAGS="-lwayland-server -lwayland-client -lwayland-server -lwayland-cursor -lpixman-1"
export COMPOSITOR_LIBS="-lGLESv2 -lEGL -lGAL -lwayland-server -lxkbcommon -lpixman-1"
export COMPOSITOR_CFLAGS="-I $WLD/include -I $WLD/include/pixman-1 -L$SDK_DIR/drivers -DLINUX=1 -DEGL_API_FB -DEGL_API_WL"
export CLIENT_CFLAGS="-I $WLD/include -I $WLD/include/cairo -I $WLD/include/pixman-1"
export CLIENT_LIBS="-lGLESv2 -lEGL -lwayland-client -lwayland-cursor -lxkbcommon -lcairo"
export SIMPLE_EGL_CLIENT_LIBS="-lGLESv2 -lEGL -lwayland-client -lwayland-cursor"
export IMAGE_LIBS="-lwayland-cursor"
export WESTON_INFO_LIBS="-lwayland-client"
export EGL_TESTS_LIBS="-lEGL -lGLESv2 -lwayland-egl -lwayland-client -lcairo"
export TEST_CLIENT_LIBS="-lwayland-client -lcairo"

./ --prefix=$WLD --disable-setuid-install --disable-x11-compositor --disable-drm-compositor --disable-rpi-compositor --disable-wayland-compositor --disable-weston-launch --disable-libunwind --disable-xwayland --disable-xwayland-test WESTON_NATIVE_BACKEND=""

make && make install

Before we launch weston, edit the default weston.ini and comment out or remove the following lines (there no gnome terminal or chrome to launch):



Lets enable double buffering (can also set triple buffering):


To launch weston, asumming your are runnnig as root user ( and output to a log file):

export XDG_RUNTIME_DIR=/tmp; weston --log=weston.log --use-gl

If weston fails to launch, check the log file weston.log.

The following samples ran without problems: