Thursday, 1 May 2025

RK3588 - Building an simple HDMI anaylzer

While developing and debugging HDMI drivers and custom video applications. You quickly come to appreciate the convenience of having a straightforward way to debug the HDMI output both video and audio in real time. This led me to develop a utility specifically designed to display basic HDMI metadata and render the incoming video and audio stream. The goal was to create a tool that could simplify the process of diagnosing HDMI-related issues when connected to an HDMI source.


One of the undervalued features of the RK3588 SoC is its built-in HDMI receiver—an often overlooked capability that eliminates the need for external HDMI-to-CSI adapters like the TC358743XBG or RK628D. This built-in receiver makes the RK3588 an ideal platform for developing such a utility, as it allows direct access to HDMI input source without additional hardware.

While the BSP hdmi rx kernel driver may not be of the highest quality, it offers several useful IOCTLs for detecting the present of a valid signal in addition to providing valuable video and audio metadata. With this in mind, the application was developed as a Weston (Wayland) client, which turned out to be more challenging than initially anticipated. The primary difficulties arose from:

  1. Video Input Rendering – Optimizing the pipeline to minimize latency between frame acquisition and on-screen rendering. For rendering, I went with GStreamer pipelines paired with WaylandSink, to ease integration the Wayland compositor.
  2. Overlaying graphical information on top of video playback – synchronizing real-time overlays with live video in a Wayland compositor like Weston required careful handling of surface layers and rendering pipelines. For rendering overlays, I adopted an approach based on Weston subsurfaces.
  3. Converting and scaling the video input – processing the raw video feed and resizing it dynamically while maintaining real-time performance introduced both technical and performance challenges. To handle conversion and scaling, I resorted to developing a custom GStreamer plugin built around RGA3, deliberately bypassing RGA2 due to the known constraints with 32-bit memory addressing (its no different to the NPU).
  4. Audio Detection – The HDMI RX kernel driver monitors for the presence of an audio stream. The challenge was to dynamically add or remove audio from the GStreamer pipeline based on its availability and without disrupting the ongoing video stream.

As illustrated in the image below, basic metadata for both the video and audio streams is displayed in the overlay window.

 

In the first video (above), the HDMI output from a Lenovo Windows laptop is used as the input source. The laptop is configured to duplicate its display over HDMI, functioning as a secondary screen. This setup effectively demonstrates minimal latency in both user interaction and video playback, highlighting the responsiveness of the HDMI capture pipeline.


The second video showcases the utility’s scaling capabilities. With the output display set to 1080p, I used a Lindy HDMI 18G Signal Generator to feed input signals at various resolutions—720p, 1080p, 4K@30Hz, and 4K@40Hz—in multiple formats including RGB, NV16, and NV12. All input signals were successfully scaled to 1080p.

During testing, I discovered a limitation: although the RK3588’s HDMI RX receiver supports NV24 format input, neither RGA3 nor RGA2 (according to documentation and my own validation) are capable of processing NV24. Interestingly, the RK3576 is equipped with the newer RGA2-Pro which support NV24, despite lacking an HDMI RX receiver.

Work was performed on a ROCK 5B board running a tailored Ubuntu image maintained by Joshua Riek.

1 comment:

  1. Hey this is a really nice tool! Is it available anywhere? I'm also currently working on some RK3588 HDMI-RX pipelines (end goal would be to have it fully working in OBS without crashing) and this seems very useful. Also have you figured out how to set the EDID for the HDMI Input? Depending on the HDMI Output device I use I cannot select all the resolutions off them

    ReplyDelete