LED-12. Displaying HDMI Contents

 I recently bought an interesting product. It is a device that converts HDMI data to USB.

This device works very much like a webcam. Just as a webcam transmits the video input through the camera lens through USB, this device transmits the video coming through HDMI through USB.

Therefore, the host computer to which this device is connected treats it almost the same as a webcam.

<HDMI Ccapture device>

It's a cheap product you can get for around $10 on Amazon or Ali Express.

This product can be usefully used when using SBC (Single Board Computer) such as Raspberry Pi. If you need to check the boot screen because the Raspberry Pi doesn't work properly without a monitor, you can use this device. After connecting the HDMI cable to the Raspberry Pi, connect this device to the other side and then connect the USB port to your laptop. You can check the screen of the Raspberry Pi on the laptop without a monitor.


<Raspberry Pi screen output using Window's VLC player>


In this article, I will try to output the HDMI output to the LED signboard.


HDMI output to LED signboard

Before displaying the HDMI output on the LED signage, let's handle the streaming from HDMI using OpenCV. This process is the same as using a webcam. If HDMI content is handled properly in OpenCV, it is not difficult at all to display the content on the LED signboard.

Controlling HDMI devices on Raspberry Pi

I will test OpenCV on the Raspberry Pi as I will be using the Raspberry Pi and Electrodragon's LED HAT as always. Connect the HDMI capture card described above to the USB 3.0 of the Raspberry Pi 4.

Note: Most HDMI capture cards support USB 3.0 or higher. Therefore, it is recommended to use a USB 3.0 port on Raspberry Pi 4 or higher.


The HDMI side does not need to be connected yet.

After booting the Raspberry Pi, open a terminal and check the device as follows. First, check the connected USB device with the lsusb command.

root@raspberrypi:~# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 534d:2109  
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Device ID 534d:2109 is a newly added HDMI capture device, but the name is not visible. Note that the device name may vary depending on the type of HDMI capture card you are using.  

Let's take a closer look with the command "ls -al /dev/v4l/by-id".

root@raspberrypi:~# ls -al /dev/v4l/by-id
total 0
drwxr-xr-x 2 root root 80 Aug  5 01:17 .
drwxr-xr-x 4 root root 80 Aug  5 01:17 ..
lrwxrwxrwx 1 root root 12 Aug  5 01:17 usb-MACROSILICON_usb_video-video-index0 -> ../../video0
lrwxrwxrwx 1 root root 12 Aug  5 01:17 usb-MACROSILICON_usb_video-video-index1 -> ../../video1

You can also check it with the "v4l2-ctl command".

If an error stating that v4l2-ctl cannot be found occurs, you can install it as follows.

root@raspberrypi:~# apt-get install v4l-utils

Again, with the v4l2-ctl command, you can see that /dev/video0 and /dev/video1 are connected to the HDMI capture card.

root@raspberrypi:~# v4l2-ctl --list-devices
bcm2835-codec-decode (platform:bcm2835-codec):
	/dev/video10
	/dev/video11
	/dev/video12

bcm2835-isp (platform:bcm2835-isp):
	/dev/video13
	/dev/video14
	/dev/video15
	/dev/video16

usb video: usb video (usb-0000:01:00.0-1.1):
	/dev/video0
	/dev/video1


Now let's figure out whether to use /dev/video0 or /dev/video1. You can use the command "v4l2-ctl --device /dev/video1 --list-formats-ext".

root@raspberrypi:~# v4l2-ctl --device /dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

root@raspberrypi:~# v4l2-ctl --device /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'MJPG' (Motion-JPEG, compressed)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
		Size: Discrete 1600x1200
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
		Size: Discrete 1360x768
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
		Size: Discrete 1280x960
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.020s (50.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
		Size: Discrete 1024x768
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.020s (50.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.020s (50.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)

You can see that /dev/video0 can be used. You can also check the resolution and FPS supported by this device.


Testing HDMI capture card using OpenCV

As explained earlier, since the HDMI capture card has the same structure as the webcam, you can program OpenCV in a way that controls the webcam.

This is a simple test code.


import cv2
import sys

print('HDMI Capture Test')
cap = cv2.VideoCapture(0)
if (cap.isOpened() == False): 
  print("Unable to read HDMI Capture")


cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
ret, img = cap.read()
if ret == False:
    print('HDMI Capture Read Error')    
    sys.exit(0)
h, w, c = img.shape
print('HDMI Frame shape H:%d, W:%d, Channel:%d'%(h, w, c))


count = 1
while cap.isOpened():
    try:
        ret, img = cap.read()
        if ret == False:
            break
        count += 1
        cv2.waitKey(1)
        cv2.imshow('HDMI Capture', img)
    except KeyboardInterrupt:
        print('Ctrl + C')
        break

print('HDMI Capture Frame read End. Total Frames are : %d'%(count))
cap.release()

<hdmi_capture.py>

Now connect the HDMI cable to the capture device. I connected my notebook HDMI port.

Run and and check whether the python program displays HDMI contents correctly.

root@raspberrypi:/usr/local/src/study# python3 hdmi_capture.py
HDMI Capture Test
HDMI Frame shape H:480, W:640, Channel:3

I was able to successfully check the following window. The image in this window is from what I have connected to my laptop's HDMI port.

<HDMI content to OpenCV output>

I have confirmed that we can properly process images from HDMI capture devices using OpenCV. Now it is the order to output this image on the LED signboard. This task is not difficult because it has been done many times before.


Connecting LED signboard

For the test, I prepared 4 LED panels of 64X32 size. And I prepared a Raspberry Pi and Electrodragon Hat and then connected it as follows.

 

It looks complicated because of the lines, but if I put it simply, it is as follows.


Final Test

I made the following simple test program. It is exactly the same as the program that casts the webcam.


import argparse, sys
import cv2
import numpy as np
from PIL import Image
from PIL import ImageDraw
from rgbmatrix import RGBMatrix, RGBMatrixOptions
import time

PANEL_W = 64
PANEL_H = 32
CHAIN_LEN = 2
CHAIN_PARALLEL = 2


# Configuration for the matrix
options = RGBMatrixOptions()
options.cols = PANEL_W
options.rows = PANEL_H
options.chain_length = CHAIN_LEN
options.parallel = CHAIN_PARALLEL

options.brightness = 80
options.gpio_slowdown = 4.0
options.show_refresh_rate = 1
options.hardware_mapping = 'regular'  # If you have an Adafruit HAT: 'adafruit-hat'

#options.pixel_mapper_config = "Mirror:V;Rotate:180"

cap = cv2.VideoCapture(0)   #HDMI Input

matrix = RGBMatrix(options = options)
canvas_w = PANEL_W * CHAIN_LEN
canvas_h = PANEL_H * CHAIN_PARALLEL


print('Matrix H:%d W:%d'%(matrix.height, matrix.width))
print('Image size H:%d W:%d'%(canvas_h, canvas_w))

if cap is None:
    print('cannot open camera')
    sys.exit(0)

double_buffer = matrix.CreateFrameCanvas()
w = double_buffer.width
print('offscreen width:' + str(w))
 
while cap.isOpened():
    try:
        s = time.time()
        ret, im = cap.read()
        if ret == False:
            break
        im = cv2.resize(im, (canvas_w, canvas_h))
        im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
        im_pil = Image.fromarray(im)


        double_buffer.SetImage(im_pil)
        e = time.time()
        #time.sleep(0.03)
        double_buffer = matrix.SwapOnVSync(double_buffer)
        #print('FPS:%4.2f'%(1 / (e - s)))
        #matrix.Clear()
        #matrix.SetImage(im_pil)
    except KeyboardInterrupt:
        break  

cap.release()
print('End' )

<hdmi.py>


Note: In the hdmi.py file I called the "cv2.VideoCapture(0)" function before creating the RGBMatrix object. I don't know why, but if I call the "cv2.VideoCapture(0)" function after "matrix = RGBMatrix(options = options)", cv2.VideoCapture(0) does not open HDMI capture card properly.


Now let's run this program.

<Test screen>

I was able to successfully display the laptop's screen on the LED signboard. This screen is not a still screen. When the screen of the laptop changes, the LED display automatically changes as well. For example, if you play a video on a laptop, the video is played on the LED display in real time.


Wrapping up

Video streaming using an HDMI cable has several advantages as follows.

  • It is possible to display HDMI content in real time. When playing video using OpenCV, it is difficult to accurately match the FPS. However, in this method using an HDMI capture card, the content is streamed in real time the same as using a webcam.
  • Audio play is possible. If the audio output of the laptop is connected to the amplifier, it is possible to output audio along with the video on the LED signboard. OpenCV is difficult to play audio at the same time.
  • It can be applied to real-time broadcasting such as TV and YouTube.


In the example above, a small display of 128X64 was used, but if a high-resolution LED signboard is used, a clean large-screen output will be possible.
For reference, I have explained how to make a high-resolution LED signboard in the Raspberry Pi in the following articles.


댓글

  1. mudasar2022년 5월 28일 오후 9:47

    I generally think of instructional content as dull but necessary for literacy. Intriguing instructional papers like this are rare. This material is instructional without being boring and intimidating. Thank you.
    Elgato capture card best Price in Pakistan

    답글삭제

댓글 쓰기

이 블로그의 인기 게시물

LED - 5. Raspberry Pi 4 + DietPi Buster + Electrodragon HAT - Part 1

이미지
This post is related to several posts written to use the RGB LED matrix in Raspberry Pi. If you haven't read the previous posts, it can be hard to understand, in which case you ㅡmight read the previous posts in advance. I've covered in previous post that Rasbian Buster and the RGB Matrix HAT don't fit together. I was able to play the video successfully. However, noise often appeared on the video screen. To solve this problem, I have tried various things such as changing the power and changing the video format, testing various Raspberry Pi models(2B,3B,4B) but I could not improve it. Clues to solving this problem can be found on Henner Zeller's git page . If you read the Troubleshooting section, there is a description of the brightness instability in the latest version of Rasbian, and introduces issue # 483 . This is due to a structural problem in the latest Raspbian kernel. In the contents, the 2016 Jessie version says there are very few problems. So I de
자세한 내용 보기

LED-11. Make a very big size 384 X 512 RGB Matrix #6(final)

이미지
 Finally, it is the day to install the 384X512 size LED signage in the cafe. Up to now, assembly, development, and testing were carried out in my office. Finished installing and testing Node-RED for minor bug fixes and electronic parts management. And to install the cafe, I disassembled the electronic parts again and took it to the cafe. Here we will install it again after reassembly. reassembly All 24 LED panels, aluminum profiles, Hub75 cables and wires were removed and reassembled in the cafe. <Rear view of reassembled LED signage> After reassembly, a simple test was conducted. Confirmed that it works without problems. The next thing to do is to fix the sign structure to the window. <Test screen after reassembly> The black line on the screen is invisible to the human eye. <LED signage installation> Four arms were connected to the LED frame and fixed to the aluminum window frame. And after playing the LED content, I checked it outside the building. <LED signage i
자세한 내용 보기