LED - 3. Let's make a large led display - Part 6 (Playing movie and noise problem)

Playing a movie on the N X M RGB LED Matrix

In the previous post, I made a N X M LED display structure. This time, I'll use that display to play movies. I installed opencv 4.1.1 on the Raspberry Pi, and I use the OpenCV to play a movie.



import argparse
import cv2
import numpy as np
from PIL import Image
from PIL import ImageDraw
from rgbmatrix import RGBMatrix, RGBMatrixOptions
import time

parser = argparse.ArgumentParser(description="RGB LED matrix Example")
parser.add_argument("--video", type=str, required = True, help="video file name")
parser.add_argument("--horizontal", type=int, default = 1, help="horizontal count")
parser.add_argument("--vertical", type=int, default = 1, help="vertical count")
args = parser.parse_args()

# Configuration for the matrix
options = RGBMatrixOptions()
options.cols = 64
options.rows = 32
options.chain_length = args.horizontal * args.vertical
options.parallel = 1
options.brightness = 80
options.pwm_bits = 11
options.gpio_slowdown = 1.0
options.show_refresh_rate = 1
options.hardware_mapping = 'adafruit-hat-pwm'  # If you have an Adafruit HAT: 'adafruit-hat'
#options.hardware_mapping = 'adafruit-hat'  # If you have an Adafruit HAT: 'adafruit-hat'
options.pwm_dither_bits = 0

matrix = RGBMatrix(options = options)

canvas_w = args.horizontal * options.cols
canvas_h = args.vertical * options.rows

print('Matrix H:%d W:%d'%(matrix.height, matrix.width))
print('Image size H:%d W:%d'%(canvas_h, canvas_w))

cap = cv2.VideoCapture(args.video)

double_buffer = matrix.CreateFrameCanvas()

while cap.isOpened():
    imgs = []
    start = time.time()
    for i in range(2):
        ret, im = cap.read()
        if(ret == False):
            break
        im = cv2.resize(im, (canvas_w, canvas_h))
        im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
        imgs.append(im)

    if(ret == False):
        break

    im_pils = []
    for img in imgs:
        for x in range (args.vertical):
            i = img[options.rows * x: options.rows * (x + 1), 0:canvas_w]
            h, w, c = i.shape
            # print('split image H:%d W:%d'%(h, w))
            if ((args.vertical - x) % 2) == 0:    #-> flip
                i = cv2.flip(i, 0) #vertical
                i = cv2.flip(i, 1) #horizontal
            if(x == 0):
                final = i
            else:
                final = cv2.hconcat([final, i])   #stack horizontally

        h, w, c = final.shape
        # print('final image H:%d W:%d'%(h, w))
        im_pil = Image.fromarray(final)
        im_pils.append(im_pil)
        # matrix.Clear()
        #matrix.SetImage(im_pil, 0)

    double_buffer.SetImage(im_pils[0])
    double_buffer.SetImage(im_pils[1], w)
    double_buffer = matrix.SwapOnVSync(double_buffer)

    elapsed = time.time() - start
    #print('elapsed:%f'%(elapsed))
    time.sleep(max([0, 0.066 - elapsed]))


<nbym_video.py>

Becareful : OpenCV's default color format is BGR, but the Pillow uses RGB color format. So you need to change the color format of the image created with opencv to RGB format.

Run the code.

root@raspberrypi:/home/pi/rpi-rgb-led-matrix/bindings/python/samples# python3 nbym_video.py --video=Frozen2.mp4 --horizontal=2 --vertical=2
Matrix H:32 W:256
Image size H:64 W:128
split image H:32 W:128


<Noise Problem>

Noise Problem


I was able to play the video successfully. However, noise often appeared on the video screen. To solve this problem, I have tried various things such as changing the power and changing the video format, testing various Raspberry Pi models(2B,3B,4B) but I could not improve it. Clues to solving this problem can be found on Henner Zeller's git page. If you read the Troubleshooting section, there is a description of the brightness instability in the latest version of Rasbian, and introduces issue # 483. This is due to a structural problem in the latest Raspbian kernel. In the contents, the 2016 Jessie version says there are very few problems. So I decided to download the 2016 Jessie Rasbian. Unfortunately, Rasberry Pi4 only supports lateast version of Debian Buster. So to use Jessie Rasbian, you need to use Raspberry 2 or 3.

Installing Rasbian Jessie Lite(2016-02-29)

Download Jessie Lite Version image at http://downloads.raspberrypi.org/raspbian_lite/images/ . Old versions of Raspbian support ssh by default. Insert SD card image into Raspberry Pi, connect LAN cable to Raspberry Pi, and boot. You can then ssh directly from the remote PC.

First of all, run raspi-config to set the necessary settings:
  • expand file system
  • setup locale, keyboard, timezone
  • console mode boot
Then run this commands.


sudo reboot
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install vim git


Installing OpenCV on the Rasbian Jessie

Installing OpenCV on Jessie is different from installing on the latest Buster. In Buster, you installed the necessary modules and then installed them with the command pip3 install opencv-python.
But Jessie needs to download the source code and build it. Installation instructions are available at https://www.pyimagesearch.com/2015/10/26/how-to-install-opencv-3-on-raspbian-jessie/.


sudo apt-get install -y python3  python-dev  python3-dev python-pip  python3-pip  python-tk  python3-tk  python-numpy  python3-numpy python
sudo apt-get install -y build-essential cmake pkg-config libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev  x264 v4l-utils openexr libqtgui4 libqt4-test 
sudo apt-get install -y libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev  libgtk2.0-dev libgtk-3-dev  libatlas-base-dev gfortran  python-qt4
sudo apt-get install -y libcanberra-gtk*  libdc1394-22-dev  libxine2-dev  libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libgtk2.0-dev  libqt4-opengl-dev qt5-default
sudo apt-get -y install libprotobuf-dev protobuf-compiler libgoogle-glog-dev libgflags-dev libgphoto2-dev libeigen3-dev libhdf5-dev doxygen

cd /usr/local/src
wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.1.0.zip
unzip opencv.zip
wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.1.0.zip
unzip opencv_contrib.zip

cd /usr/local/src/opencv-3.1.0/
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D OPENCV_EXTRA_MODULES_PATH=../..//opencv_contrib-3.1.0/modules \
    -D BUILD_TESTS=OFF \
    -D OPENCV_ENABLE_NONFREE=ON \
    -D ENABLE_PRECOMPILED_HEADERS=OFF \
    -D BUILD_opencv_python2=ON \
    -D BUILD_opencv_python3=ON \
    -D BUILD_EXAMPLES=OFF ..

make -j4
make install
ldconfig

Tip : If you get an error like:

/usr/local/src/opencv_contrib-3.1.0/modules/hdf/include/opencv2/hdf/hdf5.hpp:40:18: fatal error: hdf5.h: No such file or directory
 #include <hdf5.h>

Modify the "/opencv-3.1.0/modules/python/common.cmake" files. Include these 2 lines.


# This file is included from a subdirectory
set(PYTHON_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../")
 
#include these 2 lines at the beginning 
find_package(HDF5)
include_directories(${HDF5_INCLUDE_DIRS})

ocv_add_module(${MODULE_NAME} BINDINGS)


If the installation was successful, you can test it with:


pi@raspberrypi:~ $ python3
Python 3.4.2 (default, Sep 16 2019, 19:58:00)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'3.1.0'
>>>


Playing a movie on the Jessie again

After installing the OpenCV, I installed Henner Jeller's rpi-rgb-led-matrix on the Jessie. I already explained this on my previous post. Then I ran the above python code nbym_video.py again to compare the Rasbian Buster.

<movie playing on the Jessie>


Severe noise disappeared and much better screen realization was possible. However, there is still a slight fluctuation in brightness. This may be due to high CPU consumption in reading and converting video frames in OpenCV. Including this cause, I'll cover tuning performance and adjusting options in the next article.

Wrapping up

The Jessie version of 2016, rather than the latest version of Buster Rasbian, allows for much higher quality screens. The downside is that it's not available on the Rasberry Pi 4.

You can download the source codes here(https://github.com/raspberry-pi-maker/IoT)


댓글

이 블로그의 인기 게시물

LED-12. Displaying HDMI Contents

LED - 5. Raspberry Pi 4 + DietPi Buster + Electrodragon HAT - Part 1

LED-11. Make a very big size 384 X 512 RGB Matrix #6(final)