LED-11. Make a very big size 384 X 512 RGB Matrix #4

 A network must be used to exchange images between the LED server and the player. A method for transmitting an image over a network is possible in several ways. In most cases, in order to reduce bandwidth, data is compressed and transmitted, and the receiver undergoes a process of decoding again. However, since we will be sending images on a local LAN, we will prioritize speed over bandwidth. Therefore, in this article, we will send and receive images without decompression. For image transmission using a network, refer to the next article.

• LED - 7. Distrbuted display system for very large RGB LED matrices
  

 

 Network display

 First, let's test the network image transfer using a simple Python program.

Test

 The following is a server-side Python program. We will read the image and send it to the player.

#!/usr/bin/python3
import argparse
import numpy as np
import cv2
import time
from PIL import Image, ImageSequence
import netsender2 as netsender
import socket

canvas_w = 384
canvas_h = 512
parser = argparse.ArgumentParser(description='gif run')
parser.add_argument('--file', type=str, required=True, help='image file name')
args = parser.parse_args()

netsender.set_recv_pi([("192.168.11.11", 4321), ("192.168.11.12", 4321), ("192.168.11.13", 4321), ("192.168.11.14", 4321)])

img = cv2.imread(args.file, cv2.IMREAD_UNCHANGED)
img = cv2.resize(img, (canvas_w, canvas_h))
h, w, c = img.shape
if c == 4:
    img = cv2.cvtColor(img, cv2.COLOR_BGRA2BGR)

sleep_sec = 0.05
count = 0
total_sec = 0.0
while True:
    try:
        s = time.time()
        netsender.net_send(img)
        time.sleep(sleep_sec)
        count += 1
        total_sec += (time.time() - s)
    except KeyboardInterrupt:
        break
    except socket.timeout:
        continue

FPS = count / (total_sec)        
print('FPS:%4.2f'%(FPS))

 <server side image sender program : test_image2.py>

And this is a program on the player side. The same program will run on 4 players.

#!/usr/bin/python3
import numpy as np
import cv2
import socket
import struct
import time
import argparse
from threading import Thread

from rgbmatrix import RGBMatrix, RGBMatrixOptions
from PIL import Image, ImageDraw, ImageFont

CHUNK_SIZE = 8192 + 32
CHUNK_SIZE = 52000

H = 128
W = 384
C = 3

VSwap = False


parser = argparse.ArgumentParser(description='net run')
parser.add_argument('--port', type=int, default=4321, help='port number')
parser.add_argument('--cols', type=int, default=128, help='cols')
parser.add_argument('--rows', type=int, default=64, help='rows')
parser.add_argument('--chain_length', type=int, default=2, help='chain_length')
parser.add_argument('--parallel', type=int, default=3, help='parallel')
args = parser.parse_args()

VMapper = True


options = RGBMatrixOptions()
options.cols = args.cols
options.rows = args.rows
options.chain_length = args.chain_length
options.parallel = args.parallel
options.gpio_slowdown = 4
options.show_refresh_rate = 0
options.hardware_mapping = 'regular'  # I'm using Electrodragon HAT

#options.pixel_mapper_config = "Rotate:180"
options.pwm_bits = 7
options.pwm_dither_bits = 1
if VMapper == True:
    options.pixel_mapper_config = "V-mapper"


matrix = RGBMatrix(options=options)
double_buffer = matrix.CreateFrameCanvas()


sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("0.0.0.0", args.port))
bufsize = sock.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF)
if CHUNK_SIZE < bufsize:
    CHUNK_SIZE = bufsize

print('Rcv buf size:%d' % (CHUNK_SIZE))
sock.settimeout(60)
total = 0
buf = []
packet_cnt = 0


def reset():
    global buf, total, packet_cnt
    total = 0
    buf = []
    packet_cnt = 0
    time.sleep(0.001)

while(True):
    try:
        data, addr = sock.recvfrom(CHUNK_SIZE)
        total += len(data)
        key = int.from_bytes(data[:4], byteorder="big")
        seq = int.from_bytes(data[4:8], byteorder="big")
        cnt = int.from_bytes(data[8:12], byteorder="big")
        buf += data[12:]
        packet_cnt += 1

        if key == 1:  # last
            # if(packet_cnt != cnt):
            #     print('Total rcv cnt:%d total chunk:%d'%(packet_cnt, cnt))
            #     reset()
            #     continue
            try:
                img = np.asarray(buf, dtype=np.uint8)
                img = img.reshape(H, W, C)
                final = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
                h, w, c = final.shape
                im_pil = Image.fromarray(final)
                #print('Reshape success')

                double_buffer.SetImage(im_pil)
                VSwap = True 
                double_buffer = matrix.SwapOnVSync(double_buffer)

                reset()
            except ValueError:
                reset()
                print('Reshape Error')

    except KeyboardInterrupt:
        break
    except socket.timeout:
        reset()
        continue

 <player side image receiver program : net_player2.py>

 

 First run the player programs.

root@LEDOne:/usr/local/src/test# python3 net_player2.py 
Rcv buf size:5242880

 

root@LEDTwo:/usr/local/src/test# python3 net_player2.py 
Rcv buf size:5242880

 

root@LEDThree:/usr/local/src/test# python3 net_player2.py 
Rcv buf size:5242880

 

root@LEDFour:/usr/local/src/test# python3 net_player2.py 
Rcv buf size:5242880

 

 Now run the server program with sleep_sec = 0.05.

root@LEDSrv:/usr/local/src/test# python3 test_image2.py --file=totoro.jpg 
Snd buf size:5242880
network image receiver set to:[('192.168.11.11', 4321), ('192.168.11.12', 4321), ('192.168.11.13', 4321), ('192.168.11.14', 4321)]
^CFPS:15.76

 After running the program for a while, press Ctrl+C to check the FPS value. At sleep_sec = 0.05, about 15.40 FPS was obtained. 

 In this setting environment, it was displayed without any problem.

 

 This time, I will lower the sleep_sec value to 0.02 to increase the FPS. However, there are some problems on the player side.

 After running the program for a while, press Ctrl+C to check the FPS value. At sleep_sec = 0.02, about 30.56 FPS was obtained. 

The "Reshape Error" message sometimes appears in the player output window.

This message appears when image packets sent from the server side are not processed normally.

The reshape error doesn't happen for every image frame, but you can monitor it for occasional occurrences. The reshape error is that the image frame received through udp communication is not normal. 

The player program uses the SwapOnVSync function to update the LED display image after combining packets sent from the server. 

This series of tasks must end faster than the time interval at which images are sent from the server. If the player's processing time slows down, packet loss occurs, and image frames cannot be made normally. In this case, a 'Reshape error' occurs.

It was almost impossible to reliably implement 30 FPS in a player using Python.

The biggest reason of "Reshape error" is that this program is made of Python. Originally, the rpi-rgb-led-matrix library was created for C/C++. Python bindings were then added. Thus, if the program is re-created using the rpi-rgb-led-matrix library for C/C++, the processing speed can be much higher.

Tunning

 Several ways are available to speed up the player's processing and operate reliably.

Raspberry Pi Overclocking and reserver CPU core

By disabling one of the four CPUs in the Raspberry Pi by the Linux kernel, LED applications can use more CPU resources. If raspberry pi is used exclusively for LED players, this is a very good method. Henner Zeller also highlights this approach on the https://github.com/hzeller/rpi-rgb-led-matrix page.

Add the following text to the end of the /boot/cmdline.txt file. Note that the line must not be changed.

isolcpus=3

 And Henner Zeller explains several options on this page to improve performance. I definitely recommend reading it.

 

 You can overclock the player using the dietpi-config command.

 

Porting Python Code to C/C++

 The C++ codes are not released in this article due to the large amount. But you can download them from my Github. The operation of the C++ program is exactly the same as the Python above.

Reduce electrical noise

Both the LED matrix and the Raspberry Pi use 5V power. Thus, a single 5V power rail can be used.

But don't organize it like this. The reason is that the LED matrix uses a large amount of power, which makes the power of the raspberry pi very unstable if the same power source is used by the LED and raspberry pi. When a voltage drop occurs suddenly, the raspberry pi becomes momentarily unstable. In severe cases, system downtime may occur.

Therefore, a separate 5V power source is prepared for raspberry pi.

And to reduce electrical noise, it is also good to twist +,- cables. And where there is a lot of electrical noise, I add electromagnetic shielding devices.

Wrapping up

In this article, I tested whether it works using 1 LED server and 4 players. The program required for testing is implemented using Python. However, there was a performance issue that did not work properly at 30FPS. Although it is possible to increase the hardware performance to some extent by using methods such as Raspberry Pi CPU control and overclocking, it cannot be a fundamental solution. The way to dramatically improve performance is to port a player implemented in Python to C/C++. When implemented in C/C++, performance improvement can be expected from several times to several tens of times.

The LED server will not be ported to C/C++. The biggest reason is that the LED server is a program that does not directly control the LED, so it is not necessary to use the Raspberry Pi. You can also use a high-performance x86 system.

 In the next article, we will look at porting the LED player program to C/C++.