KimSe-Young1
LeeSeong-Won2
-
(Department of Computer Engineering, Kwangwoon University / Seoul, Korea ksy94311@office.kw.ac.kr
)
-
(Department of Computer Engineering, Kwangwoon University / Seoul, Korea swlee@office.kw.ac.kr
)
Copyright © The Institute of Electronics and Information Engineers(IEIE)
Keywords
LED matrix, Headlights, CAN, Communication overhead
1. Introduction
LED headlights for automobiles have a longer lifespan, higher brightness, and more
eco-friendly features. This is due to their low power consumption compared to conventional
light sources, such as sealed beam lamps using filament bulbs, halogen lamps, and
high-intensity discharge lamps (HID lamps) [1]. Recently, many LEDs have been arranged in a matrix form to obtain a more precise
scanning area. By finely adjusting the brightness of each LED, safe driving conditions
can be ensured by providing rich information to pedestrians or drivers of other vehicles.
They improve vehicle recognition not only at night time but also in daytime as daytime
running lights (DRLs) [2]. However, an LED matrix system capable of fine brightness or color adjustment requires
a significant amount of data, so it could be a burden on the vehicle communication
system. Accordingly, such an application would require a dedicated high-speed communication
line, causing higher prices.
Previously, the communication burden was reduced by individually controlling LED matrix
components and the entire group using a shift command [3]. However, since the architecture in that study used a dedicated communication line
and the brightness value of each LED was fixed, it is hard to apply it to recent advanced
LED matrix headlights, where brightness or color expression can be controllable. In
this study, the communication method was changed to a CAN interface, which is a standard
automobile communication protocol, to implement a more suitable interface for vehicles.
In addition, a control scheme with reduced communication volume is proposed for an
LED matrix with adjustable color and brightness. A masking technique was used to enable
simultaneous transmission to multiple LEDs with a specific pattern, and bilinear interpolation
shift was used to enable subpixel unit shift.
CAN communication is convenient for sending and receiving data because ID transmission
and error detection are possible. However, the number of effective transmission bytes
is reduced due to this additional control information. Therefore, based on the bandwidth
limitation of CAN communication and operation of advanced driver assistance system
(ADAS), a command system was designed to issue only a few commands per frame, such
as less than 10 commands. The proposed system was tested against various cases that
may appear in a general driving environment. Accordingly, we demonstrated that an
effective LED matrix control scheme was possible only when respecting certain transmission
limits.
2. Background on LED Matrix
The first electric automobile headlamps started being implemented with incandescent
lamps in 1898. Later, they progressed to halogen lamps in 1962, HIDs in 1991, LEDs
in 2004, and laser lamps in 2014. Recently, intelligent light systems using LED matrices
were introduced. In the past, the main objective was seeing farther with brighter
light; however, the latest developments are characterized by intelligent headlight
technology that is optimized to secure the driver's vision while considering the safety
of other drivers and pedestrians on the road.
To achieve diverse objectives, intelligent automotive headlight systems linked with
ADAS have multiple LEDs and adjust them in various ways. Intelligent headlight system
technologies include "Bending Light,'' which can secure visibility in the direction
of turning when cornering, "High Beam Assistance" (HBA), which automatically turns
the high beams on or off, and "Adaptive Driving Beam" (ADB), which is a more advanced
technology than HBA that blocks light emitted from a vehicle in front while brightly
illuminating the remaining area [4]. ADB has two types. "Swivel type" uses mechanical movement to prevent glare from
oncoming vehicles by rotating the headlights, while "matrix type" has a fast response
speed by using only an LED array without mechanical movement.
Current technologies being developed or used include Tesla's "Light Show" [5], Mercedes-Benz's "Multibeam LED" [6], BMW's "Adaptive LED Headlights" and "Laserlight Technology" [7], Audi's "Digital Audi Matrix LED Headlight" [8], Volkswagen's "IQ.Light" [9], and Hyundai's "Intelligent Front-lighting System (IFS)" [10]. Regarding ADBs, research using matrix types and LED arrays is being conducted. In
the case of Mercedes-Benz’s Multibeam LED, 84 LED chips are built in for each side,
allowing individual control. Four control devices calculate the optimal lighting 100
times per second using information provided by a camera and navigation system. Each
LED has 255 levels of brightness control.
Volkswagen's IQ.Light includes 128 LEDs on each side, including 48 dipped headlights
and 27 main beam headlights, as well as LEDs for various other functions, such as
short-distance lighting and signaling. These LEDs use front camera signals, digital
data obtained from the navigation system, GPS signals, the steering wheel angle, and
the current speed to control the LEDs. As intelligent headlight systems equipped with
more LEDs and functions are being developed, research should focus on reducing the
amount of data transmission.
3. Proposed LED Matrix Control System
3.1 System Configuration
The entire system consists of a hardware part, which is directly involved in LED control
according to the transmitted control commands, and a software part, which identifies
which LEDs need to be controlled to reduce the communication burden. The ADAS recognizes
and assesses numerous situations that may occur during driving and issues control
commands to increase safety and convenience [11]. The objects in the surrounding environment necessary for the headlight to shine
on is also captured by the ADAS, and the specific color and brightness values of the
LED matrix headlight are transmitted to a microprocessor.
The hardware configuration of the system is shown in Fig. 1, which outlines how the data is transmitted. After the ADAS system sends data, the
microprocessor part determines how to modify the values in the LED matrix and transmits
several commands to the controller via the CAN bus. Then, an LED matrix controller
that was implemented in FPGA hardware takes the commands and operates accordingly
to change values for the LED matrix.
Fig. 1. Hardware configuration of the system and method of transmitting data transmission.
3.2 Microprocessor and Control Commands
A dedicated high-speed communication line could be required to transmit all brightness
values of the entire LED matrix from the microprocessor to the headlights. However,
since the proposed system controls the LED matrix by issuing a low number of commands,
CAN communication is used to transmit the corresponding commands. The CAN interface
uses twisted-pair and balanced-line signaling methods to provide a simple line configuration
[12]. EMI can cause unintended operation errors in electronic products by transmitting
a signal to the twisted-pair line in the CAN system. Furthermore, its bandwidth for
data transmission is not sufficient for the all LEDs in the LED matrix. Although the
maximum speed of CAN is 1 Mbps, the recommended speed is limited to 500 Kbps for stable
use [12,13]. In this study, due to the bandwidth limitations of a CAN interface, the proposed
system issued only up to 6 commands of up to 6 bytes per frame based on the operation
characteristics of the ADAS camera.
The selection of 6 commands is based on the difference of the LED values between the
current and previous frames. The commands were selected so that the error between
the consecutive data frames is minimized. Accordingly, the headlight control pattern
received from ADAS is analyzed by the microprocessor, and the minimum LED matrix control
commands are transmitted to the headlight through the CAN interface. Because there
is a headlight on both sides of the vehicle, the command format was set so that each
can be controlled independently or simultaneously.
A single command uses a total of 48 bits with a maximum of 6 bytes and consists of
an 8-bit ``command'' field, an 8-bit ``address'' field, an 8-bit ``mask'' field, an
8-bit ``green'' field, an 8-bit ``red'' field, and an 8-bit ``blue'' field. It can
be controlled with a maximum of 36 bytes by transmitting 6 commands per frame. Fig. 2 shows the command format transmitted from the microprocessor to control the LED matrix.
In the command field presented in Fig. 3, the upper 8 bits correspond to the control data format, where the highest bit of
the control activates the writing of blue color data. The next bit is for green, and
the bit after that is for red. These bits controls the data writing of each color
to be activated. The next bit is used to move data. If it is 1, it moves. If it is
0, data is written. The next two bits indicate the direction of movement up, down,
left, and right. The last two bits control the bilinear interpolation of 1/4, 2/4,
3/4, or 4/4 LED pixel with the movement distance stated in subpixel units. The next
address byte allows a user to assign addresses 0 to 19, 20 to 39, 40 to 59, and 60
to 79, each with 20 LEDs per line, for a total of 80 LEDs.
The next byte is a mask byte, which masks specific bits of the address line when transmitting
LED values so that data can be transmitted to multiple LEDs simultaneously. By setting
1 of the address mask bit in this field indicates a dont-care condition. Subsequently,
the values of multiple LED matrix components can be set as the following BGR color
value simultaneously. Using this approach, all LEDs can be turned on/off, as shown
in Fig. 5, and they can be controlled with a specific pattern to respond to changes in the
overall value of LEDs.
The last 3 command bytes of correspond to blue, green, and red values, respectively.
Accordingly, color can be controlled by specifying values of 0 to 255. Color expression
can be used for various purposes, such as navigation direction indicators or vehicle
condition scanning on the ground.
Fig. 2. Command format sent from microprocessor to control LED Matrix.
Fig. 3. Command byte format.
Fig. 4. Mask byte format.
Fig. 5. Mask address for controlling with a specific pattern of LEDs.
3.3 Hardware Configuration
The hardware components are presented in Fig. 3 (the microprocessor, control, driver, and LEDs). The microprocessor selects a series
of up to 6 commands to minimize the error by receiving the headlight value of the
current frame from the ADAS and comparing it with the value specified in the current
LED matrix. The control part manages the address mask to select the region of target
LED addresses using an address mask for updating and controls the shift for bilinear
interpolation. Accordingly, the controller can set the brightness value by adjusting
the write enable signal of each LED in the matrix according to the transmitted command.
It can also control the shift direction of the LED matrix, in which LEDs are connected
in the form of a mesh. The driver part generates a PWM signal for LEDs. In this driver
configuration PWM signals for LEDs that are connected in series are also transmitted
in series.
The shift part of the control unit was implemented as shown in Fig. 7. The LED signal can be shifted by selecting ``up,'' ``down,'' ``left,'' ``and'' ``right.''
This framework was designed to perform bilinear interpolation of the LED value at
the current position and the transmitted adjacent LED value through a filter.
Fig. 6. Shift movement for control with specific pattern of LEDs.
Fig. 7. Bilinear Interpolation Shift Filter.
3.4 Software Configuration
3.4.1 Bilinear Interpolation Shift
The software in the microprocessor decides which LED should be changed according to
Algorithm 1. When the vehicle moves up and down or left and right, the headlight beaming direction
is adjusted to the opposite or same direction of the movement. To cope with this in
the LED matrix headlights, it is possible by changing the overall position using a
shift operation rather than transmitting new LED brightness values to reduce the amount
of communication. However, depending on the speed of the vehicle, shifts of less than
one pixel often occur.
The proposed system was designed to enable bilinear interpolated 4-step subpixel shift
using the current pixel value and neighboring pixel values in the shifting direction.
Because the movement of the vehicle occurs in an arbitrary direction based on various
situations, when the LED matrix brightness values in the new frame of the ADAS are
$L_{t}$, there are 16 cases of shifting the previous brightness value $L_{t-1}$ and
a single case of not shifting. The case with the minimum error value was checked against
the total of 17 cases, including 16 cases where the previous brightness value $L_{t-1}$
is shifted and one case when it is not shifted.
Algorithm 1. LED matrix update algorithm.
3.4.2 LED Address Mask
Regarding turning on the headlight or when the overall brightness value is moved by
the shift command, the error in the brightness value of the edge in the opposite direction
of the shift increases greatly. The LED address mask function can simultaneously assign
one value to multiple LEDs via one command. Through this approach, the error value
in the opposite edge can be reduced.
For the edge opposite to the shift direction, the value $L_{t}$ which is transmitted
from the ADAS and compared to the average value of the $L_{t}$, which is $L_{e}$,
and the value $L_{s}$, which is the LED values after a shift. Then, whether a masked
write operation is performed on the opposite edge is decided by checking the error.
If the error $\left| L_{t}-L_{e}\right| $ is less than the error $\left| L_{t}-L_{s}\right|
$, the value $L_{e}$ is sent to the corresponding LED matrix edge using the address
mask operation.
For the edge opposite to the shift direction, the value $L_{t}$ which is transmitted
from the ADAS and compared to the average value of the $L_{t}$, which is $L_{e}$,
and the value $L_{s}$, which is the LED values after a shift. Then, whether a masked
write operation is performed on the opposite edge is decided by checking the error.
If the error $\left| L_{t}-L_{e}\right| $ is less than the error $\left| L_{t}-L_{s}\right|
$, the value $L_{e}$ is sent to the corresponding LED matrix edge using the address
mask operation.
3.4.3 Modification Per Pixel
When the maximum number of commands to the LED matrix is limited to 6 commands per
frame, up to 6 commands can be used to modify pixels if no shift and mask commands
are necessary. On the other hand, up to 4-pixel modification commands can be used
to modify pixels if both shift and mask commands are used. If $L_{p}$ is the pixel
values of the LED Matrix after the shift and mask process, the pixel error value $L_{d}=\left|
L_{t}-L_{p}\right| $ is calculated, and then the pixel values of $L_{d}$ are sorted
to obtain the pixel locations of maximum errors for remaining commands to send.
4. Experiment
The microprocessor part was implemented using a Raspberry Pi 4 (8GB), and the LED
driver was implemented using a ZYNQ-7000 of ZYNQ start KIT. The Raspberry Pi 4 and
ZYNQ start KIT were connected using PiCAN and Pmod CAN. The LED matrix used was a
WS2812B matrix. Dash cam videos were used for the experiments instead of the ADAS
data. In the experiments, we used three special driving situations, which can significantly
change the LED matrix values: a speed bump, sharp left turn, and sharp right turn,
as shown in Fig. 9. For practical use, the RGB value of the LED must be used to make a decision. However,
in the experiment, the brightness value Y of YCbCr was used and shown to avoid complicated
side effects of color mixing.
Fig. 8. The proposed LED matrix headlight system.
Fig. 9. Images from videos for experiments.
4.1 Number of Data Sent
The system was designed to use a speed of 500 kbps, which is a stable speed for CAN
communication. However, the number of data to send should be much smaller because
of other devices sharing the same CAN line. Fig. 10 shows how the number of new LED data sent affects the mean absolute error (MAE).
By sending several new data for maximum error positions, the MAE is maintained within
a manageable range. It also shows that the effect of sending more than 6 data is not
significant.
Fig. 10. MAE when transmitting 6, 7, 8, 9, and 10 data without shift and line filling operation.
4.2 Shift Operations
Shift experiments were conducted for certain changes. For example, in the case of
up and down movement, the image shift when crossing a speed bump was considered. The
image shift during left and right turns was also studied. The results of shift experiments
are plotted in the Fig. 11 with the MAE. As shown in the Fig. 11, the shift operation significantly reduces the MAE, especially when subunit shift
is allowed.
Fig. 11. MAE with 1, 1/2, and 1/4 unit shift level operations. The number of data sent is 6, and no line filling is used.
4.3 Line Filling Operation
Although the system was designed to keep the data in the line of the other side for
the shift operation, new data can emerge in that area, which can increase the MAE.
A line filling operation using the mask function of the system can reduce the error.
As shown in Fig. 12, the error is high at the moment of sharp movement. The figure also shows that the
error is decreased by using the line filling operation.
Fig. 12. MAE with line filling operations. The shift amount was set to 1/4 units, and the minimum number of data sent was 4.
5. Conclusion
In this study, we proposed an algorithm that can adjust the brightness of LEDs based
on the structural design of LED matrix control and CAN interface. By inserting a filter
that uses the surrounding values during a shift, subpixel unit shift was possible.
The proposed LED matrix headlight system can update the LED matrix values to follow
up the whole information given by the ADAS system with significantly small amount
of control commands. The experiment shows that the proposed system can update the
LED matrix headlight less than 6 commands per frame.
ACKNOWLEDGMENTS
This work was supported in part by the National Research Foundation of Korea (NRF)
funded by the Korea government(MSIT) (NRF-2021R1F1A1060183), and in part by the MOTIE
(Ministry of Trade, Industry, and Energy) in Korea, under the Fostering Global Talents
for Innovative Growth Program (P0017308) supervised by the Korea Institute for Advancement
of Technology (KIAT), and in part by the Research Grant of Kwangwoon University in
2022. We thank Professor Kung-Yen Lee for his assistance during the research.
REFERENCES
DONAHOE, Daniel N. Thermal aspects of LED automotive headlights. In: 2009 IEEE Vehicle
Power and Propulsion Conference. IEEE, 2009. p. 1193-1199.
Paul A. Thompson, “Daytime Running Lamps (DRLs) for Pedestrian Protection,” SAE Technical
Paper 2003-01-2072, (May, 2003): p. 1.
Bong-hyo Kim, Seong-Won Lee, “Implementation of the Smart Headlight Control System
Using an LED Matrix”, JKICS vol. 46, no. 3, (Mar, 2021), p. 549-55.
Hyukmin lee, “Technology Trends and Prospects of intelligent Lamp”, Journal of the
Korean Society of Automotive Engineers, (Sep, 2018), P. 41-43.
https://blog.naver.com/teslakr_official/222947658542
https://group-media.mercedes-benz.com/marsMediaSite/en/instance/ko/MULTIBEAM-LED-headlamps-The-future-of-light.xhtml?oid=9905065
https://www.bmw.com.bn/en/topics/offers-and-services/accessories-pages/bmw-accessory-showroom.bmw-adaptive-led-headlights.html
https://www.audi.com/en/innovation/e-mobility/matrix-led-headlights.html
https://www.volkswagen-newsroom.com/en/evolution-of-light-4261/headlight-technologies-iqlight-4267
https://tech.hyundaimotorgroup.com/article/genesiss-cutting-edge-intelligent-front-lighting-system/
Ziebinski, Adam, et al. "Review of advanced driver assistance systems (ADAS)." AIP
Conference Proceedings. Vol. 1906. No. 1. AIP Publishing LLC, 2017.
Son, Chang-Koan, et al. "A Study on Automotive Diagnostic System using CAN, CAN FD,
FlexRay." The Journal of The Institute of Internet, Broadcasting and Communication
16.6 (2016): 311-318.
Seo, Sukhyun. "Performance Analysis of CAN-FD Based Network Against Network Topology."
IEMEK Journal of Embedded Systems and Applications 12.6 (2017): 351-358.
https://www.youtube.com/watch?v=78HRi0cIr-8
https://www.youtube.com/watch?v=U6ytVFVPYds