CN111193901A - An image transmission method, imaging device, system and vehicle - Google Patents
An image transmission method, imaging device, system and vehicle Download PDFInfo
- Publication number
- CN111193901A CN111193901A CN201811362126.8A CN201811362126A CN111193901A CN 111193901 A CN111193901 A CN 111193901A CN 201811362126 A CN201811362126 A CN 201811362126A CN 111193901 A CN111193901 A CN 111193901A
- Authority
- CN
- China
- Prior art keywords
- data stream
- image data
- image
- pixel
- main
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 57
- 238000003384 imaging method Methods 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000015654 memory Effects 0.000 claims abstract description 36
- 238000012545 processing Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000000295 complement effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
本发明公开了一种图像的传输方法、成像设备、系统及车辆，所述方法包括：获得图像数据流；将图像数据流中的每一帧图像的各个像素的数据进行分离，形成主图图像数据流和副图图像数据流；所述主图图像数据流中包括每一帧图像的主图图像，用于表达图像的主体内容，能够被直接用于显示；将分离成的主图图像数据流和副图图像数据流写入存储器；从存储器中读出主图图像数据流并发送给接收设备，以使得所述接收设备根据所述主图图像数据流进行图像显示。本发明能够减轻接收设备CPU的运行负担，实现更快的图像显示。
The invention discloses an image transmission method, an imaging device, a system and a vehicle. The method includes: obtaining an image data stream; separating data of each pixel of each frame of image in the image data stream to form a main image image Data stream and sub-image data stream; the main image data stream includes the main image of each frame of image, which is used to express the main content of the image and can be directly used for display; the main image data that will be separated into The stream and the sub-picture image data stream are written into the memory; the main-picture image data stream is read from the memory and sent to the receiving device, so that the receiving device performs image display according to the main-picture image data stream. The invention can reduce the running burden of the CPU of the receiving device and realize faster image display.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image transmission method, an imaging device, an imaging system, and a vehicle.
Background
With the development of the smart industry, robot obstacle avoidance, Simultaneous Localization And Mapping (SLAM) technology, And unmanned technology, imaging devices (e.g., cameras) are widely used in these fields. Most of the imaging devices currently employ Charge Coupled Devices (CCD) and Complementary Metal-Oxide-Semiconductor (CMOS) image sensors, and CMOS image sensors occupy more and more important positions in the field of image processing by virtue of their characteristics of low cost, low power consumption, high integration degree, and the like.
Currently, most CMOS image sensors on the market output Bayer (Bayer) images, and each color component in each pixel of most Bayer images occupies 12 bits (bit) depth, has 4096 color levels, and has more information than 8bit (having 256 color levels) images. Therefore, in order to obtain more information, the imaging device is required to transmit all the 12-bit data of each color component of each pixel to a receiving device (such as a computer, a server, etc.), so as to perform operations of extracting, identifying, etc. the image content. However, in general, an image needs to be displayed on a display device in time for a person to watch, and then whether the imaging device is focused correctly, whether a window is accurate, and the like are judged from a subjective angle. However, most display devices (general-purpose displays) currently support display of images of only 8bit color component depth, and images of 12bit color component depth are difficult to directly display on a display device. Therefore, there is a need for a transmission method of image data so that a display device of a receiving device can conveniently perform image display.
Disclosure of Invention
In view of the technical drawbacks and disadvantages of the prior art, embodiments of the present invention provide a method, an imaging apparatus, a system, and a vehicle for transmitting an image, which overcome or at least partially solve the above problems.
As an aspect of the embodiments of the present invention, there is provided an image transmission method including:
obtaining an image data stream;
separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying;
writing the separated main image data stream and sub image data stream into a memory;
a primary image data stream is read from a memory and transmitted to a receiving device for image display by the receiving device in accordance with the primary image data stream.
In one embodiment, the method further comprises:
receiving an auxiliary graph demand instruction sent by receiving equipment;
and reading the secondary image data stream from the memory and sending the secondary image data stream to the receiving device, so that the receiving device performs bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain the image data stream.
In one embodiment, the separating the data of the pixels of each frame of image in the image data stream to form the main image data stream and the sub-image data stream includes:
separating each color component data of each pixel in each frame image in the image data stream according to high M bits and low N bits; wherein M + N is greater than a predicted maximum allowed transmission bit number threshold;
forming a main image data stream based on the high M-bit data separated from the color component data of each pixel; wherein M is less than or equal to a predicted maximum allowed transmission bit number threshold;
a sub-picture image data stream is formed based on the lower N-bit data separated from the respective color component data in each pixel.
In one embodiment, the forming of the main image data stream based on the high M-bit data separated from the respective color component data of each pixel includes:
forming a main image data stream in the order of pixels by using the high M-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the main image data stream;
the forming of a sub-picture image data stream based on the low N-bit data separated from the respective color component data in each pixel includes:
taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold;
or, each will
The low N-bit data separated from each color component data in each adjacent pixel is combined as data of a corresponding pixel in the sub-image data stream, and the sub-image data stream is formed in the order of pixels.
In one embodiment, the relationship between M and N is α N, α is a preset multiple, α is greater than or equal to 1, and M and N are natural numbers.
In one embodiment, M-8, N-8; or M is 8 and N is 4.
As another aspect of an embodiment of the present invention, there is provided an image forming apparatus including: an image sensor, a processor, and a memory;
the image sensor is used for collecting images and forming an image data stream;
the processor to obtain an image data stream; separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying; writing the separated main image data stream and sub image data stream into a memory; a primary image data stream is read from a memory and transmitted to a receiving device for image display by the receiving device in accordance with the primary image data stream.
In one embodiment, the processor is further configured to:
receiving an auxiliary graph demand instruction sent by receiving equipment;
and reading the secondary image data stream from the memory and sending the secondary image data stream to the receiving device, so that the receiving device performs bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain the image data stream.
In one embodiment, the processor is specifically configured to:
separating each color component data of each pixel in each frame image in the image data stream according to high M bits and low N bits; wherein M + N is greater than a predicted maximum allowed transmission bit number threshold;
forming a main image data stream based on the high M-bit data separated from the color component data of each pixel; wherein M is less than or equal to a predicted maximum allowed transmission bit number threshold;
a sub-picture image data stream is formed based on the lower N-bit data separated from the respective color component data in each pixel.
In one embodiment, the processor is specifically configured to:
forming a main image data stream in the order of pixels by using the high M-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the main image data stream;
the processor is specifically configured to:
taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold;
or, each will
The low N-bit data separated from each color component data in each adjacent pixel is combined as data of a corresponding pixel in the sub-image data stream, and the sub-image data stream is formed in the order of pixels.
In one embodiment, the processor is a field programmable gate array.
In one embodiment, the memory stores the primary image data stream and the secondary image data stream separately.
As a further aspect of an embodiment of the present invention, there is provided an image transmission system including the above-described imaging device and receiving device;
and the receiving device is used for receiving the main image data stream sent by the imaging device and displaying images according to the main image data stream.
In one embodiment, the receiving device is further configured to:
sending a sub-map requirement instruction to the imaging equipment;
and receiving a secondary image data stream sent by an imaging device, and performing bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain an image data stream.
As a final aspect of the embodiment of the present invention, there is provided a vehicle including the above-described image forming apparatus and receiving apparatus;
and the receiving device is used for receiving the main image data stream sent by the imaging device and displaying images according to the main image data stream.
In one embodiment, the receiving device is further configured to:
sending a sub-map requirement instruction to the imaging equipment;
and receiving a secondary image data stream sent by an imaging device, and performing bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain an image data stream.
The embodiment of the invention at least realizes the following technical effects:
according to the image transmission method, the imaging device, the system and the vehicle provided by the embodiment of the invention, the separation of the data of each pixel of each frame of image in the image data stream can be realized in a processor chip (such as an FPGA) with an image processing capacity, a main image data stream and a secondary image data stream are formed, and the transmission of the image data stream is realized after the main image data stream and the secondary image data stream are cached in a memory. According to the embodiment of the invention, according to the requirement of the receiving equipment, when the receiving equipment only needs to receive and display the main image, only the data of the data stream of the main image needs to be transmitted to the receiving equipment, so that for some receiving equipment with low CPU performance, the transmitted data volume is reduced, the operation burden of the CPU can be reduced undoubtedly, and faster image display is realized.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a schematic diagram of a prior art image pixel data storage;
FIG. 2 is a flowchart of an image transmission method according to an embodiment of the present invention;
FIG. 3 is a flowchart of an image transmission method for transmitting a complete image data stream according to an embodiment of the present invention;
FIG. 4 is a flow chart of image pixel data separation according to an embodiment of the present invention;
FIG. 5 is a first schematic diagram illustrating storage of a primary image data stream and a secondary image data stream according to an embodiment of the present invention;
FIG. 6 is a second schematic diagram of the storage of the main image data stream and the sub image data stream according to the embodiment of the present invention;
fig. 7 is a structural view of an image forming apparatus provided in an embodiment of the present invention;
fig. 8 is a schematic diagram of an image transmission system according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to make the present invention better understood by those skilled in the art, some technical terms related to the present invention are explained below:
FPGA: Field-Programmable Gate Array.
Bayer image: that is, a bayer image is an image formed using a bayer array, which is one of the main technologies for realizing a CCD or CMOS sensor to capture a color image.
A CCD image sensor: charge Coupled Device, ccd image sensor.
A CMOS image sensor: complementary Metal-Oxide-Semiconductor (CMOS) image sensors.
DMA: direct Memory Access, can be implemented by hardware devices of different speeds.
DDR: double Data Rate, Double Data synchronous dynamic random access memory, is one of the memories.
In implementing the embodiments of the present invention, the inventors found that the transmission of image data can be performed in the following manner in general:
as shown in fig. 1, taking transmission of Bayer image data with a depth of 12 bits as an example, due to transmission requirements of network data, 1 byte can generally store 8 bits of data, and in general, for data transmission, only storage of image data is concerned, that is, each 12 bits of pixel data is split and spliced, so that each byte stores 8 bits of data. For 12-bit image data, it is generally necessary to split and join data of each pixel to form whole byte data, and to mashup the data together before transmission. For example P in FIG. 11_8bit、P2_8bit、P3_8bit、P4_8bitRespectively 1 st to 4 th pixels (only 4 pixels are taken as an example here, it should be noted that, in a frame of image data, several pixel data may be included), 8bit of data, and P1_4bit、P2_4bit、P3_4bit、P4_4bitThe other 4 bits of data of the 1 st to 4 th pixels respectively,wherein P is1_8bit、P2_8bit、P3_8bit、P4_8bitOccupy 1 byte to store, P, respectively1_4bitAnd P2_4bitSpliced together to take 1 byte to store, P3_4bitAnd P4_4bitThe splicing together occupies 1 byte for storage, and by analogy, the pixel data in the whole frame of image data can be split and spliced to form a structure similar to that shown in fig. 1 for storage.
Thus, when the receiving device needs to display an image, the receiving device needs to traverse each pixel from the above structure, recover the data of each pixel back to 12-bit image data, and convert the data into 8-bit image data in a linear stretching or nonlinear mapping manner for display. This increases the burden on a Central Processing Unit (CPU) of the receiving apparatus to some extent, and for a receiving apparatus with lower performance, there is a risk of affecting image display. In addition, the mode of splicing and mashup transmission of the pixel data in the image also increases the difficulty of traversing and recovering each pixel, so that the processing of the receiving equipment is slow, and the real-time display of the image is difficult.
Various specific embodiments of the image transmission method, the imaging device, the system and the vehicle according to the embodiments of the present invention are described in detail below.
Referring to fig. 2, a method for transmitting an image according to an embodiment of the present invention includes the following steps:
and S21, obtaining an image data stream.
And S22, separating the data of each pixel of each frame image in the image data stream to form a main image data stream and a secondary image data stream.
The main image data stream comprises a main image of each frame of image, and the main image data stream is used for expressing the main content of the image and can be directly used for displaying.
And S23, writing the separated main image data stream and the sub image data stream into a memory.
S24, reading the main image data stream from the memory and transmitting to the receiving device to cause the receiving device to display an image in accordance with the main image data stream.
According to the image transmission method provided by the embodiment of the invention, when the receiving device only needs to display the image in a certain time period and no requirement or low requirement is provided for the display quality of the image, only the main image data stream needs to be sent to the receiving device for the operation processes of display, processing and the like of the receiving device, so that the transmitted data volume is reduced, the operation load of a CPU (central processing unit) of the receiving device can be reduced, and faster image display is realized.
The main image according to the embodiment of the present invention generally includes most of the information of the image, and is used to express the main content of the image, and can be directly used for display. The secondary image, in general, contains some detail information of the image that complements the content of the primary image, and cannot be used directly for display.
In the embodiment of the present invention, each frame of image in the image data stream is formed by combining a plurality of pixels, each pixel further includes a plurality of color components, and data of each color component occupies a certain byte. The obtained image data may be various, for example, an image with a depth of 8bit color component, an image with a depth of 12bit color component, an image with a depth of 16bit color component, and the like, and the embodiment of the present invention is not limited thereto.
For convenience of description, an image of the Tbit color component depth (the size of T is equal to the depth of each color component in a pixel, and the size may be 4, 8, 12 or other numerical values), which is simply referred to as Tbit, for example, an image of the 12-bit color component depth is simply referred to as 12-bit, although the specific size of the color component depth is not limited in the embodiments of the present invention.
In one embodiment, when the receiving device needs to display the entire content of the image or apply the original image data stream, the obtained image data stream needs to be completely transmitted to the receiving device. Specifically, referring to fig. 3, the method includes the following steps:
and S31, receiving the sub-map requirement instruction sent by the receiving equipment.
And S32, reading the sub-image data stream from the memory and sending the sub-image data stream to the receiving device, so that the receiving device performs bit operation on the data of each pixel of each frame of image in the main image data stream and the sub-image data stream to obtain an image data stream.
Here, the receiving device, upon receiving the data of the respective pixels of each frame image in the main image data stream and the sub image data stream, performs bit arithmetic processing on the received data so that the main image data stream and the sub image data stream in the separated state can be restored to the original image data streams.
Thus, the receiving device can recover the original image data stream through simple bit operation, and the subsequent processing (such as storage, display, data analysis, etc.) of the receiving device is facilitated. In addition, the speed of bit operation is the fastest for most of the CPUs of the receiving devices, so that the operation load of the CPUs of the receiving devices can be reduced. For a lower performance receiving device, the risk of image display can be reduced.
Compared with the method of splicing and mashup transmission of the pixel data in the image, the method separates the data of each pixel of each frame of image in the image data stream, and selects whether to read out the sub-image data stream from the memory according to the requirement of the receiving equipment. The method avoids the data of all pixels in the image from being spliced and mashup together for transmission, reduces the difficulty of traversing and recovering each pixel, and enables the receiving equipment to process more quickly, thereby displaying the image in real time.
In one embodiment, referring to fig. 4, in the step S22, the data of each pixel of each frame image in the image data stream is separated to form the main image data stream and the sub-image data stream, which can be implemented as follows:
s41, separating each color component data of each pixel in each frame image in the image data stream according to high M bits and low N bits; wherein M + N is greater than a predicted maximum allowed transmission bit number threshold.
S42, a main image data stream is formed based on the high M-bit data separated from the color component data of each pixel. Wherein M is less than or equal to a predicted maximum allowed transmission bit number threshold.
S43, a sub-picture data stream is formed based on the low N-bit data separated from each color component data in each pixel.
It should be noted that, the manner of separating the data of each pixel of each frame image in the image data stream may be various, for example, the data may be separated according to the upper M bits and the lower N bits of each color component, or may be separated according to the lower M bits and the upper N bits of each color component, or may be a separation manner in which the N bits in each color component are a main image data stream, and the other bits are a sub image data stream, and the like, and the embodiment of the present invention is not limited thereto.
Here, the maximum allowable transmission bit number threshold refers to a maximum number of bits that data of each pixel of each frame of image in the image data stream can be allowed to be transmitted during the image transmission process; or the maximum number of bits that the receiving device can allow for the display of data for individual pixels of each frame of image in the image data stream. For example, in network data transmission, each Byte (Byte) is generally transmitted according to 8 bits, and the maximum allowed transmission bit number threshold may be 8.
In the embodiment of the present invention, the value of M + N is generally greater than the predicted maximum allowable transmission bit number threshold; if the value of M + N is less than or equal to the predicted maximum allowable transmission bit number threshold value, the image data stream does not need to be separated and can be directly used for image transmission and display. In addition, both M and N should be equal to or less than the predetermined maximum allowed transmission bit number threshold, i.e., the maximum value of M and N should not exceed the predetermined maximum allowed transmission bit number threshold.
In one embodiment, the formation of the main image data stream based on the high M-bit data separated from the color component data of each pixel in step S42 described above may be implemented by:
the high M-bit data separated from the color component data of each pixel is used as the data of the corresponding pixel in the main image data stream, and the main image data stream is formed according to the order of the pixels.
The sub-image data stream formed in step S43 based on the low-N bit data separated from the color component data in each pixel may be implemented in one of the following two ways, see the following first way and second way:
the first method is as follows: taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold.
Or adopting a second mode: will each be
The low N-bit data separated from each color component data in each adjacent pixel is combined as data of a corresponding pixel in the sub-image data stream, and the sub-image data stream is formed in the order of pixels.
In the embodiment of the present invention, each color component data of each pixel in each frame image in the image data stream is described in detail in a manner of separating the high M bits and the low N bits.
In one embodiment, it is assumed that M and N have a relationship of α N, α is a preset multiple, α is greater than or equal to 1, and M and N are natural numbers.
In one embodiment, M-8, N-8; or M is 8 and N is 4.
Of course, the embodiments of the present invention do not limit specific values of α, M, and N, and other possible cases of α, M, and N are not listed here as long as all values that can implement the above technical solutions provided by the embodiments of the present invention are feasible.
For ease of understanding, two practical examples are illustrated:
example one:
the method comprises the steps of obtaining an image data stream, wherein the obtained image data stream is a 16-bit image when α is equal to 1, M is equal to 8 and N is equal to 8, separating each color component data of each pixel in each frame image in the 16-bit image data stream according to the upper 8 bits and the lower 8 bits, forming a main image data stream according to the sequence of the pixels by using the upper 8 bits of data separated from each color component data of each pixel as the data of the corresponding pixel in the main image data stream, and forming a sub-image data stream according to the sequence of the pixels by using the lower 8 bits of data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream.
The obtained 16-bit image data stream is separated into a main image data stream of 8 bits high and a sub-image data stream of 8 bits low. According to the method of carrying out data transmission and storage by taking bytes as units, the main image data stream with 8 bits high or the secondary image data stream with 8 bits low can be directly used for data transmission and storage.
Referring to fig. 5, for an image with an original size of 16bit in an image data stream, the separated main image data stream and sub-image data stream are stored in the memory in the following pixel order:
a1 is the high 8-bit data for the first pixel, and A2 is the high 8-bit data … … An for the second pixel, which is the high 8-bit data for the nth pixel.
B1 is the low 8-bit data of the first pixel, and B2 is the low 8-bit data … … Bn of the second pixel is the low 8-bit data of the nth pixel.
Wherein the pixel value of each pixel in the main image data stream is:
C1＝A1；
C2＝A2；
……
Cn＝An；
in the above formula, C1 is the pixel value of the first pixel in the main image, and C2 is the pixel value … … Cn of the second pixel in the main image is the pixel value of the nth pixel in the main image.
The pixel values of the pixels in the sub-image data stream are:
C1′＝B1；
C2′＝B2；
……
Cn′＝Bn；
in the above equation, C1 ' is the pixel value of the first pixel in the sub-map image, and C2 ' is the pixel value … … Cn ' of the second pixel in the sub-map image is the pixel value of the nth pixel in the sub-map image.
Example two:
the method comprises the steps of obtaining an image data stream, wherein the obtained image data stream is a 12-bit image when α is 2, M is 8 and N is 4, separating each color component data of each pixel in each frame image in the 12-bit image data stream according to high M bits and low N bits, taking high 8-bit data separated from each color component data of each pixel as data of a corresponding pixel in a main image data stream, forming the main image data stream according to the sequence of the pixels, combining low 4-bit data separated from each color component data of every two adjacent pixels to form data of the corresponding pixel in a sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels.
In this case, the obtained 12-bit image data stream is separated into a main image data stream of 8 bits higher and a sub-image data stream of 4 bits lower. The main image data stream with a high 8bit can be directly transmitted and stored, and the secondary image data stream with a low 4bit can be transmitted and stored after being combined into the image data stream with an 8 bit. That is, one pixel data is stored in one byte in the main image data stream; in the sub-image data stream, two adjacent pixel data of the original image are stored within one byte, and the size of the main image data stream is twice as large as that of the sub-image data stream. Thus, the transmission and storage of the combined sub-image data stream are also facilitated.
After separating all the obtained image data streams and forming a main image data stream and a sub image data stream, the FPGA writes data of the main image data stream and data of the sub image data stream in the memory in order of pixels. The memory may be DDR or any other memory device having the same function, and is not limited herein.
Referring to fig. 6, for an original 12bit image, the separated main image data stream and sub-image data stream are stored in the memory in pixel order as follows:
p1 is the high 8-bit data of the first pixel, P2 is the high 8-bit data … … Pn of the second pixel is the high 8-bit data of the nth pixel;
q1 is the low 4-bit data of the first pixel, Q2 is the low 4-bit data … … Qn of the second pixel is the low 4-bit data of the nth pixel;
m1 is the first set of 8bit data consisting of Q1 and Q2, M2 is the second set of 8bit data consisting of Q3 and Q4, … … Mn/2Is a combination of Qn-1 and Qn
Group 8bit data.
If the receiving device only needs to apply the main image, the data stream of the main image is transmitted to the receiving device. For example, the pixel values of the pixels of the main image stream are:
I1＝P1；
I2＝P2；
……
In＝Pn；
in the above formula, I1 is the pixel value of the first pixel In the main image, and I2 is the pixel value … … In of the second pixel In the main image is the pixel value of the nth pixel In the main image.
If the receiving device needs to apply the original image, the main image data stream and the sub-image data stream are both sent to the receiving device, and the receiving device performs bit operation on the data of the main image data stream and the sub-image data stream to obtain the pixel value of the original image data stream.
The process of the receiving device for bit operation is as follows:
I1′＝(P1<<4)|Q1；
I2′＝(P2<<4)|Q2；
……
In′＝(Pn<<4)|Qn；
in the above formula, I1 ' is the pixel value of the first pixel In the original image data stream obtained by bit operation, I2 ' is the pixel value … … In ' of the second pixel In the original image data stream obtained by bit operation is the pixel value of the nth pixel In the original image data stream obtained by bit operation; "Pn < < 4" indicates that the upper 8bit data of the nth pixel is shifted to the left by 4 bits to complement the 4bit data of Qn after Pn.
According to the image transmission method provided by the embodiment of the invention, the original image data stream is separated into the main image data stream and the sub-image data stream, and the main image data stream and the sub-image data stream are selectively sent to the receiving device according to the requirement of the receiving device, or the main image data stream and the sub-image data stream are sent to the receiving device, so that the situation that the data of each pixel of each frame image in the original image data stream are mixed together and sent to the receiving device is avoided, and according to the capability and the requirement of the receiving device, the transmission efficiency can be improved while the display quality of the image of the receiving device is ensured.
Referring to fig. 7, an image forming apparatus according to an embodiment of the present invention includes: an image sensor 71, a processor 72, and a memory 73.
An image sensor 71 for image acquisition, forming an image data stream;
a processor 72 for obtaining a stream of image data; separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying; writing the separated main image data stream and sub image data stream into the memory 73; the main image data stream is read from the memory 73 and transmitted to the receiving device to enable the receiving device to display an image in accordance with the main image data stream.
In one embodiment, the processor 72 may be any of a variety of processor chips with image Processing capabilities, such as a conventional FPGA, a Digital Signal Processor (DSP), or other similar processor.
In one embodiment, processor 72 is further configured to: receiving an auxiliary graph demand instruction sent by receiving equipment; the sub-image data stream is read from the memory 73 and sent to the receiving device so that the receiving device performs a bit operation on the data of each pixel of each frame of image in the main image data stream and the sub-image data stream to obtain an image data stream.
In one embodiment, processor 72 is specifically configured to: separating each color component data of each pixel in each frame image in the image data stream according to high M bits and low N bits; wherein M + N is greater than a predicted maximum allowed transmission bit number threshold; forming a main image data stream based on the high M-bit data separated from the color component data of each pixel; wherein M is less than or equal to a predicted maximum allowed transmission bit number threshold; a sub-picture image data stream is formed based on the lower N-bit data separated from the respective color component data in each pixel.
In one embodiment, processor 72 is specifically configured to: forming a main image data stream in the order of pixels by using the high M-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the main image data stream; taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold; or, each will
The low N-bit data separated from each color component data in each adjacent pixel is combined as data of a corresponding pixel in the sub-image data stream, and the sub-image data stream is formed in the order of pixels.
In one embodiment, the memory 73 stores the primary image data stream and the secondary image data stream separately. Thereby facilitating the separate reading of the primary image data stream and the secondary image data stream while avoiding blending the two together.
An image transmission system according to an embodiment of the present invention, shown in fig. 8, includes an imaging device 81 and a receiving device 82. For a specific operation process of the imaging device 81, reference is made to the specific implementation of the imaging device corresponding to fig. 7, which is not described herein again. The receiving device 82 is configured to receive the main image data stream transmitted from the imaging device 81 and perform image display based on the main image data stream.
In one embodiment, the receiving device 82 is further configured to: transmitting a sub-map requirement instruction to the imaging device 81; and receiving the secondary image data stream sent by the imaging device 81, and performing bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain an image data stream.
The receiving device may be, for example, a computer, a server, or other devices, and the embodiment of the present invention is not limited thereto.
In addition, the embodiment of the invention also provides a vehicle, which comprises the imaging device and the receiving device; for specific working processes of the imaging device and the receiving device, reference is made to the specific implementation of the imaging device and the receiving device corresponding to fig. 7 and fig. 8, which is not described herein again.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (16)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811362126.8A CN111193901B (en) | 2018-11-15 | 2018-11-15 | Image transmission method, imaging device, system and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811362126.8A CN111193901B (en) | 2018-11-15 | 2018-11-15 | Image transmission method, imaging device, system and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111193901A true CN111193901A (en) | 2020-05-22 |
| CN111193901B CN111193901B (en) | 2021-11-23 |
Family
ID=70708891
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811362126.8A Active CN111193901B (en) | 2018-11-15 | 2018-11-15 | Image transmission method, imaging device, system and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111193901B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113747046A (en) * | 2020-05-29 | 2021-12-03 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1438612A (en) * | 1995-02-03 | 2003-08-27 | 株式会社东芝 | Image information encoding/decoding system |
| CN1506971A (en) * | 2002-12-09 | 2004-06-23 | ������������ʽ���� | Semiconductor device, image data processing apparatus and method |
| US20080303758A1 (en) * | 2007-06-08 | 2008-12-11 | Yoshihisa Ooishi | Display Device |
| CN103858436A (en) * | 2011-10-17 | 2014-06-11 | 索尼公司 | Transmission device, transmission method and reception device |
| US20140328538A1 (en) * | 2013-05-03 | 2014-11-06 | Samsung Electronics Co., Ltd. | Image signal processing device and method and image processing system using the same |
| CN106791858A (en) * | 2017-01-09 | 2017-05-31 | 国家新闻出版广电总局电影数字节目管理中心 | A kind of high sampling rate rgb video data lossless compression method |
-
2018
- 2018-11-15 CN CN201811362126.8A patent/CN111193901B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1438612A (en) * | 1995-02-03 | 2003-08-27 | 株式会社东芝 | Image information encoding/decoding system |
| CN1506971A (en) * | 2002-12-09 | 2004-06-23 | ������������ʽ���� | Semiconductor device, image data processing apparatus and method |
| US20080303758A1 (en) * | 2007-06-08 | 2008-12-11 | Yoshihisa Ooishi | Display Device |
| CN103858436A (en) * | 2011-10-17 | 2014-06-11 | 索尼公司 | Transmission device, transmission method and reception device |
| US20140328538A1 (en) * | 2013-05-03 | 2014-11-06 | Samsung Electronics Co., Ltd. | Image signal processing device and method and image processing system using the same |
| CN106791858A (en) * | 2017-01-09 | 2017-05-31 | 国家新闻出版广电总局电影数字节目管理中心 | A kind of high sampling rate rgb video data lossless compression method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113747046A (en) * | 2020-05-29 | 2021-12-03 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
| CN113747046B (en) * | 2020-05-29 | 2023-06-06 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111193901B (en) | 2021-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10110936B2 (en) | Web-based live broadcast | |
| US10848696B2 (en) | Apparatus for encoding image, apparatus for decoding image and image sensor | |
| CN104902193B (en) | A kind of method for being split processing and display to image data based on FPGA | |
| US20160381338A1 (en) | System and method for storing image data in parallel in a camera system | |
| KR20200011000A (en) | Device and method for augmented reality preview and positional tracking | |
| CN105979201A (en) | Intelligent wearable device based on parallel processor | |
| CN110225316B (en) | Software and hardware cooperative multi-channel video processing device and system | |
| CN111193901B (en) | Image transmission method, imaging device, system and vehicle | |
| US11302035B2 (en) | Processing images using hybrid infinite impulse response (TTR) and finite impulse response (FIR) convolution block | |
| US10304213B2 (en) | Near lossless compression scheme and system for processing high dynamic range (HDR) images | |
| US11803933B2 (en) | Data processing method and sensor device for performing the same | |
| CN108174084A (en) | panoramic video processing method and terminal device | |
| JP2004312073A (en) | Image processing device and camera thereof | |
| WO2023010755A1 (en) | Hdr video conversion method and apparatus, and device and computer storage medium | |
| CN111726621B (en) | Video conversion method and device | |
| US8482438B2 (en) | Data processing device and data processing method | |
| CN113033616B (en) | High-quality video reconstruction method, device, equipment and storage medium | |
| CN103428496A (en) | Device for real-time transmission, compression and storage of images of space TDICCD (Time Delay and Integration Charge Coupled Device) camera | |
| TWI847256B (en) | A video encoding method, a video decoding method and related apparatus | |
| CN115967784A (en) | Image transmission processing system and processing method based on MIPI CSI C-PHY protocol | |
| CN109688314B (en) | Camera system and method with low delay, less cache and controllable data output mode | |
| CN108012074A (en) | Camera chain and the operation method for the camera chain | |
| CN114283065A (en) | A hardware-accelerated ORB feature point matching system and matching method | |
| CN109274955B (en) | Compression and synchronization method and system for light field video and depth map, and electronic equipment | |
| CN207924741U (en) | A kind of super-resolution image reconstruction system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |