CN115988316A - Image acquisition system and method based on virtual main board and hardware main board - Google Patents
Image acquisition system and method based on virtual main board and hardware main board Download PDFInfo
- Publication number
- CN115988316A CN115988316A CN202211669131.XA CN202211669131A CN115988316A CN 115988316 A CN115988316 A CN 115988316A CN 202211669131 A CN202211669131 A CN 202211669131A CN 115988316 A CN115988316 A CN 115988316A
- Authority
- CN
- China
- Prior art keywords
- main board
- model
- image
- virtual
- hardware
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 49
- 230000008054 signal transmission Effects 0.000 claims abstract description 38
- 230000002093 peripheral effect Effects 0.000 claims abstract description 13
- 230000008569 process Effects 0.000 claims abstract description 8
- 238000004891 communication Methods 0.000 claims description 12
- 238000003672 processing method Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000012937 correction Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 5
- 230000001360 synchronised effect Effects 0.000 claims description 5
- 230000006870 function Effects 0.000 description 19
- 238000010586 diagram Methods 0.000 description 10
- 238000011161 development Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Studio Devices (AREA)
Abstract
本申请提供基于虚拟主板和硬件主板的图像获取系统及方法。所述系统包括:虚拟主板和硬件主板;虚拟主板上配置有图像采集模型、信号传输模型、图像处理模型及外围模型;图像采集模型监听虚拟主板对摄像头的配置信息,将虚拟主板对摄像头的配置信息和抓取图像的请求发送至硬件主板,并从硬件主板接收摄像头采集的图像;硬件主板在接收到抓取图像的请求后,控制与硬件主板相连的摄像头采集图像,并将图像传输至图像采集模型;信号传输模型从图像采集模型接收图像,并转发至图像处理模型;图像处理模型对图像进行处理,并存储。本申请规避了单独使用虚拟主板,无法实时抓取现实图像的缺点,有效提高虚拟主板获取图像的速度。
The application provides an image acquisition system and method based on a virtual main board and a hardware main board. The system includes: a virtual main board and a hardware main board; an image acquisition model, a signal transmission model, an image processing model and a peripheral model are configured on the virtual main board; The information and the request for capturing the image are sent to the hardware main board, and the image collected by the camera is received from the hardware main board; after receiving the request for capturing the image, the hardware main board controls the camera connected to the hardware main board to collect the image, and transmits the image to the image The acquisition model; the signal transmission model receives images from the image acquisition model and forwards them to the image processing model; the image processing model processes and stores the images. This application avoids the shortcoming of being unable to capture real images in real time by using the virtual main board alone, and effectively improves the image acquisition speed of the virtual main board.
Description
技术领域technical field
本申请属于系统仿真技术领域,特别是涉及一种基于虚拟主板和硬件主板的图像获取系统及方法。The application belongs to the technical field of system simulation, and in particular relates to an image acquisition system and method based on a virtual main board and a hardware main board.
背景技术Background technique
现有的摄像头图像处理调试和嵌入式软件开发存在如下缺点:Existing camera image processing debugging and embedded software development have the following shortcomings:
1)ISP的内部设计和功能更新后需要FPGA或者SoC部门将ISP集成进一个FPGA或SoC内部,才能启动操作系统,加载驱动等一系列工作后,才能实现从摄像头抓取实时图像并给到ISP处理,测试调优。1) After the internal design and functions of the ISP are updated, the FPGA or SoC department needs to integrate the ISP into an FPGA or SoC to start the operating system, load the driver, and a series of tasks before capturing real-time images from the camera and sending them to the ISP. Processing, testing and tuning.
2)时间久,成本高,在ISP版本频繁更新的情况下,无法做到快速迭代更新,拖累ISP新版本推向市场的速度。2) It takes a long time and the cost is high. In the case of frequent updates of the ISP version, it is impossible to achieve rapid iterative updates, which hinders the speed of the new version of the ISP to the market.
发明内容Contents of the invention
本申请提供基于虚拟主板和硬件主板的图像获取系统及方法,用于提高虚拟主板获取图像的速度。The present application provides an image acquisition system and method based on a virtual mainboard and a hardware mainboard, which are used to increase the speed of image acquisition by the virtual mainboard.
第一方面,本申请实施例提供一种基于虚拟主板和硬件主板的图像获取系统,包括:虚拟主板和与所述虚拟主板相连的硬件主板;其中:所述虚拟主板上配置有图像采集模型、信号传输模型、图像处理模型以及外围模型;所述虚拟主板通过一虚拟化平台构建;所述图像采集模型监听所述虚拟主板对摄像头的配置信息,将所述虚拟主板对摄像头的配置信息和抓取图像的请求发送至所述硬件主板,并从所述硬件主板接收硬件主板的摄像头采集的图像;所述硬件主板在接收到所述抓取图像的请求后,控制与所述硬件主板相连的摄像头采集图像,并将采集的图像传输至所述图像采集模型;所述信号传输模型分别与所述虚拟主板和所述图像采集模型通信相连,从所述图像采集模型接收所述图像,并将接收的所述图像转发至所述图像处理模型;所述图像处理模型分别与所述虚拟主板和所述信号传输模型通信相连,对从所述信号传输模型接收到的图片进行处理。In the first aspect, an embodiment of the present application provides an image acquisition system based on a virtual main board and a hardware main board, including: a virtual main board and a hardware main board connected to the virtual main board; wherein: the virtual main board is configured with an image acquisition model, Signal transmission model, image processing model and peripheral model; The virtual main board is constructed through a virtualization platform; The image acquisition model monitors the configuration information of the virtual main board to the camera, and captures the configuration information of the virtual main board to the camera The request for getting an image is sent to the hardware main board, and the image collected by the camera of the hardware main board is received from the hardware main board; after receiving the request for capturing the image, the hardware main board controls the The camera collects images, and transmits the collected images to the image collection model; the signal transmission model is connected to the virtual main board and the image collection model respectively, receives the image from the image collection model, and sends The received image is forwarded to the image processing model; the image processing model is respectively connected in communication with the virtual main board and the signal transmission model, and processes the pictures received from the signal transmission model.
在第一方面的一种实现方式中,所述外围模型至少包括CPU模型、I2C控制模型、双倍速率同步动态随机存储器模型。In an implementation manner of the first aspect, the peripheral models include at least a CPU model, an I2C control model, and a double-rate synchronous dynamic random access memory model.
在第一方面的一种实现方式中,所述硬件主板上配置有:连接模块,用于与所述虚拟主板建立通信连接;配置信息控制模块,从所述图像采集模型获取所述虚拟主板对摄像头的配置信息,并将该配置信息写入与所述硬件主板相连的摄像头;图像采集模块,在接收到所述抓取图像的请求后,控制与所述硬件主板相连的摄像头采集图像,并将采集的图像存储至所述硬件主板的存储器;图像发送模块,从所述存储器读取所述图像,并将所述图像发送至所述图像采集模型。In an implementation manner of the first aspect, the hardware main board is configured with: a connection module configured to establish a communication connection with the virtual main board; a configuration information control module configured to acquire the pair of virtual main boards from the image acquisition model configuration information of the camera, and write the configuration information into the camera connected to the hardware main board; the image acquisition module, after receiving the request for grabbing an image, controls the camera to collect images connected to the hardware main board, and The collected image is stored in the memory of the hardware motherboard; the image sending module reads the image from the memory and sends the image to the image collection model.
在第一方面的一种实现方式中,所述图像采集模型包括:监听模块,监听所述虚拟主板对摄像头的配置信息,将所述虚拟主板对摄像头的配置信息和抓取图像的请求发送至所述硬件主板,并从所述硬件主板接收硬件主板的摄像头采集的图像;第一寄存器子模块,模拟摄像头寄存器配置和寄存器读写;I2C接口子模块,模拟与所述I2C控制模型进行通信,以通过所述I2C控制模型控制所述寄存器子模型。In an implementation manner of the first aspect, the image acquisition model includes: a monitoring module, which monitors the configuration information of the virtual main board to the camera, and sends the configuration information of the virtual main board to the camera and the request for capturing images to The hardware main board receives images collected by the camera of the hardware main board from the hardware main board; the first register submodule simulates camera register configuration and register reading and writing; the I2C interface submodule simulates communication with the I2C control model, to control the register submodel through the I2C control model.
在第一方面的一种实现方式中,所述信号传输模型包括:第二寄存器模块,模拟串行接口寄存器配置和寄存器读写;图片转发模块,从所述图像采集模型接收所述图像,并将接收的所述图像转发至所述图像处理模型。In an implementation manner of the first aspect, the signal transmission model includes: a second register module that simulates serial interface register configuration and register reading and writing; a picture forwarding module that receives the image from the image acquisition model, and The received image is forwarded to the image processing model.
在第一方面的一种实现方式中,所述图像处理模型至少包括以下一种模块:图片格式转换模块,自动白平衡调整模块,自动对焦模块,色彩校正矩阵模块。In an implementation manner of the first aspect, the image processing model includes at least one of the following modules: an image format conversion module, an automatic white balance adjustment module, an automatic focus module, and a color correction matrix module.
在第一方面的一种实现方式中,所述虚拟主板为嵌入式SoC虚拟主板;所述虚拟主板支持的操作系统包括Windows、Linux、FreeRTOS、VxWorks。In an implementation manner of the first aspect, the virtual main board is an embedded SoC virtual main board; the operating systems supported by the virtual main board include Windows, Linux, FreeRTOS, and VxWorks.
在第一方面的一种实现方式中,所述虚拟主板包括驱动加载模块,加载所述图像采集模型、所述信号传输模型、所述图像处理模型以及所述外围模型的驱动。In an implementation manner of the first aspect, the virtual main board includes a driver loading module for loading drivers of the image acquisition model, the signal transmission model, the image processing model, and the peripheral model.
在第一方面的一种实现方式中,所述虚拟主板和所述硬件主板通过网络连接,USB连接或者PCIe主从模式连接。In an implementation manner of the first aspect, the virtual main board and the hardware main board are connected through a network, a USB connection or a PCIe master-slave mode connection.
第二方面,本申请实施例提供一种虚拟化图像处理方法,应用于如上所述的基于虚拟主板和硬件主板的图像获取系统,所述方法包括:图像采集模型监听虚拟主板对摄像头的配置信息,将所述虚拟主板对摄像头的配置信息和抓取图像的请求发送至硬件主板;所述硬件主板在接收到所述抓取图像的请求后,控制与所述硬件主板相连的摄像头采集图像,并将采集的图像传输至所述图像采集模型;所述图像采集模型从所述硬件主板接收硬件主板的摄像头采集的图像,并将接收到的图像发送至信号传输模型;所述信号传输模型从所述图像采集模型接收所述图像,并将接收的所述图像转发至所述图像处理模型;所述图像处理模型分别与所述虚拟主板和所述信号传输模型通信相连,对从所述信号传输模型接收到的图片进行处理。In the second aspect, the embodiment of the present application provides a virtualized image processing method, which is applied to the above-mentioned image acquisition system based on the virtual main board and the hardware main board, and the method includes: the image acquisition model monitors the configuration information of the virtual main board to the camera , sending the configuration information of the virtual main board to the camera and the request for capturing images to the hardware main board; after receiving the request for capturing images, the hardware main board controls the camera connected to the hardware main board to collect images, And transmit the image collected to the image acquisition model; the image acquisition model receives the image collected by the camera of the hardware motherboard from the hardware motherboard, and sends the received image to the signal transmission model; the signal transmission model from The image acquisition model receives the image, and forwards the received image to the image processing model; the image processing model is respectively connected to the virtual main board and the signal transmission model in communication, and is connected to the signal from the signal The pictures received by the transmission model are processed.
本申请实施例提供的基于虚拟主板和硬件主板的图像获取系统,将虚拟平台与真实硬件开发平台通过网络连接起来,规避了单独使用虚拟平台,无法实时抓取现实图像的缺点,更好的利用了真实摄像头可以实时拍摄图像的优点,有效提高虚拟主板获取图像的速度。The image acquisition system based on the virtual main board and the hardware main board provided by the embodiment of the present application connects the virtual platform and the real hardware development platform through the network, avoiding the shortcomings of using the virtual platform alone and being unable to capture real images in real time, and making better use of The advantages of the real camera that can capture images in real time are realized, and the speed of image acquisition by the virtual main board is effectively improved.
附图说明Description of drawings
图1显示为本申请实施例所述的基于虚拟主板和硬件主板的图像获取系统的整体原理结构示意图;Fig. 1 shows the schematic diagram of the overall principle structure of the image acquisition system based on the virtual main board and the hardware main board described in the embodiment of the present application;
图2显示为本申请实施例所述的基于虚拟主板和硬件主板的图像获取系统中虚拟主板的原理结构示意图;Fig. 2 shows the schematic structural diagram of the principle structure of the virtual main board in the image acquisition system based on the virtual main board and the hardware main board described in the embodiment of the present application;
图3显示为本申请实施例所述的基于虚拟主板和硬件主板的图像获取系统中图像采集模型的原理结构示意图;Fig. 3 shows the schematic structural diagram of the principle structure of the image acquisition model in the image acquisition system based on the virtual main board and the hardware main board described in the embodiment of the present application;
图4显示为本申请实施例所述的基于虚拟主板和硬件主板的图像获取系统中传输接口控制模型的原理结构示意图;Fig. 4 shows the schematic structural diagram of the principle structure of the transmission interface control model in the image acquisition system based on the virtual main board and the hardware main board described in the embodiment of the present application;
图5显示为本申请实施例所述的基于虚拟主板和硬件主板的图像获取系统中图像处理模型的原理结构示意图;Fig. 5 shows the schematic structural diagram of the principle structure of the image processing model in the image acquisition system based on the virtual main board and the hardware main board described in the embodiment of the present application;
图6显示为本申请实施例所述的基于虚拟主板和硬件主板的图像获取系统中硬件主板的原理结构示意图;Fig. 6 shows the principle structural diagram of the hardware motherboard in the image acquisition system based on the virtual motherboard and the hardware motherboard described in the embodiment of the present application;
图7显示为本申请实施例所述的虚拟化图像处理方法的流程示意图。FIG. 7 is a schematic flowchart of a method for processing a virtualized image according to an embodiment of the present application.
元件标号说明Component designation description
100 基于虚拟主板和硬件主板的图像获取系统100 Image acquisition system based on virtual motherboard and hardware motherboard
110 虚拟主板110 virtual motherboard
120 图像采集模型120 Image Acquisition Model
121 监听模块121 Monitoring module
122 第一寄存器子模块122 The first register sub-module
123 I2C接口子模块123 I2C interface sub-module
130 信号传输模型130 Signal Transmission Model
131 第二寄存器模块131 Second register module
132 图片转发模块132 Image forwarding module
140 图像处理模型140 Image processing model
141 图片格式转换模块141 Image format conversion module
142 自动白平衡调整模块142 Automatic white balance adjustment module
143 自动对焦模块143 auto focus module
144 色彩校正矩阵模块144 Color Correction Matrix Module
150 外围模型150 peripheral models
151 CPU模型151 CPU model
152 I2C控制模型152 I2C control model
153 DDR模型153 DDR model
160 存放图片的文件夹160 Folder for storing pictures
170 硬件主板170 hardware motherboard
171 连接模块171 Connection module
172 配置信息控制模块172 configuration information control module
173 图像采集模块173 Image acquisition module
174 图像发送模块174 Image sending module
101 电子设备101 Electronic equipment
1001 处理器1001 Processor
1002 存储器1002 memory
S100~S400 步骤S100~S400 Steps
具体实施方式Detailed ways
以下通过特定的具体实例说明本申请的实施方式,本领域技术人员可由本说明书所揭露的内容轻易地了解本申请的其他优点与功效。本申请还可以通过另外不同的具体实施方式加以实施或应用,本说明书中的各项细节也可以基于不同观点与应用,在没有背离本申请的精神下进行各种修饰或改变。需说明的是,在不冲突的情况下,以下实施例及实施例中的特征可以相互组合。Embodiments of the present application are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.
需要说明的是,以下实施例中所提供的图示仅以示意方式说明本申请的基本构想,遂图式中仅显示与本申请中有关的组件而非按照实际实施时的组件数目、形状及尺寸绘制,其实际实施时各组件的型态、数量及比例可为一种随意的改变,且其组件布局型态也可能更为复杂。It should be noted that the diagrams provided in the following embodiments are only schematically illustrating the basic idea of the application, and only the components related to the application are shown in the diagrams rather than the number, shape and Dimensional drawing, the type, quantity and proportion of each component can be changed arbitrarily during actual implementation, and the component layout type may also be more complicated.
本申请以下实施例提供了一种虚拟化图像处理方法、系统、存储介质及电子设备,通过软件模型实现摄像头图像处理的功能。The following embodiments of the present application provide a virtualized image processing method, system, storage medium, and electronic device, and realize the image processing function of the camera through a software model.
图1显示为本实施例中基于虚拟主板和硬件主板的图像获取系统100的整体原理结构示意图;如图1所示,本实施例提供一种基于虚拟主板和硬件主板的图像获取系统100,该基于虚拟主板和硬件主板的图像获取系统100包括:虚拟主板110和与所述虚拟主板110相连的硬件主板170,所述虚拟主板110和所述硬件主板170通过但不限于网络连接,USB连接或者PCIe主从模式连接。本实施例中以所述虚拟主板110和所述硬件主板170通过网络连接为例进行说明。Fig. 1 shows the overall schematic structural diagram of the
图2显示为本实施例所述的基于虚拟主板110和硬件主板170的图像获取系统100中虚拟主板110的原理结构示意图;如图2所示,所述虚拟主板110上配置有图像采集模型120、信号传输模型130、图像处理模型140以及外围模型150;其中,所述外围模型150至少包括CPU(Central Processing Unit,中央处理器)模型151、I2C(Inter-IntegratedCircuit,两线式串行总线)控制模型152、DDR(Double Data Rate,双倍速率同步动态随机存储器)模型。Fig. 2 shows the schematic structural diagram of the principle structure of the virtual
其中,所述虚拟主板110通过一虚拟化平台构建。Wherein, the
于本实施例中,所述虚拟化平台为但不限于QEMU(Quick EMUlator,虚拟操作系统模拟器)、SNPS VDK(Synopsys VoIP Development Kit,软件开发工具包)、Intel Simics(英特尔全系统虚拟机器)等。其中,本实施例中,所述虚拟主板110通过QEMU虚拟化平台构建。In this embodiment, the virtualization platform is but not limited to QEMU (Quick EMUlator, virtual operating system simulator), SNPS VDK (Synopsys VoIP Development Kit, software development kit), Intel Simics (Intel full system virtual machine) wait. Wherein, in this embodiment, the
QEMU是一个通用的、开源的计算机仿真器和虚拟主板110,支持全系统仿真,可以支持x86(微处理器执行的计算机语言指令集)、ARM(Advanced RISC Machine,高级精简指令集计算机)、MIPS(Million Instructions Per Second,单字长定点指令平均执行速度)、PowerPC(Performance Optimization With Enhanced RISC–Performance Computing,精简指令集(RISC)架构的中央处理器)、RISC-V(基于精简指令集原则的开源指令集架构)等体系架构,并支持Windows(微软视窗操作系统)、Linux(操作系统内核)、FreeRTOS(嵌入式实时操作系统)、VxWorks(实时操作系统)等操作系统,上层软件在虚拟主板110上不做任何修改运行。QEMU is a general-purpose, open-source computer emulator and
其中,QEMU虚拟化平台内置有多个用于构建虚拟主板的设备模型模块,例如,CPU、DDR、中断控制器、串口、I2C控制器、ISP(Image Signal Processing,图像信号处理)、MIPICSI(MIPI CSI-2,Camera Serial Interface,相机串行接口)控制器、摄像头等设备模型模块。构建虚拟主板时,将CPU,DDR,中断控制器,串口,I2C控制器、ISP、MIPI CSI控制器,摄像头等需要的设备模型通过相应的接口连接起来,分配各个设备模型模块的地址空间和中断号等,最终构成一个类似硬件嵌入式主板的虚拟主板110。Among them, the QEMU virtualization platform has built-in multiple device model modules for building virtual motherboards, such as CPU, DDR, interrupt controller, serial port, I2C controller, ISP (Image Signal Processing, image signal processing), MIPICSI (MIPI CSI-2, Camera Serial Interface, camera serial interface) controller, camera and other device model modules. When building a virtual motherboard, connect the required device models such as CPU, DDR, interrupt controller, serial port, I2C controller, ISP, MIPI CSI controller, camera, etc. through corresponding interfaces, and allocate the address space and interrupt of each device model module number, etc., finally forming a
具体地,于本实施例中,所述虚拟主板110为嵌入式SoC虚拟主板110。即本实施例使用QEMU搭建一个完整的嵌入式SoC虚拟主板110,包含CPU模型,I2C控制模式,DDR模型153等必要部件。Specifically, in this embodiment, the virtual
其中,所述虚拟主板110支持的操作系统包括但不限于Windows,Linux,FreeRTOS,VxWorks。Wherein, the operating systems supported by the
于本实施例中,所述虚拟主板110包括驱动加载模块,加载所述图像采集模型120,所述信号传输模型130,所述图像处理模型140以及所述外围模型150的驱动。In this embodiment, the
所以本实施例中的基于虚拟主板和硬件主板的图像获取系统100可以在虚拟主板110上启动例如ARM64的完整Linux或其他操作系统,也可以在虚拟主板110上加载摄像头,I2C控制器,MIPI CSI控制器,ISP等设备驱动。Therefore, the
本实施例中,采用但不限于C语言开发构建图像采集模型120,信号传输模型130,图像处理模型140等部件,并将图像采集模型120,信号传输模型130,图像处理模型140和所述虚拟主板110集成起来。In this embodiment, the
本实施例中,构建图像采集模型120,信号传输模型130,图像处理模型140等就是利用软件把硬件的功能模拟出来,实现其输入输出和寄存器访问控制,使其对外的软件接口跟真实硬件一致。In this embodiment, constructing the
于本实施例中,所述图像采集模型120与所述虚拟主板110通信相连,监听所述虚拟主板110对摄像头的配置信息,将所述虚拟主板110对摄像头的配置信息和抓取图像的请求发送至所述硬件主板170,并从所述硬件主板170接收硬件主板170的摄像头采集的图像。In this embodiment, the
具体地,于本实施例中,如图3所示,所述图像采集模型120包括:监听模块121,第一寄存器子模块122,以及I2C接口子模块123。Specifically, in this embodiment, as shown in FIG. 3 , the
其中,所述监听模块121监听所述虚拟主板110对摄像头的配置信息,将所述虚拟主板110对摄像头的配置信息和抓取图像的请求发送至所述硬件主板170,并从所述硬件主板170接收硬件主板170的摄像头采集的图像;所述第一寄存器子模块122模拟摄像头寄存器配置和寄存器读写;所述I2C接口子模块123用于模拟I2C接口与所述I2C控制模型152进行通信,以通过所述I2C控制模型152控制所述寄存器子模型。Wherein, the
所以本实施例的图像采集模型120可以实现以下功能:Therefore, the
1)实现TCP client并跟硬件主板170连接;1) Realize TCP client and connect with
2)监听虚拟主板110和摄像头驱动对摄像头的配置,将相应寄存器配置通过TCP发给硬件主板170;2) monitor the virtual
3)监听虚拟主板110对摄像头抓取图像的配置,并将抓取图像的请求通过TCP发给硬件主板170;3) monitor the configuration of the virtual
4)通过TCP拿到硬件主板170传过来的摄像头抓取图像,将数据发给后端的信号传输模型130,由信号传输模型130传输到图像处理模型140进行图像处理。4) Get the image captured by the camera transmitted from the
所以本实施例的图像采集模型120还可以实现以下硬件摄像头的功能:Therefore, the
1)功能配置寄存器1) Function configuration register
2)I2C接口,以支持I2C控制器对摄像头寄存器的读写。2) I2C interface to support I2C controller to read and write camera registers.
于本实施例中,所述信号传输模型130分别与所述虚拟主板110和所述图像采集模型120通信相连,从所述图像采集模型120接收所述图像,并将接收的所述图像转发至所述图像处理模型140。In this embodiment, the signal transmission model 130 is connected to the virtual
于本实施例中,所述信号传输模型130为但不限于MIPI CSI控制器(MIPI CSI-2,Camera Serial Interface,相机串行接口)模型。例如还可以为DVP(Digital Video Port,数字视频端口)、(Serial Peripheral Interface,串行外围设备接口)控制器模型。In this embodiment, the signal transmission model 130 is, but not limited to, a MIPI CSI controller (MIPI CSI-2, Camera Serial Interface, camera serial interface) model. For example, it can also be DVP (Digital Video Port, digital video port), (Serial Peripheral Interface, serial peripheral interface) controller model.
具体地,于本实施例中,如图4所示,所述信号传输模型130还包括:第二寄存器模块131和图片转发模块132。Specifically, in this embodiment, as shown in FIG. 4 , the signal transmission model 130 further includes: a second register module 131 and a picture forwarding module 132 .
其中,所述第二寄存器模块131模拟串行接口寄存器配置和寄存器读写;所述图像转发模块132从所述图像采集模型120接收所述图像,并将接收的所述图像转发至所述图像处理模型140。Wherein, the second register module 131 simulates serial interface register configuration and register reading and writing; the image forwarding module 132 receives the image from the
本实施例中,MIPI CSI控制器模型可以实现以下硬件相机串行接口功能:In this embodiment, the MIPI CSI controller model can realize the following hardware camera serial interface functions:
1)功能配置寄存器;1) Function configuration register;
2)接收图像采集模型120发来的图片数据,并转发给后端的图像处理模型140。2) Receive the picture data sent by the
于本实施例中,所述图像处理模型140分别与所述虚拟主板110和所述信号传输模型130通信相连,对从所述信号传输模型130接收到的图片进行处理,并将处理后的图片发送至所述双倍速率同步动态随机存储器模型进行存储。In this embodiment, the
具体地,于本实施例中,如图5所示,所述图像信号处理模型140至少包括以下一种模块:图片格式转换模块141、自动白平衡调整模块142、自动对焦模块143、色彩校正矩阵模块144。Specifically, in this embodiment, as shown in FIG. 5 , the image
所以本实施例的所述图像处理模型140可以实现以下硬件ISP功能:Therefore, the
1)图片格式转换;1) Image format conversion;
2)自动白平衡调整;2) Automatic white balance adjustment;
3)自动对焦;3) Auto focus;
4)色彩校正矩阵。4) Color correction matrix.
如图7所示,本实施例中,所述硬件主板170上配置有:连接模块171、配置信息控制模块172、图像采集模块173以及图像发送模块174。As shown in FIG. 7 , in this embodiment, the hardware
于本实施例中,所述连接模块171用于与所述虚拟主板110建立通信连接。In this embodiment, the
于本实施例中,所述配置信息控制模块172从所述图像采集模型获取所述虚拟主板110对摄像头的配置信息,并将该配置信息写入与所述硬件主板170相连的摄像头。In this embodiment, the configuration information control module 172 obtains the camera configuration information of the virtual
于本实施例中,所述图像采集模块173在接收到所述抓取图像的请求后,控制与所述硬件主板170相连的摄像头采集图像,并将采集的图像存储至所述硬件主板170的存储器(DDR)。In this embodiment, after the image capture module 173 receives the request for capturing an image, it controls the camera connected to the
于本实施例中,所述图像发送模块174从所述存储器读取所述图像,并将所述图像发送至所述图像采集模型。In this embodiment, the image sending module 174 reads the image from the memory, and sends the image to the image acquisition model.
本实施例的基于虚拟主板和硬件主板的图像获取系统100的工作过程如下:The working process of the
一、在QEMU ARM64虚拟主板110上启动Linux操作系统,QEMU的图像采集模型实现如下功能:1. Start the Linux operating system on the QEMU ARM64
2)实现TCP(Transmission Control Protocol,传输控制协议)client(客户端)并跟硬件主板170连接,2) Realize TCP (Transmission Control Protocol, Transmission Control Protocol) client (client) and connect with
3)监听虚拟主板110和摄像头驱动对摄像头的配置,将相应寄存器配置通过TCP发给硬件主板1703) Monitor the configuration of the camera by the virtual
4)监听虚拟主板110对摄像头抓取图像的配置,并将抓取图像的请求通过TCP发给硬件主板1704) Monitor the configuration of the virtual
5)通过TCP拿到硬件主板170传过来的摄像头抓取图像,将数据发给后端的MIPICSI->ISP部件进行图像处理。5) Capture the image from the camera transmitted by the
二、在硬件主板170上启动Linux操作系统,并运行应用程序,该应用程序实现如下功能:Two, start the Linux operating system on the hardware
1)实现TCP server(服务器)并跟QEMU的连接;1) Implement TCP server (server) and connect with QEMU;
2)通过TCP拿到虚拟主板110发送的摄像头配置信息,并通过调用I2C接口将配置信息写入摄像头;2) Get the camera configuration information sent by the
3)监听虚拟主板110发送的抓取图像请求,基于配置硬件开发板上的MIPI CSI控制器,DMA(Direct Memory Access,直接存储器访问)控制器等部件将摄像头拍到的图片放到DDR的缓存(buffer)内部,然后通过TCP将图像数据发送给虚拟主板110。3) Listen to the capture image request sent by the virtual
所以本实施例中的基于虚拟主板110和硬件主板170的图像获取系统具有如下效果:Therefore, the image acquisition system based on the virtual
1)在虚拟主板110和真实硬件上启动Linux,加载驱动后通过网络连接,将虚拟主板110与真实硬件开发平台通过网络连接起来;1) start Linux on the
2)真实硬件上摄像头抓取的图像可以实时的传输给虚拟主板110的摄像头,并进一步传输给虚拟主板110的后端部件,规避了单独使用虚拟主板110,无法实时抓取现实图像的缺点,更好的利用了真实摄像头可以实时拍摄图像的优点;2) The image captured by the camera on the real hardware can be transmitted to the camera of the virtual
3)硬件主板170的应用对虚拟主板110和硬件主板170的连接,数据传输等部分做了隔离隐藏,因此虚拟主板110上层的系统和应用感知不到自己对摄像头的配置,数据处理其实是两个不同平台分工合作的结果,在虚拟主板110的系统软件和应用来看,它们配置的是虚拟主板110上的摄像头,拿到的也是虚拟主板110上摄像头抓取的实时图像,因此可以支持完整的驱动应用开发,以及摄像头调优工作。3) The application of the hardware
4)保证了开发出的应用等可以不加修改的移植到其他真实硬件平台上4) It ensures that the developed applications can be transplanted to other real hardware platforms without modification
5)通过在真实硬件的分时复用可以支持多个虚拟主板110同时连接使用5) Multiple
如图7所示,本实施例还提供一种虚拟化图像处理方法,应用于如上所述的基于虚拟主板和硬件主板的图像获取系统100,所述方法包括:As shown in FIG. 7 , this embodiment also provides a virtualized image processing method, which is applied to the above-mentioned
步骤S100,图像采集模型监听虚拟主板对摄像头的配置信息,将所述虚拟主板对摄像头的配置信息和抓取图像的请求发送至硬件主板;Step S100, the image acquisition model monitors the configuration information of the virtual main board to the camera, and sends the configuration information of the virtual main board to the camera and the request for capturing images to the hardware main board;
步骤S200,所述硬件主板在接收到所述抓取图像的请求后,控制与所述硬件主板相连的摄像头采集图像,并将采集的图像传输至所述图像采集模型;Step S200, after receiving the request for capturing images, the hardware mainboard controls the camera connected to the hardware mainboard to collect images, and transmits the collected images to the image collection model;
步骤S300,所述图像采集模型从所述硬件主板接收硬件主板的摄像头采集的图像,并将接收到的图像发送至信号传输模型;Step S300, the image acquisition model receives the image collected by the camera of the hardware motherboard from the hardware motherboard, and sends the received image to the signal transmission model;
步骤S400,所述信号传输模型从所述图像采集模型接收所述图像,并将接收的所述图像转发至所述图像处理模型;Step S400, the signal transmission model receives the image from the image acquisition model, and forwards the received image to the image processing model;
步骤S500,所述图像处理模型分别与所述虚拟主板和所述信号传输模型通信相连,对从所述信号传输模型接收到的图片进行处理,并将处理后的图片发送至所述双倍速率同步动态随机存储器模型进行存储。Step S500, the image processing model communicates with the virtual main board and the signal transmission model respectively, processes the pictures received from the signal transmission model, and sends the processed pictures to the double rate Synchronous DRAM model for storage.
本申请中,基于虚拟主板和硬件主板的图像获取系统100可以实现本实施例所述的虚拟化图像处理方法,故而所述虚拟化图像处理方法的具体实现细节参见所述基于虚拟主板和硬件主板的图像获取系统100的细节描述,在此不再重述。但本发明所述的虚拟化图像处理方法的实现系统包括但不限于本实施例列举的基于虚拟主板和硬件主板的图像获取系统100,所属领域的技术人员可以清楚地了解到,为了描述的方便和简洁,仅以上述各功能单元、模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能单元、模块完成,即将所述系统/装置的内部结构划分成不同的功能单元或模块,以完成以上描述的全部或者部分功能。In this application, the
在本申请所提供的几个实施例中,应该理解到,所揭露的系统、装置或方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅是示意性的,例如,模块/单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个模块或单元可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或模块或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed system, device or method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules/units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or units can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules or units may be in electrical, mechanical or other forms.
作为分离部件说明的模块/单元可以是或者也可以不是物理上分开的,作为模块/单元显示的部件可以是或者也可以不是物理模块,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块/单元来实现本申请实施例的目的。例如,在本申请各个实施例中的各功能模块/单元可以集成在一个处理模块中,也可以是各个模块/单元单独物理存在,也可以两个或两个以上模块/单元集成在一个模块/单元中。A module/unit described as a separate component may or may not be physically separate, and a component shown as a module/unit may or may not be a physical module, that is, it may be located in one place, or it may be distributed to multiple network units superior. Part or all of the modules/units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, each functional module/unit in each embodiment of the present application may be integrated into one processing module, each module/unit may exist separately physically, or two or more modules/units may be integrated into one module/unit in the unit.
本领域普通技术人员应该还可以进一步意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。Those of ordinary skill in the art should further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the hardware and software interchangeability, the composition and steps of each example have been generally described in terms of functions in the above description. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.
本申请实施例提供的基于虚拟主板和硬件主板的图像获取系统,能够通过软件模型模拟摄像头,相机串行接口(MIPI CSI),图像信号处理(ISP)等各个硬件组件的功能,通过软件模型实现摄像头图像处理的功能,在虚拟主板上可以启动各种不同的操作系统,并支持完整的摄像头数据流应用开发。The image acquisition system based on the virtual main board and the hardware main board provided by the embodiment of the present application can simulate the functions of the camera, camera serial interface (MIPI CSI), image signal processing (ISP) and other hardware components through the software model, and realize it through the software model With the function of camera image processing, various operating systems can be started on the virtual motherboard, and it supports the development of complete camera data stream applications.
上述实施例仅例示性说明本申请的原理及其功效,而非用于限制本申请。任何熟悉此技术的人士皆可在不违背本申请的精神及范畴下,对上述实施例进行修饰或改变。因此,举凡所属技术领域中具有通常知识者在未脱离本申请所揭示的精神与技术思想下所完成的一切等效修饰或改变,仍应由本申请的权利要求所涵盖。The above-mentioned embodiments are only illustrative to illustrate the principles and effects of the present application, but are not intended to limit the present application. Any person familiar with the technology can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical ideas disclosed in the application shall still be covered by the claims of the application.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211669131.XA CN115988316A (en) | 2022-12-23 | 2022-12-23 | Image acquisition system and method based on virtual main board and hardware main board |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211669131.XA CN115988316A (en) | 2022-12-23 | 2022-12-23 | Image acquisition system and method based on virtual main board and hardware main board |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115988316A true CN115988316A (en) | 2023-04-18 |
Family
ID=85971748
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211669131.XA Pending CN115988316A (en) | 2022-12-23 | 2022-12-23 | Image acquisition system and method based on virtual main board and hardware main board |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115988316A (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6711283B1 (en) * | 2000-05-03 | 2004-03-23 | Aperio Technologies, Inc. | Fully automatic rapid microscope slide scanner |
US20180035017A1 (en) * | 2016-07-26 | 2018-02-01 | Samsung Electronics Co., Ltd. | Image pickup device and electronic system including the same |
CN114554091A (en) * | 2022-02-23 | 2022-05-27 | 中国计量大学 | Rapid image acquisition method for express delivery mobile terminal |
CN114827589A (en) * | 2022-04-29 | 2022-07-29 | 广州文远知行科技有限公司 | Shooting device drive test method and device and storage medium |
CN115174886A (en) * | 2022-08-25 | 2022-10-11 | 吉林大学 | Mobile phone lens supporting stereoscopic virtual reality and augmented reality and display method |
CN115185711A (en) * | 2022-09-06 | 2022-10-14 | 湖北芯擎科技有限公司 | Data interaction method and device based on virtual camera, electronic equipment and medium |
-
2022
- 2022-12-23 CN CN202211669131.XA patent/CN115988316A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6711283B1 (en) * | 2000-05-03 | 2004-03-23 | Aperio Technologies, Inc. | Fully automatic rapid microscope slide scanner |
US20180035017A1 (en) * | 2016-07-26 | 2018-02-01 | Samsung Electronics Co., Ltd. | Image pickup device and electronic system including the same |
CN114554091A (en) * | 2022-02-23 | 2022-05-27 | 中国计量大学 | Rapid image acquisition method for express delivery mobile terminal |
CN114827589A (en) * | 2022-04-29 | 2022-07-29 | 广州文远知行科技有限公司 | Shooting device drive test method and device and storage medium |
CN115174886A (en) * | 2022-08-25 | 2022-10-11 | 吉林大学 | Mobile phone lens supporting stereoscopic virtual reality and augmented reality and display method |
CN115185711A (en) * | 2022-09-06 | 2022-10-14 | 湖北芯擎科技有限公司 | Data interaction method and device based on virtual camera, electronic equipment and medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5700582B2 (en) | Multiprocessor complete mutual access method and system | |
CN105721780A (en) | Embedded image processing system and method based on SoC FPGA | |
TWI531958B (en) | Mass storage virtualization for cloud computing | |
JP3705663B2 (en) | Device bridging method and apparatus | |
JP2020129386A (en) | Low power computer imaging | |
WO2021008183A1 (en) | Data transmission method and apparatus, and server | |
WO2012138046A1 (en) | Terminal device and method for executing an android application on a general-purpose operating system, and computer-readable recording medium for same | |
WO2021008185A1 (en) | Data transmission method and apparatus, and server | |
CN116383175A (en) | Data loading method, device, equipment and computer readable medium | |
CN113760578A (en) | Method, device, equipment and computer program for cross-process rapid transmission of big data | |
CN110865868A (en) | Low-delay control method, device and equipment | |
US20130151735A1 (en) | I/o virtualization and switching system | |
CN110766600B (en) | Image processing system with distributed architecture | |
CN115988316A (en) | Image acquisition system and method based on virtual main board and hardware main board | |
EP4148568A1 (en) | Method for realizing live migration, chip, board, and storage medium | |
CN111654539B (en) | Cloud-protogenesis-based Internet of things operating system construction method and system and electronic equipment | |
JP2009064303A (en) | Imaging apparatus, shared resource management method, and program | |
CN117336425A (en) | Image processing circuit, image processing apparatus, image processing method, electronic device, and chip | |
CN108415655B (en) | Touch data processing method and device, all-in-one machine and storage medium | |
CN117555832A (en) | DMA interrupt configuration method, device, product and medium of virtual machine direct PCIe device | |
CN117632843A (en) | Data processing method and device, system on chip and electronic equipment | |
CN116033283A (en) | Virtualized image processing system, method and electronic device | |
CN115499575B (en) | Image data processing method, multimedia processing chip and electronic equipment | |
CN114564418A (en) | A data reading and writing method, system, terminal device and storage medium | |
CN113835816A (en) | Virtual machine desktop display method, device, equipment and readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |