CN109597030B - Method and device for positioning objects in storage room based on visible light signals - Google Patents

Abstract

The invention discloses a method and a device for positioning objects in a storage room based on visible light signals. The invention can complete the positioning only by shooting the target object and acquiring the light rays generated by the diffuse reflection of the target object without directly using a monitoring camera to shoot the light; according to the visible light signal-based method for positioning the objects in the warehouse, after the long exposure period and the short exposure period are set and the objects to be positioned are shot at the same time, two images are divided, so that accurate brightness gain is obtained, and therefore an accurate positioning position can be obtained; by establishing a model of brightness gain and position, the distance between the object to be positioned and each light-emitting device can be accurately obtained, and thus an accurate positioning position is obtained.

Description

A method and device for locating objects in a storage room based on visible light signals

Technical Field

The present invention relates to a method and a device for positioning an object in a storage room, and in particular to a method and a device for positioning an object in a storage room based on visible light signals.

Background Art

With the rapid development of my country's economy in recent years, the market demand for the logistics industry has continued to expand. As an important part of the logistics system, warehousing plays a vital role in the development of modern logistics. The early warehouse cargo positioning mechanism was completed manually, marking the cargo body and finding the target cargo through manual comparison. However, due to the expansion of the market demand for the logistics industry and the sharp increase in the number of goods, the early warehouse cargo positioning mechanism not only requires a lot of manpower and time expenditure, but also easily causes human errors. At present, QR codes are usually used to mark goods, and manual handheld cameras are required to scan and identify the QR codes attached to the goods to complete the positioning of the goods. However, this method not only increases the additional cost of the QR code, but also reduces the accuracy of the positioning system due to manual participation.

The existing technology also uses radio frequency identification (RFID) technology to identify goods. The advantage of this method is that it removes the restrictions on the label attachment method and the identification method, and can more easily obtain the information of each piece of goods, thereby locating the goods. However, since RFID equipment is relatively expensive, these devices will increase the overall cost of the system, making the cost of goods positioning technology based on radio frequency identification technology higher and more difficult to implement.

Summary of the invention

The object of the present invention is to provide a method and device for locating objects in a storage room based on visible light signals, so as to solve the problems of low positioning accuracy and high cost in the existing method for locating objects in a storage room.

In order to achieve the above tasks, the present invention adopts the following technical solutions:

A method for locating objects in a storage room based on visible light signals, wherein the method uses visible light signals emitted by multiple light-emitting devices to locate objects to be located in the storage room, wherein N light-emitting devices are arranged in the storage room, N ≥ 5, and the light emitted by the N light-emitting devices is irradiated on the objects to be located;

The method described is performed according to the following steps:

Step 1: Control the on/off state of each light emitting device so that at least one light emitting device is in the on state at the same time;

When controlling the on/off state of each light emitting device, each light emitting device switches the on/off state according to its own on/off rule;

Repeat step 1 until step 2 is completed, and then execute step 2;

Step 2: Acquire a first image of the object to be positioned illuminated by all light-emitting devices in the form of exposure of image pixel columns one by one:

Each time a column of pixels in the first image is exposed, a brightness accumulation value of all light-emitting devices on the object to be located is obtained;

Step 3, obtaining a sequence of brightness accumulation values of all pixel columns in the first image, and obtaining an image brightness gain sequence;

Converting the image brightness gain sequence into an image form, wherein the image brightness gain sequence in the image form has a horizontal coordinate of pixels and a vertical coordinate of brightness gain;

Step 4, obtaining the brightness gain of each light-emitting device from the image brightness gain sequence in the form of an image;

Step 5: According to the brightness gain of each light-emitting device, the distance between the object to be located and each light-emitting device is obtained using Formula I, in units of m:

以及

为第1个发光装置、第2个发光装置、第n个发光装置以及第N个发光装置在待定位物体上的入射角，单位为°；Wherein, d ₁ , d ₂ , d _n and d _N are the distances between the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device and the object to be located, and the unit is m; RSS ₁ , RSS ₂ , RSS _n and RSS _N are the brightness gains of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device obtained in step 5; G is a positioning parameter, G>0; φ ₁ , φ ₂ , φ _n and φ _N are the emission angles of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device, and the unit is °;

as well as

is the incident angle of the first light-emitting device, the second light-emitting device, the nth light-emitting device, and the Nth light-emitting device on the object to be positioned, in degrees;

Step 6: Obtain the position of the object to be located according to the position of each light-emitting device in the storage room and the distance between each light-emitting device and the object to be located;

Positioning is completed.

Furthermore, in step 1, each light emitting device switches on and off according to its own on and off rule, specifically including:

Step 11, obtaining a 4M-bit binary digital frame code of each light-emitting device, where M is a positive integer;

When obtaining the binary digital frame code of the nth light emitting device, the first 4m bits are the leading code, m is a positive integer, m<M, the 4m+nth bit of the binary number is opposite to the binary numbers of all the remaining bits except the leading code, n=1, 2, ..., N;

Step 12: All light emitting devices start to light up and down simultaneously according to their respective 4M-bit binary digital frame codes, where 0 represents that the light emitting device is off, and 1 represents that the light emitting device is on.

Furthermore, in the step 12, all the light-emitting devices start to light up and off at the same time according to their respective 4M-bit binary frame codes. A string of binary frame codes controls the light-emitting devices to light up and off within a frame duration. The frame duration includes multiple pulse signal cycles in microseconds. Each binary digit in the frame code of the light-emitting device controls the light-up and off state of the light-emitting device within a pulse signal cycle.

Furthermore, in the step 2, while acquiring the first image of the object to be positioned illuminated by all the light-emitting devices in the form of image pixel column exposure, acquiring the second image of the object to be positioned illuminated by all the light-emitting devices in the form of image pixel column exposure; wherein the exposure time of the first image is one pulse signal cycle, the unit is microsecond, and the exposure time of the second image is S pulse signal cycles, the unit is microsecond, S>1, wherein S is the ratio between the maximum sensitivity and the minimum sensitivity of the hardware for acquiring the first image and the second image, the first image is acquired using the maximum sensitivity of the acquisition hardware, and the second image is acquired using the minimum sensitivity of the acquisition hardware.

Furthermore, a QR code is attached to the object to be located. Before acquiring the first image and the second image of the object to be located, the object to be located is found by identifying the QR code information of all objects in the warehouse.

Furthermore, the acquisition of the first image of the object to be positioned illuminated by all the light emitting devices in the form of exposure of each image pixel column is performed according to the following steps:

Step A, acquiring a first original image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure of image pixel columns one by one and within an exposure time of one pulse signal cycle;

Step B, intercepting an image including only the object to be located in the first original image by setting ROI, to obtain a first image;

The acquisition of the second image of the object to be positioned illuminated by all the light emitting devices in the form of exposure of each image pixel column is performed according to the following steps:

Step a, acquiring a second original image of the object to be located that is illuminated by all the light-emitting devices in the form of exposure of image pixel columns one by one and within S pulse signal cycles as the exposure time;

Step b: intercepting an image including only the object to be located in the second original image by setting ROI to obtain a second image.

Furthermore, when obtaining the sequence of brightness accumulation values of all pixel columns in the first image in step 3, the sequence of brightness accumulation values of all pixel columns in the first image is obtained by calculating the sum of the grayscale values of each column in the first image; the sequence of brightness accumulation values of all pixel columns in the second image is obtained by calculating the sum of the grayscale values of each column in the second image;

The first image brightness accumulated value sequence is divided by the second image brightness accumulated value sequence to obtain an image brightness gain sequence.

Furthermore, in step 4, the brightness gain of each light-emitting device is obtained from the image brightness gain sequence in the form of an image, and is performed according to the following steps:

Step 41, finding the brightness gain waveform of the preamble code in the image brightness gain sequence in the image form, and obtaining the horizontal coordinate length D occupied by the brightness gain waveform of the preamble code, where the unit is pixel, and D is a positive integer;

Step 42: Use Formula II to obtain the horizontal coordinate length d occupied by the one-bit preamble brightness gain waveform, in pixels, where d is a positive integer:

Wherein, m is a positive integer;

The horizontal axis length occupied by the one-bit preamble code brightness gain waveform is the horizontal axis length occupied by the one-bit binary frame code brightness gain waveform;

Step 43, in the image brightness gain sequence in the image form, taking the last horizontal coordinate point of the leading code brightness gain waveform as the starting point, taking the horizontal coordinate length occupied by the one-bit binary frame coding brightness gain waveform as the step length, successively find the horizontal coordinate length range of the brightness gain waveforms of the 1st to Nth light-emitting devices in pixels, and the absolute value of the difference between the brightness gain mean value corresponding to the brightness gain within the horizontal coordinate length range of the brightness gain waveform of each light-emitting device and the brightness gain corresponding to the highest point in the image brightness gain sequence in the image form is the brightness gain of each light-emitting device.

A storage indoor object positioning device based on visible light signals, used to implement the storage indoor object positioning method based on visible light signals, the object to be positioned is pasted with QR code information, and the storage indoor object positioning device includes multiple wireless sensor modules, multiple light-emitting devices, an image acquisition module and a positioning data processing module;

The wireless sensor module is connected to the visible light signal transmission module to obtain a 4M-bit binary frame code for each light-emitting device, wherein the frame code of the n-th light-emitting device, the first 4m bits are the leading code, m<M, m is a positive integer, M is a positive integer, the 4m+n-th binary number is opposite to the binary numbers of all other bits except the leading code, so that at least one light-emitting device is in a lighting state at the same time, n=1, 2, ..., N;

Where 0 represents that the light-emitting device is off, and 1 represents that the light-emitting device is on;

The light emitting device is connected to the image acquisition module and is used to start lighting up and down at the same time according to the respective 4M-bit binary digital frame codes, and at least one light emitting device is in the lighting state at the same time:

The image acquisition module is connected to the positioning data processing module and is used to find the object to be located by identifying the two-dimensional code information of all objects in the warehouse;

It is also used to obtain a first image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure column by column, and to obtain a second image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure column by column; wherein the exposure time of the first image is one pulse signal cycle, in microseconds, and the exposure time of the second image is S pulse signal cycles, in microseconds, S>1, wherein S is the ratio between the maximum sensitivity and the minimum sensitivity of the hardware for acquiring the first image and the second image, the first image is acquired using the maximum sensitivity of the image acquisition module, and the second image is acquired using the minimum sensitivity of the image acquisition module;

The positioning data processing module is used to obtain the sum of the grayscale values of each column in the first image to obtain the brightness accumulation value sequence of all pixel columns in the first image; by obtaining the sum of the grayscale values of each column in the second image, the brightness accumulation value sequence of all pixel columns in the second image is obtained;

Dividing the first image brightness accumulated value sequence by the second image brightness accumulated value sequence to obtain an image brightness gain sequence;

Converting the image brightness gain sequence into an image form, wherein the image brightness gain sequence in the image form has a horizontal coordinate of pixels and a vertical coordinate of brightness gain;

The positioning data processing module is also used to obtain the brightness gain of each light-emitting device from the image brightness gain sequence in the form of an image;

The positioning data processing module is also used to obtain the distance of the object to be positioned from each light-emitting device according to the brightness gain of each light-emitting device using Formula I, in units of m:

以及

为第1个发光装置、第2个发光装置、第n个发光装置以及第N个发光装置在待定位物体上的入射角，单位为°；Wherein, d ₁ , d ₂ , d _n and d _N are the distances between the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device and the object to be located, and the unit is m; RSS ₁ , RSS ₂ , RSS _n and RSS _N are the brightness gains of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device obtained in step 5; G is a positioning parameter, G>0; φ ₁ , φ ₂ , φ _n and φ _N are the emission angles of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device, and the unit is °;

as well as

is the incident angle of the first light-emitting device, the second light-emitting device, the nth light-emitting device, and the Nth light-emitting device on the object to be positioned, in degrees;

The positioning data processing module is also used to obtain the position of the object to be positioned according to the position of each light-emitting device in the storage room and the distance between each light-emitting device and the object to be positioned.

Furthermore, the wireless sensor module is an ISA100.11a sensor.

Compared with the prior art, the present invention has the following technical effects:

1. The method for locating objects in a storage room based on visible light signals provided by the present invention obtains accurate brightness gain by setting a long exposure period and a short exposure period to simultaneously shoot the object to be located, and then divides the two images, thereby obtaining an accurate positioning position;

2. The method for locating objects in a storage room based on visible light signals provided by the present invention can accurately obtain the distance between the object to be located and each light-emitting device by establishing a model of brightness gain and position, thereby obtaining an accurate positioning position;

3. The storage room object positioning device based on visible light signals provided by the present invention can obtain the position of the object by collecting the image of the object to be positioned in the storage room. There is no need to deploy other positioning equipment. Accurate positioning can be achieved by using existing storage room cameras and LED lights.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a flow chart of a method for locating objects in a storage room based on visible light signals provided by the present invention;

FIG2 is a schematic diagram of frame coding of six LED lights provided in one embodiment of the present invention;

FIG3 is a schematic diagram of a rolling shutter principle provided in one embodiment of the present invention;

FIG4 is a brightness gain sequence in image form of a first image provided in one embodiment of the present invention;

FIG5 is a brightness gain sequence in image form of a second image provided in one embodiment of the present invention;

FIG6 is an image brightness gain sequence in the form of an image provided in one embodiment of the present invention;

FIG. 7 is an image brightness gain sequence in the form of an image after linear regression provided in one embodiment of the present invention.

DETAILED DESCRIPTION

The present invention sends visible light signals by turning on and off LED lights. The visible light signals are irradiated on the object to be located and diffusely reflected. The diffuse reflection effect captured by the camera is then used to analyze the position of the object to be located. The present invention does not need to directly use a camera to capture the light, but only needs to capture the object to be located and obtain the diffusely reflected light generated by the object to be located to complete the positioning.

Visible light signal: Due to the rolling shutter effect, when the camera captures flashing LED lights with a sufficiently short exposure time, alternating light and dark stripes will appear in the captured photos. The signal emitted by the LED light can be inferred through the brightness value, frequency and other characteristics of these stripes, thus realizing communication based on visible light signals.

Embodiment 1

This embodiment discloses a method for locating an object in a storage room based on visible light signals. The method uses visible light signals emitted by multiple light-emitting devices to locate an object to be located in the storage room. N light-emitting devices are arranged in the storage room, where N ≥ 5. The light emitted by the N light-emitting devices is irradiated on the object to be located.

The light-emitting device is a device that can emit visible light signals, and may be a fluorescent lamp or a diode, etc. In this embodiment, the light-emitting device is an LED lamp.

The method described is performed according to the following steps:

Step 1: Control the on/off state of each light emitting device so that at least one light emitting device is in the on state at the same time;

When controlling the on/off state of each light emitting device, each light emitting device switches the on/off state according to its own on/off rule;

In this step, each light emitting device is turned on and off at intervals to ensure that the object to be positioned can receive light emitted by at least one light emitting device at the same time.

The on and off rules can be obtained by encoding each device.

Specifically, the step 1 comprises:

Step 11, obtaining a 4M-bit binary digital frame code of each light-emitting device, where M is a positive integer;

When obtaining the binary digital frame code of the nth light emitting device, the first 4m bits are the leading code, m is a positive integer, m<M, the 4m+nth bit of the binary number is opposite to the binary numbers of all the remaining bits except the leading code, n=1, 2, ..., N;

Preferably, in the preamble code, the last four digits of the preamble code represent the number of the area where the LED lamp is located.

In this embodiment, in order to conveniently find out which LED lights locate the object to be located, the area information of the LED lights is set in the last 4 bits of the leading code. For example, when the leading code is 10100001, it means that the LED light is an LED light in the 0001 area.

In this embodiment, a series of binary frame codes is firstly set for each LED lamp, so that the LED lamp flashes according to the rules of the frame codes to send visible light signals.

As a preferred method, the frame is encoded as a multi-bit binary number.

The frame code is a 4M-bit binary number, M≥2;

The frame code of the nth light-emitting device is obtained, the first 4m bits are the leading code, m<M, the 4m+nth bit binary number is opposite to the binary numbers of all the remaining bits except the leading code, n=1,2,…,N, so that at least one LED lamp is in the lighting state at the same time.

In this embodiment, N=6, 6 LED lights are set in the warehouse, M=6, the frame code is a 4*6-bit binary number, m=2, the first 8 bits are the leading code, and the leading code of the first 8 bits is 10101001, wherein the last four bits of the leading code 1001 indicate that the LED light is in the 1001 area.

The frame code of the first LED light is: 101010010111111111111111;

The frame code of the second LED light is: 101010011011111111111111;

The frame code of the third LED light is: 101010011101111111111111;

The frame code of the fourth LED light is: 101010011110111111111111;

The frame code of the fifth LED light is: 101010011111011111111111;

The frame code of the sixth LED light is: 101010011111101111111111.

As shown in FIG2 , from the above coding, it can be seen that the first eight bits of the frame coding of all lights are exactly the same, which is the leading code. From the ninth bit to the fourteenth bit, each bit corresponds to a different state of an LED light from other LED lights, that is, the ninth bit of the first LED light frame coding is 0, all the remaining bits of the first LED light frame coding are 1, and the ninth bits of all the other LED light frame codings are 1. The tenth bit of the second LED light frame coding is 0, all the remaining bits of the second LED light frame coding are 1, and the tenth bit of the frame coding of all the other LED lights is 1; that is, as shown in FIG2 , at the same time, there is only one LED light whose state is different from the states of all the other LED lights, and the light with different states can be in a lit state or an off state, so as to achieve a high duty cycle to ensure that the visible light signal can be accurately transmitted.

Step 12: All light emitting devices start to light up and down simultaneously according to their respective 4M-bit binary digital frame codes, where 0 represents that the light emitting device is off, and 1 represents that the light emitting device is on.

For each light-emitting device, the light-emitting device is controlled to turn on and off according to each binary number of a string of binary frame codes, where 0 represents the light-emitting device is off and 1 represents the light-emitting device is on;

When the light-emitting device is on and off, the brightness of the light-emitting device on the object to be located is different;

In this embodiment, the LED light is on for a period of time and off for a period of time, which may cause the LED light to flicker. At this time, according to the binary LED light frame code, the LED light can be turned on for a long period of time and off for a long period of time.

Optionally, when all the light-emitting devices in step 12 are turned on and off simultaneously according to their respective strings of binary frame codes, a string of binary frame codes controls the light-emitting devices to turn on and off within a frame duration, and the frame duration includes multiple pulse signal cycles in microseconds. Each binary digit in the frame code of the light-emitting device controls the on and off state of the light-emitting device within a pulse signal cycle.

In this embodiment, since the frame code of the LED light is a binary number, 0 and 1 in the binary number can represent the on and off or off and on of the LED light respectively.

In this embodiment, as shown in FIG2 , the flashing waveform of the first LED light in the figure, the first eight bits of the first LED light are the preamble code, at this time starting from the first bit of the frame code, when the first preamble code is 1, the first LED light is on for a pulse signal cycle, then the second preamble code is 0, the first LED light is off for a pulse signal cycle, the third preamble code is 1 again, the first LED light is on for a pulse signal cycle, then the fourth preamble code is 0 again, the first LED light is off for a pulse signal cycle; starting from the ninth bit of the frame code, the ninth bit of the frame code is 0, then the first LED light is off for a pulse signal cycle, and then the frame code from the tenth bit to the twenty-fourth bit is always 1, so the first LED light is always on for fifteen pulse signal cycles.

In this embodiment, a pulse signal period can be 10 microseconds, 1 microsecond, etc., which is not limited in this embodiment. As a preferred implementation, a pulse signal period is 300 microseconds.

Likewise, the second LED light, the third LED light, the fourth LED light, the fifth LED light and the sixth LED light flash similarly to the first LED light.

Repeat step 1 until step 2 is completed, and then execute step 2;

Since the positions of objects in the warehouse are not fixed, in order to facilitate the camera to capture the object to be located, a QR code is affixed to the object to be located. Before obtaining the first image and the second image of the object to be located, the object to be located is found by identifying the QR code information of all objects in the warehouse.

When the goods in the warehouse change, the surveillance camera scans the QR code within its shooting range. For existing QR codes, the system relocates the object to be located and updates the information in the server. For newly scanned QR codes, the system obtains the goods information and locates it, then associates the goods information with the location information and stores it. For unscanned QR codes, the system considers that the goods have been taken away and deletes the information from the server.

Step 2: Acquire a first image of the object to be positioned illuminated by all light-emitting devices in the form of exposure of image pixel columns one by one:

Each time a column of pixels in the first image is exposed, a cumulative value of the brightness of all light-emitting devices on the object to be located is obtained;

In this embodiment, a CMOS camera is used to capture a first image of the object to be located, wherein the CMOS camera captures images using a rolling shutter method, and thus the first image of the object to be located is obtained in the form of column-by-column exposure. Since the first image is exposed column by column, light irradiated on the object to be located by all light-emitting devices can be captured within the time used to expose one column. In other words, the accumulated brightness value of the light-emitting devices on the object to be located can be obtained by using the image within the time used to expose one column. Therefore, alternating light and dark stripes will appear in the first image to correspond to the on and off light emitted by all light-emitting devices.

Since the camera captures the complex background while capturing the visible light signal, the signal-to-noise ratio of the final extracted signal will be relatively low, so signal extraction is performed.

Since there is noise in the first image, the accuracy of positioning is reduced. In this embodiment, the second image is obtained while the first image is obtained, and the second image is used as the background layer of the first image.

Optionally, in the step 2, while acquiring the first image of the object to be positioned illuminated by all the light-emitting devices in the form of image pixel column exposure, acquiring the second image of the object to be positioned illuminated by all the light-emitting devices in the form of image pixel column exposure; wherein the exposure time of the first image is one pulse signal cycle, the unit is microsecond, and the exposure time of the second image is S pulse signal cycles, the unit is microsecond, wherein S is the ratio between the maximum sensitivity and the minimum sensitivity of the hardware for acquiring the first image and the second image, S>1, the first image is acquired using the maximum sensitivity of the acquisition hardware, and the second image is acquired using the minimum sensitivity of the acquisition hardware.

First, when acquiring images, the object to be located should be highlighted first, and other objects that do not need to be located should be avoided as much as possible.

In this embodiment, the exposure time for obtaining the second image is different from the exposure time for obtaining the first image. The exposure time for obtaining the second image is S pulse signal cycles, where S is the ratio between the maximum sensitivity and the minimum sensitivity of the hardware for obtaining the first image and the second image, and S>1. In this way, the two photos will have the same background layer, and since the exposure time of the second image is longer, the second image is directly regarded as the background layer of the first image.

Optionally, the acquiring of the first image of the object to be positioned illuminated by all the light emitting devices in the form of column-by-column exposure is performed according to the following steps:

Step A, acquiring a first original image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure of image pixel columns one by one and within an exposure time of one pulse signal cycle;

Step B, intercepting an image including only the object to be located in the first original image by setting ROI, to obtain a first image;

The acquisition of the second image of the object to be positioned illuminated by all the light emitting devices in the form of column-by-column exposure is performed according to the following steps:

Step a, acquiring a second original image of the object to be located that is illuminated by all the light-emitting devices in the form of exposure of image pixel columns one by one and within S pulse signal cycles as the exposure time;

Step b: intercepting an image including only the object to be located in the second original image by setting ROI to obtain a second image.

In this embodiment, an identifiable QR code can be attached to the surface of the object to be located. The camera identifies the object to be located through the QR code on the surface of the object to be located. After the object to be located is identified and found, a photo of the object to be located is taken. At this time, some other interfering objects will inevitably appear in the photo. Therefore, a simple rectangular ROI area frame is used to extract the image containing the object to be located to obtain a first image.

The process of obtaining the second image is the same as the process of obtaining the first image, and both are obtained by setting ROI.

Step 3, obtaining a sequence of brightness accumulation values of all pixel columns in the first image, and obtaining an image brightness gain sequence;

Converting the image brightness gain sequence into an image form, wherein the image brightness gain sequence in the image form has a horizontal coordinate of pixels and a vertical coordinate of brightness gain;

Preferably, when obtaining the sequence of brightness accumulation values of all pixel columns in the first image in step 3, the sequence of brightness accumulation values of all pixel columns in the first image is obtained by calculating the sum of the grayscale values of each column in the first image; the sequence of brightness accumulation values of all pixel columns in the second image is obtained by calculating the sum of the grayscale values of each column in the second image;

The first image brightness accumulated value sequence is divided by the second image brightness accumulated value sequence to obtain an image brightness gain sequence.

Due to the effect of rolling shutter, as shown in Figure 3, the CMOS sensor collects a column of pixels every other collection period t _r , t _r represents the readout time after the CMOS sensor collects a column of data. Since the cache of the CMOS sensor only allows the readout of one column of pixel data at the same time, the collection time of each column of pixels is staggered with the collection period t _r . When the exposure time is t _e , the photon receiving time of each photo is (Y-1)t _r +t _e , Y represents the total number of columns exposed for a picture, and the sampling rate of the camera can reach 1/t _r , which is significantly higher than the data transmission rate of the LED light. Therefore, the information of each pulse cycle can be collected. At the same time, since the camera resolution can generally reach 1024*768, a photo can completely capture a data frame containing 24 pulse signals.

After obtaining the first image and the second image with different exposure times, since the long-exposure photo can be directly regarded as the background layer of the short-exposure photo, the grayscale value added by column of the first photo is directly divided by the grayscale value added by column of the second photo to obtain the pixel gain of each column of the image, that is, the image brightness gain sequence.

For example, assuming the grayscale value of the first image is:

The gray value sequence of the first image is:

[1.2 1.5 2.8 2.5]

The grayscale value of the second image is:

The gray value sequence of the second image is:

[1.2 1.6 2.0 1.9]

The image brightness gain sequence is:

[1 0.93 1.4 1.3]

Then the brightness gain of the first column of the image is 1, the brightness gain of the second column of the image is 0.93, the brightness gain of the third column of the image is 1.4, and the brightness gain of the fourth column of the image is 1.3.

In this embodiment, the brightness gain of the first image is shown in FIG4, the brightness gain of the second image is shown in FIG5, and the image brightness gain sequence in image form is shown in FIG6, wherein the abscissas of the first image and the second image are both pixels, and the ordinates are grayscale values. The abscissas of the image brightness gain sequence in image form are pixels, and the ordinates are brightness gains.

Step 4: Obtain a pixel column of each light-emitting device, and obtain a brightness gain of each light-emitting device from an image brightness gain sequence in an image form;

In this embodiment, the step 4 is to obtain the brightness gain of each LED lamp from the image brightness gain sequence in the form of an image, and is performed according to the following steps:

Step 41, finding the brightness gain waveform of the preamble code in the image brightness gain sequence in the image form, and obtaining the horizontal coordinate length D occupied by the brightness gain waveform of the preamble code, where the unit is pixel, and D is a positive integer;

In this embodiment, the image brightness gain sequence in the form of an image is obtained as shown in FIG4. Since each signal has a preamble code, in this embodiment, the 4-bit preamble code is 10101001, so the preamble codes of multiple lights are the same. After superposition, it will be displayed as a corresponding waveform in FIG6. Multiple 1s are superimposed as the highest point in FIG6, and multiple 0s are superimposed as the lowest point in the figure. The highest point is the brightness when multiple lights are all on, and the lowest point is the brightness when these lights are all off. In FIG6, the horizontal axis is pixels, and the vertical axis is brightness gain. The horizontal axis of an 8-bit preamble code is [294, 709], that is, the horizontal axis length D occupied by the 8-bit preamble code brightness gain waveform is 415 pixels;

Step 42: Use Formula II to obtain the horizontal coordinate length d occupied by the one-bit preamble brightness gain waveform, in pixels, where d is a positive integer:

Wherein, m is a positive integer;

The horizontal axis length occupied by the one-bit preamble code brightness gain waveform is the horizontal axis length occupied by the one-bit binary frame code brightness gain waveform;

In this embodiment, the horizontal axis length d occupied by the one-bit preamble code brightness gain waveform is 51.875, that is, the horizontal axis length occupied by the one-bit binary number frame coding brightness gain waveform is 51.875 pixels.

In addition, the present method also provides a method for obtaining the column width occupied by a frame-coded brightness gain waveform of a binary number, which can obtain a more accurate result, specifically:

The preamble is detected by smoothing the second-order derivative, as shown in FIG6 . At this time, the horizontal coordinates of the two preambles are [294, 709] and [1675, 2090], respectively. By calculation, the length of a single frame is 1141.7. A frame contains 24 steps, and a step is 47.569 pixels.

A linear regression is performed on the parts of the leading codes at both ends with a step size of 47.569 pixels, and the result shown in FIG7 is obtained. In FIG7 , the horizontal axis is pixels and the vertical axis is brightness gain.

Step 43, in the image brightness gain sequence in the image form, taking the last horizontal coordinate point of the leading code brightness gain waveform as the starting point, taking the horizontal coordinate length occupied by the frame coding brightness gain waveform of a binary number as the step length, successively find the horizontal coordinate length range of the brightness gain waveforms of the first to Nth light-emitting devices, and the absolute value of the difference between the brightness gain mean value corresponding to the horizontal coordinate length range of the brightness gain waveform of each light-emitting device and the brightness gain corresponding to the highest point in the image brightness gain sequence in the image form is the brightness gain of each light-emitting device.

In this embodiment, in FIG5 , the six segments of the horizontal coordinate length ranges corresponding to the six LED lights are: [757803] [804 851] [852 898] [899 946] [947 993] [994 1041];

The gains of the six LED lights are:

[0.11190443,0.07794306,0.01776551,0.0409358,0.16835417,0.1267388].

Step 5: According to the brightness gain of each light-emitting device, the distance between the object to be located and each light-emitting device is obtained using Formula I, in meters:

以及

为第1个发光装置、第2个发光装置、第n个发光装置以及第N个发光装置在待定位物体上的入射角，单位为°；Wherein, d ₁ , d ₂ , d _n and d _N are the distances between the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device and the object to be located, and the unit is m; RSS ₁ , RSS ₂ , RSS _n and RSS _N are the brightness gains of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device obtained in step 5; G is a positioning parameter, G>0; φ ₁ , φ ₂ , φ _n and φ _N are the emission angles of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device, and the unit is °;

as well as

is the incident angle of the first light-emitting device, the second light-emitting device, the nth light-emitting device, and the Nth light-emitting device on the object to be positioned, in degrees;

The value of the positioning parameter G is related to the distance between the camera and the object, the amount of light entering the camera's aperture, and the degree of reflection of the wall.

In this embodiment, the coordinates of the LED lamp are (x _i , y _i , 0) (all LED lamps are at the same height, known), and the coordinates of the located object are (x, y, z), then:

Among them, θ represents the angle between the photographed surface of the located object and the x-axis;

but:

At this time, there are five unknowns in the formula. As long as five or more equations are listed, that is, at least 5 LED lights are set in the positioning system, the coordinates and orientation angle θ of the located object can be obtained through optimization problems.

Step 6: Obtain the position of the object to be located according to the position of each light-emitting device in the storage room and the distance between each light-emitting device and the object to be located;

Positioning is completed.

After the distance between each light emitting device and the object to be located is determined, the position of the object to be located can be determined according to the position of each light emitting device.

Embodiment 2

A storage indoor object positioning device based on visible light signals, used to implement a storage indoor object positioning method based on visible light signals, wherein the object to be positioned is pasted with QR code information, and the storage indoor object positioning device comprises a wireless sensor module, a plurality of light-emitting devices, an image acquisition module and a positioning data processing module;

In this embodiment, the light emitting device may be a diode light emitting device, a fluorescent lamp, etc. Preferably, the light emitting device is an LED lamp.

The image acquisition module can be a camera on a mobile phone, or an image acquisition device such as a camera. Preferably, the image acquisition module is a surveillance camera installed in the storage room, and the camera is a CMOS camera to save the cost of the positioning device.

The positioning data processing module can be a cloud server or a local computer or other device capable of processing images.

The wireless sensor module is connected to the visible light signal transmission module to obtain a 4M-bit binary frame code for each light-emitting device, wherein the frame code of the n-th light-emitting device, the first 4m bits are the leading code, m<M, m is a positive integer, M is a positive integer, the 4m+n-th binary number is opposite to the binary numbers of all the remaining bits except the leading code, so that at least one light-emitting device is in a lighting state at the same time, n=1, 2, ..., N;

Where 0 represents that the light-emitting device is off, and 1 represents that the light-emitting device is on;

The wireless sensor module is also used to send a string of binary frame codes of each light-emitting device to each light-emitting device;

Preferably, the wireless sensor module is an ISA100.11a sensor node, and the multiple wireless sensor modules constitute an ISA100.11a wireless sensor network.

ISA100.11a sensors are used to control the flashing of each light-emitting device. Multiple ISA100.11a sensors form an ISA100.11a industrial wireless network, which improves the signal transmission efficiency. And because ISA100.11a sensors can be remotely controlled, frame encoding of each light-emitting device can be remotely controlled, thereby achieving flexible positioning.

The ISA100.11a wireless sensor network is used to remotely control the on and off state of the light emitting device.

The light emitting device is connected to the image acquisition module and is used to start lighting up and down at the same time according to the respective 4M-bit binary digital frame codes, and at least one light emitting device is in the lighting state at the same time:

The image acquisition module is connected to the positioning data processing module and is used to find the object to be located by identifying the two-dimensional code information of all objects in the warehouse;

It is also used to obtain a first image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure column by column, and to obtain a second image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure column by column; wherein the exposure time of the first image is one pulse signal cycle, in microseconds, and the exposure time of the second image is S pulse signal cycles, in microseconds, S>1, wherein S is the ratio between the maximum sensitivity and the minimum sensitivity of the hardware for acquiring the first image and the second image, the first image is acquired using the maximum sensitivity of the image acquisition module, and the second image is acquired using the minimum sensitivity of the image acquisition module;

The positioning data processing module is used to obtain the sum of the grayscale values of each column in the first image to obtain the brightness accumulation value sequence of all pixel columns in the first image; by obtaining the sum of the grayscale values of each column in the second image, the brightness accumulation value sequence of all pixel columns in the second image is obtained;

Dividing the first image brightness accumulated value sequence by the second image brightness accumulated value sequence to obtain an image brightness gain sequence;

The positioning data processing module is also used to obtain the brightness gain of each light-emitting device from the image brightness gain sequence in the form of an image;

The positioning data processing module is also used to obtain the distance of the object to be positioned from each light-emitting device according to the brightness gain of each light-emitting device using Formula I, in units of m:

以及

为第1个发光装置、第2个发光装置、第n个发光装置以及第N个发光装置在待定位物体上的入射角，单位为°；Wherein, d ₁ , d ₂ , d _n and d _N are the distances between the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device and the object to be located, and the unit is m; RSS ₁ , RSS ₂ , RSS _n and RSS _N are the brightness gains of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device obtained in step 5; G is a positioning parameter, G>0; φ ₁ , φ ₂ , φ _n and φ _N are the emission angles of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device, and the unit is °;

as well as

is the incident angle of the first light-emitting device, the second light-emitting device, the nth light-emitting device, and the Nth light-emitting device on the object to be positioned, in degrees;

The value of the positioning parameter G is related to the distance between the camera and the object, the amount of light entering the camera's aperture, and the degree of reflection of the wall.

The positioning data processing module is also used to obtain the position of the object to be positioned according to the position of each light-emitting device in the storage room and the distance between each light-emitting device and the object to be positioned.

Claims (10)

1. A method for locating objects in a storage room based on visible light signals, wherein the method uses visible light signals emitted by multiple light-emitting devices to locate objects to be located in the storage room, wherein N light-emitting devices are arranged in the storage room, N ≥ 5, and the light emitted by the N light-emitting devices is irradiated on the objects to be located; The method described is performed according to the following steps: Step 1: Control the on/off state of each light emitting device so that at least one light emitting device is in the on state at the same time; When controlling the on/off state of each light emitting device, each light emitting device switches the on/off state according to its own on/off rule; It is characterized in that Repeat step 1 until step 2 is completed, and then execute step 2; Step 2: Acquire a first image of the object to be positioned illuminated by all light-emitting devices in the form of exposure of image pixel columns one by one: Each time a column of pixels in the first image is exposed, a brightness accumulation value of all light-emitting devices on the object to be located is obtained; Step 3, obtaining a sequence of brightness accumulation values of all pixel columns in the first image, and obtaining an image brightness gain sequence; Converting the image brightness gain sequence into an image form, wherein the image brightness gain sequence in the image form has a horizontal coordinate of pixels and a vertical coordinate of brightness gain; Step 4, obtaining the brightness gain of each light-emitting device from the image brightness gain sequence in the form of an image; Step 5: According to the brightness gain of each light-emitting device, the distance between the object to be located and each light-emitting device is obtained using Formula I, in units of m:

Wherein, d ₁ , d ₂ , d _n and d _N are the distances between the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device and the object to be located, and the unit is m. RSS ₁ , RSS ₂ , RSS _n and RSS _N are the brightness gains of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device obtained in step 5. G is a positioning parameter, G>0.

as well as

is the emission angle of the first light-emitting device, the second light-emitting device, the nth light-emitting device, and the Nth light-emitting device, in degrees,

as well as

is the incident angle of the first light-emitting device, the second light-emitting device, the nth light-emitting device, and the Nth light-emitting device on the object to be positioned, in degrees; Step 6: Obtain the position of the object to be located according to the position of each light-emitting device in the storage room and the distance between each light-emitting device and the object to be located; Positioning is completed. 2. The method for locating objects in a storage room based on visible light signals according to claim 1, characterized in that in step 1, each light emitting device switches its on/off state according to its own on/off rule, specifically comprising: Step 11, obtaining a 4M-bit binary digital frame code of each light-emitting device, where M is a positive integer; When obtaining the binary digital frame code of the nth light emitting device, the first 4m bits are the leading code, m is a positive integer, m<M, the 4m+nth bit of the binary number is opposite to the binary numbers of all the remaining bits except the leading code, n=1, 2, ..., N; Step 12: All light emitting devices start to light up and down simultaneously according to their respective 4M-bit binary digital frame codes, where 0 represents that the light emitting device is off, and 1 represents that the light emitting device is on. 3. The method for locating objects in a storage room based on visible light signals as described in claim 2 is characterized in that in the step 12, all the light-emitting devices start to light up and off at the same time according to their respective 4M-bit binary frame codes, and a string of binary frame codes controls the light-emitting devices to light up and off within a frame duration, and the frame duration includes multiple pulse signal cycles in microseconds, and each binary digit in the frame code of the light-emitting device controls the light-up and off state of the light-emitting device within a pulse signal cycle. 4. The method for locating objects in a storage room based on visible light signals as described in claim 3 is characterized in that, while in the step 2, the first image of the object to be located illuminated by all light-emitting devices is obtained in the form of image pixel column exposure, the second image of the object to be located illuminated by all light-emitting devices is obtained in the form of image pixel column exposure; wherein the exposure time of the first image is one pulse signal cycle, the unit is microsecond, and the exposure time of the second image is S pulse signal cycles, the unit is microsecond, S>1, wherein S is the ratio between the maximum sensitivity and the minimum sensitivity of the hardware for acquiring the first image and the second image, the first image is acquired using the maximum sensitivity of the acquisition hardware, and the second image is acquired using the minimum sensitivity of the acquisition hardware. 5. The method for locating objects in a storage room based on visible light signals as described in claim 4 is characterized in that a QR code is affixed to the object to be located, and before obtaining the first image and the second image of the object to be located, the object to be located is found by identifying the QR code information of all objects in the warehouse. 6. The method for locating an object in a storage room based on visible light signals according to claim 5, characterized in that the step of acquiring a first image of the object to be located illuminated by all light-emitting devices in the form of exposure of each image pixel column is performed according to the following steps: Step A, acquiring a first original image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure of image pixel columns one by one and within an exposure time of one pulse signal cycle; Step B, intercepting an image including only the object to be located in the first original image by setting ROI, to obtain a first image; The acquisition of the second image of the object to be positioned illuminated by all the light emitting devices in the form of exposure of each image pixel column is performed according to the following steps: Step a, acquiring a second original image of the object to be located that is illuminated by all the light-emitting devices in the form of exposure of image pixel columns one by one and within S pulse signal cycles as the exposure time; Step b: intercepting an image including only the object to be located in the second original image by setting ROI to obtain a second image. 7. The method for locating objects in a storage room based on visible light signals as claimed in claim 6, characterized in that, when obtaining the sequence of brightness accumulation values of all pixel columns in the first image in step 3, the sequence of brightness accumulation values of all pixel columns in the first image is obtained by calculating the sum of the grayscale values of each column in the first image; and the sequence of brightness accumulation values of all pixel columns in the second image is obtained by calculating the sum of the grayscale values of each column in the second image; The first image brightness accumulated value sequence is divided by the second image brightness accumulated value sequence to obtain an image brightness gain sequence. 8. The method for locating objects in a storage room based on visible light signals according to claim 7, wherein the step 4 of obtaining the brightness gain of each light-emitting device from the image brightness gain sequence in the form of an image is performed according to the following steps: Step 41, finding the brightness gain waveform of the preamble code in the image brightness gain sequence in the image form, and obtaining the horizontal coordinate length D occupied by the brightness gain waveform of the preamble code, where the unit is pixel, and D is a positive integer; Step 42: Use Formula II to obtain the horizontal coordinate length d occupied by the one-bit preamble brightness gain waveform, in pixels, where d is a positive integer:

Wherein, m is a positive integer; The horizontal axis length occupied by the one-bit preamble code brightness gain waveform is the horizontal axis length occupied by the one-bit binary frame code brightness gain waveform; Step 43, in the image brightness gain sequence in the image form, taking the last horizontal coordinate point of the brightness gain waveform of the leading code as the starting point, taking the horizontal coordinate length occupied by the frame encoding brightness gain waveform of a binary number as the step length, successively find the horizontal coordinate length range of the brightness gain waveforms of the 1st to Nth light-emitting devices in pixels, and the absolute value of the difference between the brightness gain mean value corresponding to the brightness gain within the horizontal coordinate length range of the brightness gain waveform of each light-emitting device and the brightness gain corresponding to the highest point in the image brightness gain sequence in the image form is the brightness gain of each light-emitting device. 9. A device for positioning objects in a storage room based on visible light signals, characterized in that it is used to implement the method for positioning objects in a storage room based on visible light signals as described in any one of claims 1 to 8, wherein two-dimensional code information is attached to the object to be positioned, and the device for positioning objects in the storage room comprises a plurality of wireless sensor modules, a plurality of light-emitting devices, an image acquisition module, and a positioning data processing module; The wireless sensor module is connected to the visible light signal transmission module to obtain a 4M-bit binary frame code for each light-emitting device, wherein the frame code of the n-th light-emitting device, the first 4m bits are the leading code, m<M, m is a positive integer, M is a positive integer, the 4m+n-th binary number is opposite to the binary numbers of all other bits except the leading code, so that at least one light-emitting device is in a lighting state at the same time, n=1, 2, ..., N; Where 0 represents that the light-emitting device is off, and 1 represents that the light-emitting device is on; The light emitting device is connected to the image acquisition module and is used to start lighting up and down at the same time according to the respective 4M-bit binary digital frame codes, and at least one light emitting device is in the lighting state at the same time; The image acquisition module is connected to the positioning data processing module and is used to find the object to be located by identifying the two-dimensional code information of all objects in the warehouse; It is also used to obtain a first image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure column by column, and to obtain a second image of the object to be positioned that is illuminated by all the light-emitting devices in the form of exposure column by column; wherein the exposure time of the first image is one pulse signal cycle, in microseconds, and the exposure time of the second image is S pulse signal cycles, in microseconds, S>1, wherein S is the ratio between the maximum sensitivity and the minimum sensitivity of the hardware for acquiring the first image and the second image, and the first image is acquired using the maximum sensitivity of the image acquisition module, and the second image is acquired using the minimum sensitivity of the image acquisition module; The positioning data processing module is used to obtain the sum of the grayscale values of each column in the first image to obtain the brightness accumulation value sequence of all pixel columns in the first image; by obtaining the sum of the grayscale values of each column in the second image, the brightness accumulation value sequence of all pixel columns in the second image is obtained; Dividing the first image brightness accumulated value sequence by the second image brightness accumulated value sequence to obtain an image brightness gain sequence; Converting the image brightness gain sequence into an image form, wherein the image brightness gain sequence in the image form has a horizontal coordinate of pixels and a vertical coordinate of brightness gain; The positioning data processing module is also used to obtain the brightness gain of each light-emitting device from the image brightness gain sequence in the form of an image; The positioning data processing module is also used to obtain the distance of the object to be positioned from each light-emitting device according to the brightness gain of each light-emitting device using Formula I, in units of m:

Wherein, d ₁ , d ₂ , d _n and d _N are the distances between the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device and the object to be located, and the unit is m; RSS ₁ , RSS ₂ , RSS _n and RSS _N are the brightness gains of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device obtained in step 5; G is a positioning parameter, G>0; φ ₁ , φ ₂ , φ _n and φ _N are the emission angles of the first light-emitting device, the second light-emitting device, the nth light-emitting device and the Nth light-emitting device, and the unit is °;

as well as

is the incident angle of the first light-emitting device, the second light-emitting device, the nth light-emitting device, and the Nth light-emitting device on the object to be positioned, in degrees; The positioning data processing module is also used to obtain the position of the object to be positioned according to the position of each light-emitting device in the storage room and the distance between each light-emitting device and the object to be positioned. 10. The device for locating objects in a storage room based on visible light signals as claimed in claim 9, characterized in that the wireless sensor module is an ISA100.11a sensor.

Images