JP2021077951A - Apparatus for identifying terminal based on correspondence level with target on motion state, program, and method - Google Patents

Abstract

To provide a terminal identification apparatus capable of determining a correspondence relation between a terminal and a target existed under an environment which may be existed in the terminal by using information relating to the communication with the terminal without depending on only a mutual position relation.SOLUTION: A main terminal identification apparatus includes: terminal motion state determination means of acquiring communication radio wave information from communication means performing a communication with a terminal, and determining terminal motion state information on the basis of the communication radio wave information; target motion state determination means of acquiring environment information as information according to an environment from a sensor sensing the environment in which the terminal may be existed, and determining the target motion state information from the environment information by detecting a prescribed target; and correspondence relation determination means of calculating a level that the determined terminal motion state information and the determined target motion state information are corresponded, determining whether or not the terminal according to the terminal motion state information and the target according to the target motion state information are in the correspondence relation on the basis of the level, and determining the information according to the correspondence relation between the terminal and the target.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for acquiring information about a terminal as a communication target, particularly a technique for identifying the terminal.

The upcoming 5th generation mobile communication system (5G) will make it possible to realize high-performance communication such as high speed, large capacity, low delay, and multi-terminal connection by using radio waves in the millimeter wave band. Here, millimeter waves have high straightness and are less likely to be diffracted, and there is a high possibility that their propagation will be blocked or attenuated by an object existing between the terminal and the base station. Therefore, in 5G, there is a big problem that the received power is sharply lowered due to the intervention of such an object, and the communication quality is significantly deteriorated.

Here, if it is possible to predict a decrease in the received power due to such an object, it is possible to maintain the communication quality by controlling the communication path of the terminal. As a technique for predicting the received power, for example, Non-Patent Document 1 uses time-series three-dimensional environment information acquired by an RGB-D camera capable of generating a depth image in addition to an RGB image, and is used in a communication device by machine learning. A method for predicting received power is disclosed.

Specifically, in this method, for the time-series environmental information that is the result of imaging the communication environment, the time-series information of the received power measured (at the same time as imaging) at the terminal that received the radio wave from the transmitting station is used as the correct answer data. A machine learning model is constructed using the associated teacher data, and the predicted value of the received power at the receiving terminal is determined from the time-series environmental information acquired by the RGB-D camera using the constructed machine learning model. ing.

Further, for example, Patent Document 1 is a technique for improving the estimation accuracy of communication quality in the initial operation stage of a system or in a communication environment where the frequency of occurrence is low in the communication quality prediction processing by machine learning as disclosed in Non-Patent Gazette 1. Is disclosed. Specifically, in the virtual space formed from the measurement results of the communication environment, the information by the RGB-D camera is virtually generated, and the communication quality of the receiving terminal in the virtual space is estimated by the propagation simulation. It is said that it is possible to generate a large amount of training data above and build a machine learning model with improved estimation accuracy.

JP-A-2018-148297 Japanese Unexamined Patent Publication No. 2016-212675 Japanese Unexamined Patent Publication No. 2019-144941 Japanese Unexamined Patent Publication No. 2019-174164

Rishi Nishio, "Wireless Communication Quality Prediction and Communication Control by Machine Learning", Shingaku Giho, Vol. 118, No. 57, SR2018-1, pp. 1-8, 2018

However, in the prior art as disclosed in Non-Patent Document 1 described above, for example, in a normal situation where a plurality of objects are included in the time-series environmental information, when the correct answer data is obtained among these objects. It is not possible to identify the terminal corresponding to the received power information of the series. As a result, it becomes very difficult to build a machine learning model with high estimation accuracy that can be used in actual operation. That is, it is determined whether or not the object (which may include the terminal) captured by the RGB-D camera and the terminal related to the measured received power are the same target (whether or not they have a corresponding relationship). Since it cannot be done, it is not possible to generate highly reliable training data in the first place.

Even if a machine learning model as disclosed in Non-Patent Document 1 is applied, for example, an object (terminal) detected from time-series environmental information generated by an RGB-D camera is the machine learning model. It is completely unknown whether or not it is related to the received power predicted by, and therefore, even on the transmitting station side, which terminal is connected to the predicted received power, and even In the first place, it becomes impossible to determine even whether or not the predicted received power is related to any terminal.

In this regard, in the technique disclosed in Patent Document 1, the correspondence relationship between the terminal imaged by the RGB-D camera and the measured received power can be determined in the virtual space, so that, for example, a situation in which a plurality of terminals can move. It is possible to build a machine learning model corresponding to. However, even if such a model is applied, the terminal cannot be identified after all in the scene of predicting the received power in the actual operation, and the same problem as that of Non-Patent Gazette 1 as described above occurs.

Here, Patent Document 2 discloses an object recognition system that grasps the behavior of a mobile terminal and associates the mobile terminal with a moving object detected by a computer. In this system, each mobile terminal sequentially detects various state quantities such as acceleration acting on its own terminal, transmits the time-series detection result to the server as terminal behavior data, and the server receives an image input from the camera. The behavior of moving objects existing in the shooting range is monitored based on the signal, and data indicating the time change of the state amount related to the behavior of each moving object is stored in the server-side storage unit as estimation result history data. Next, by comparing the terminal behavior data transmitted from the mobile terminal with the estimation result history data for each mobile body, the correspondence relationship between the mobile body existing in the shooting range and the mobile terminal is specified.

However, in the system according to Patent Document 2, the only criterion for specifying the correspondence is the physical state quantity caused by the behavior of the mobile body, and each mobile terminal is a base station, which is a matter of concern. No information related to the communication to be performed is used. In addition, since it is necessary for the mobile terminal side to acquire information on a predetermined state that changes due to its own behavior, equipment such as a sensor capable of detecting the predetermined state is indispensable for each mobile terminal. Furthermore, since it becomes necessary to sequentially transmit the detection results, it becomes a technically troublesome situation.

Further, Patent Document 3 describes the flow line information of a person acquired by using a camera or the like in an environment in which one or more persons possessing a wireless receiver are staying or moving, and the wireless reception. A technique for associating with a machine (a person who owns it) is disclosed. Specifically, this technology identifies the time zone in which the person is staying in the predetermined area from the flow line information of the person, and the wireless transmitter installed at the position corresponding to the predetermined area and the receiver possessed by the person. From the reception history information generated by communication with and, a person who is likely to be staying in a predetermined area in that time zone is determined, and the person is associated with the flow line information that specifies the time zone. It has become a thing.

However, in the technique of Patent Document 3, in order to associate the flow line information of the person who possesses the wireless receiver with the flow line information of the person acquired by the camera, it is a major premise that the person stays in a predetermined area. It has become. Of course, by setting a large number of predetermined areas, the probability that the person will stay in the predetermined area increases, but in that case, it becomes necessary to install wireless transmitters in each of the large number of predetermined areas.

Further, Patent Document 4 discloses a technique of acquiring received electromagnetic wave information in a terminal that has received an electromagnetic wave from an electromagnetic wave source (beacon) and estimating the position of the terminal. Specifically, this technique generates learning data including this received electromagnetic wave information, object recognition information acquired from an object recognition means capable of recognizing a movable object, and terminal position information related to the correct position of the terminal. , A model for terminal position estimation is generated, and using the generated model, the position estimation target is based on the feature data including the received electromagnetic wave information and the object recognition information at the time of position estimation acquired from the position estimation target terminal. It determines the position of the terminal.

However, in the technique of Patent Document 4, the correspondence between the terminal related to the received electromagnetic wave information and the object (terminal) related to the object recognition information is basically based on the terminal position estimated from the received electromagnetic wave information and the object recognition information. Since it is performed based on the geometrical positional relationship (for example, distance) with the acquired object position, for example, if there are many objects near the estimated terminal position, there is a possibility that incorrect mapping will be performed. Will be high. Further, in reality, it is necessary to arrange a plurality of beacons in the area, and it is not realistic to apply them to the entire general communication area.

Therefore, the present invention provides information related to communication with the terminal without relying on the mutual positional relationship or only the positional relationship for the correspondence relationship between the terminal and the object existing in the environment in which the terminal can exist. It is intended to provide devices, programs and methods that can be utilized and determined.

According to the present invention, communication radio wave information, which is information related to communication radio waves, is acquired from a communication means that communicates with a terminal, and based on the communication radio wave information, terminal motion, which is information related to a change in the position of the terminal. Terminal movement state determining means for determining state information,
An object that acquires environmental information that is information related to the environment from a sensor that senses the environment in which the terminal can exist, detects a predetermined object from the environmental information, and is information related to a change in the detection position of the object. Target exercise state determination means for determining exercise state information,
The degree of correspondence between the determined terminal movement state information and the determined target movement state information is calculated, and based on the corresponding degree, the terminal related to the terminal movement state information and the target movement state information are related. A terminal identification device having a correspondence relationship determining means for determining whether or not a target has a correspondence relationship and determining information relating to the correspondence relationship between the terminal and the target is provided.

As one embodiment of the terminal identification device according to the present invention, the communication radio wave information includes time-series information of the frequency of the received radio wave, time-series information of the intensity of the received radio wave, and time-series information of the radiation direction of the radio wave related to communication. , And at least one of the round trip time time series information in the communication,
The terminal motion state information is determined from at least one of the frequency shift, the hourly variation of the intensity, the hourly variation of the radiation direction, and the hourly variation of the round trip time. Information related to the speed or angular velocity of the terminal.
The target motion state information is information related to the speed or angular velocity of the target determined from the change in the detection position of the target per hour.
The corresponding degree is the degree to which the information related to the speed or the angular velocity of the terminal and the information related to the speed or the angular velocity of the target match.
It is also preferable that the correspondence relationship determining means determines whether or not the terminal and the target have a correspondence relationship based on the degree of matching.

Further, in the above embodiment, the terminal motion state information includes the deviation of the frequency, the variation of the intensity per hour, the variation of the radiation direction per hour, and the variation of the round trip time per hour. Information on the speed or angular velocity of a plurality of terminals determined from at least two of the above.
The degree of agreement is the degree to which each of the information related to the speed or the angular velocity of the plurality of terminals matches the information related to the speed or the angular velocity of the target, and is calculated in a plurality.
Whether the terminal and the target have a correspondence relationship based on the ratio of the plurality of matching degrees that are equal to or higher than a predetermined value, or the sum of the plurality of matching degrees or the sum of the weights. It is also preferable to determine whether or not.

Further, as another embodiment of the terminal identification device according to the present invention, the terminal motion state means also determines terminal position information, which is information related to the position of the terminal, based on the communication radio wave information.
The target movement state determining means also determines the target position information, which is the information related to the detection position of the target detected from the environmental information.
The correspondence relationship determining means calculates the degree of position matching, which is the degree to which the determined terminal position information and the determined target position information match, and based on the degree of position matching, the terminal and the target are connected to each other. It is also preferable to determine whether or not there is a correspondence relationship.

Further, as a further embodiment of the terminal identification device according to the present invention, the above-mentioned sensors are a plurality of sensors provided at different positions from each other.
The target motion state determining means acquires a plurality of environmental information from each of the plurality of sensors, determines the target motion state information from each of the plurality of environmental information, and among the determined target motion state information, For those whose detection positions of the objects related to the target motion state information are close to a predetermined value or more, and / or for those whose feature amounts of the target detection areas related to the target motion state information are similar to each other more than a predetermined value. It is also preferable that the target exercise state information relates to the same target.

Further, as a further embodiment of the terminal identification device according to the present invention, the target motion state determining means is separated from other terminal identification devices existing in the vicinity in the other terminal identification device (from the sensor described above). Acquire the target exercise state information determined from the environmental information acquired from the sensor of
The correspondence relationship determining means also calculates the degree of correspondence between the acquired target movement state information and the determined terminal movement state information, and based on the corresponding degree, the terminal and the target have a correspondence relationship. It is also preferable to determine whether or not there is.

Further, as a further other embodiment of the terminal identification device according to the present invention, the terminal includes a plurality of terminals installed or carried on a mobile body.
This terminal identification device
An approach target identification means for identifying a plurality of targets that are presumed to approach each other later than a predetermined value based on the detected positions of the plurality of detected objects.
Information relating to the fact that it is presumed to approach at least one of at least one terminal determined to have a correspondence relationship with any of the plurality of identified targets by the correspondence relationship determination means. It is also preferable to further have a communication control means for transmitting the signal.

According to the present invention, the learning data includes the environmental information and the information related to the target detected from the environmental information, and the terminal identification device as described above makes a correspondence relationship with the target. A learning model for estimating terminal radio wave information constructed by learning data in which information related to the radio wave of the terminal determined to exist is used as correct answer data is provided.

According to the present invention, further, data including the environmental information and information related to the target detected from the environmental information is input to the learning model according to the present invention described above, and the terminal output by the learning model. A terminal radio wave information estimation means that estimates information related to this target radio wave as a terminal based on the information related to the radio wave of
A communication relay device is provided which has a communication control means for determining whether or not to switch the communication path of the target as a terminal based on the information related to the estimated radio wave.

According to the present invention, it is also a program for operating a computer capable of determining a correspondence relationship between a terminal and a predetermined target.
Communication radio wave information, which is information related to communication radio waves, is acquired from a communication means that communicates with the terminal, and terminal motion state information, which is information related to a position change of the terminal, is determined based on the communication radio wave information. Terminal movement state determination means and
Environmental information, which is information related to the environment, is acquired from a sensor that senses the environment in which the terminal can exist, and the target is detected from the environmental information, and the target movement, which is information related to the change in the detection position of the target. Target exercise state determination means for determining state information,
The degree of correspondence between the determined terminal movement state information and the determined target movement state information is calculated, and based on the corresponding degree, the terminal related to the terminal movement state information and the target movement state information are related. A terminal identification program that functions a computer as a correspondence relationship determining means for determining whether or not a target has a correspondence relationship and determining information related to the correspondence relationship between the terminal and the target is provided.

According to the present invention, a computer-based terminal identification method capable of determining the correspondence between a terminal and a predetermined target is further provided.
Communication radio wave information, which is information related to communication radio waves, is acquired from a communication means that communicates with the terminal, and terminal motion state information, which is information related to a position change of the terminal, is determined based on the communication radio wave information. On the other hand, environmental information, which is information related to the environment, is acquired from a sensor that senses the environment in which the terminal can exist, and the predetermined target is detected from the environmental information to relate to a change in the detection position of the target. Steps to determine the target movement state information, which is information,
The degree of correspondence between the determined terminal movement state information and the determined target movement state information is calculated, and based on the corresponding degree, the terminal related to the terminal movement state information and the target movement state information are related. Provided is a terminal identification method having a step of determining whether or not a target has a correspondence relationship and determining information relating to the correspondence between the terminal and the target.

According to the terminal identification device, the program and the method of the present invention, the correspondence between the terminal and the object existing in the environment in which the terminal can exist is the said, without relying on the mutual positional relationship or only the positional relationship. It is possible to make a decision using information related to communication with the terminal.

It is a schematic diagram which shows one Embodiment of the terminal identification apparatus by this invention. It is a schematic diagram for demonstrating one Embodiment of the processing in the terminal movement state determination part, the target movement state determination part, and the correspondence relation determination part which concerns on this invention. It is a schematic diagram for demonstrating one application example of the terminal identification method by this invention. It is a schematic diagram for demonstrating one Embodiment of the processing in the model construction part and the terminal radio wave information estimation part which concerns on this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Terminal identification device]
FIG. 1 is a schematic view showing an embodiment of a terminal identification device according to the present invention.

The base station 1 as the terminal identification device according to the present invention shown in FIG. 1 is a communication relay device compatible with 5G (fifth generation mobile communication system) in the present embodiment, and is a plurality of communication terminals compatible with 5G. It is a device capable of performing communication with the terminal 2. Further, in the present embodiment, the base station 1 includes a camera 103, which is an RGB-D camera capable of capturing a situation in a communication area with a predetermined angle of view.

The (RGB and depth) image data (video data) generated by the camera 103 includes
(A) A person who possesses or carries a terminal 2
(B) Moving objects such as automobiles, motorcycles, railroad cars, robots, drones, etc. equipped with, including, or boarding the terminal 2, and (c) equipment, facilities, buildings, fixed objects, etc. on which the terminal 2 is installed. There is a possibility that such "objects" are included (imaged), and in addition, people, moving objects, facilities, facilities, buildings, and fixed (including plants such as trees) that are not related to the terminal 2 are fixed. An "object" such as an object can also be included.

By the way, while such an "object" can determine the correspondence with the terminal 2 to be identified, it also becomes an obstacle to communication radio waves because it exists between the terminal 2 and the base station 1. obtain. In particular, 5G, which is the communication method of the present embodiment, uses radio waves in the millimeter wave band for communication, but the millimeter wave has high straightness and is less likely to be diffracted, so that the terminal 2 and the base station 1 are used. There is a high possibility that the propagation will be blocked or attenuated by the "object" that exists between and.

Therefore, in 5G, it is a big problem that the received radio wave power at the base station 1 is sharply lowered due to the intervention of such an "object", and the communication quality is significantly deteriorated. Here, if it is possible to predict the decrease in the received radio wave power at the base station 1 due to such an "object", it is possible to maintain the communication quality by controlling the communication path of the terminal 2 or the like. Therefore, the base station 1 of the present embodiment not only determines the correspondence relationship between the terminal 2 and the "target", but also predicts such received radio wave power and controls the communication path, as will be described later. It is a device that also enables.

Similarly, in FIG. 1, the base station 1 first determines the correspondence between the terminal 2 and the "target", as a specific feature thereof.
(A) "Communication radio wave information" which is information related to radio waves of communication from a communication means (in this embodiment, a communication interface 101, a communication control unit 114, and a communication history information storage unit 102) for communicating with the terminal 2. And the terminal movement state determination unit 111 that determines the "terminal movement state information" which is the information related to the position change of the terminal 2 based on the "communication radio wave information".
(B) "Environmental information" (camera image data in this embodiment), which is information related to the environment, is acquired from a sensor (camera 103 in this embodiment) that senses the environment in which the terminal 2 can exist, and this "environment" is obtained. A target movement state determination unit 112 that detects a predetermined "target" from "information" and determines "target movement state information" that is information related to a change in the detected position of the detected "target".
(C) Calculate the "correspondence degree" which is the degree of correspondence between the determined "terminal exercise state information" and the determined "target exercise state information", and based on this "correspondence degree", "terminal exercise" It is determined whether or not the terminal 2 related to the "state information" and the "target" related to the "target exercise state information" have a correspondence relationship, and the information related to the correspondence relationship between the terminal 2 and the "target" is ". It has a correspondence relationship determination unit 113 that determines "terminal target correspondence information".

Here, the "communication radio wave information" of the above (A) is (a) time-series information of the frequency of the received radio wave, (b) time-series information of the intensity of the received radio wave, and (c) radiation of the radio wave related to the communication. The "terminal motion state information" that includes at least one of the time-series information of the orientation and the time-series information of the round trip time in (d) communication and is determined from the time-series information is (a') the received radio wave. Frequency shift, (b') fluctuations in the intensity of received radio waves per hour, (c') fluctuations in the radiation direction of radio waves related to communication, and (d') time of round trip time in communication. It is also preferable that the information is related to the speed or angular speed of the terminal 2 determined from at least one of the fluctuations of the hit.

On the other hand, it is also preferable that the "target motion state information" in (B) above is information related to the velocity or angular velocity of the "target" determined from the change in the detection position of the "target" per time.

Further, in these preferred forms, the "correspondence" of the above (C) is set as the degree to which the information related to the speed or the angular velocity of the terminal 2 and the information related to the speed or the angular velocity of the "target" match. The correspondence relationship determination unit 113 determines whether or not the terminal 2 and the "target" have a correspondence relationship based on such "correspondence degree".

As described above, according to the base station 1 as the terminal identification device according to the present invention, the correspondence between the terminal 2 and the "object" existing in the environment in which the terminal 2 can exist (detected from the "environmental information"). Can be determined by comparing each "movement state" derived from the "communication radio wave information" of the terminal 2 and the position change of the detected "target" with each other. That is, the "terminal target correspondence information" can be determined from the comparison of the respective "exercise states" without relying on the mutual positional relationship between the terminal 2 and the "target".

As will be described later, in the base station 1, an embodiment in which "terminal target correspondence information" is determined based on not only the mutual "movement state" of the terminal 2 and the "target" but also the mutual positional relationship. It is also possible to take. However, in any case, the base station 1 can determine the correspondence relationship between the terminal 2 and the "target" without relying only on the mutual positional relationship. As a result, for example, when a large number of "objects" exist in the vicinity of the terminal 2 to be identified (when detected), an erroneous correspondence relationship occurs because the difference in position between them is small. It is also possible to significantly reduce the risk of being determined by considering those "exercise states".

Although 5G is adopted as the communication method between the base station 1 and the terminal 2 in the present embodiment, naturally other communication methods such as LTE may be used, and the terminal identification according to the present invention. The communication between the device and the terminal may be based on various other wireless communication standards. For example, even in a communication method in which the problem of shielding by an object is not as remarkable as 5G, there are not a few situations in which it is desired to accurately identify a subordinate terminal using an "object" detected from "environmental information". For example, there are various terminal identification needs, such as determining the relationship between the browsing page of a user who is determined to have a corresponding relationship with a certain terminal and the flow line of the user and utilizing it for marketing and management. On the other hand, according to the terminal identification device according to the present invention, it is possible to carry out such a terminal identification process with high accuracy.

Further, the terminal identification device according to the present invention is not limited to a communication relay device such as a base station. For example, as a dedicated device for terminal identification processing, it may be provided in a form of being connected to a communication relay device / equipment such as a base station. Further, as the terminal identification device according to the present invention, it is also possible to use a cloud server, a non-cloud server device, a personal computer (PC), a notebook computer, a tablet computer, or the like equipped with the terminal identification program according to the present invention. is there.

Furthermore, at least one of the above (A) to (C), which is a component of the terminal identification device (base station 1) according to the present invention, is a device different from the other components. It is not impossible to take. For example, it is possible to realize the above-mentioned functions (A) to (C) by the whole of a plurality of servers. Here, even in such a case, all of these can be regarded as a terminal identification device or system that implements the terminal identification method according to the present invention.

[Functional configuration of terminal identification device, terminal identification program]
Similarly, in the functional block diagram of FIG. 1, the base station 1 has a communication interface 101, a communication history information storage unit 102, a camera 103, and a camera image storage as an embodiment of the terminal identification device and the communication relay device according to the present invention. It has a unit 104 and a processor memory. Here, the processor memory stores an embodiment of a communication relay program including the terminal identification program according to the present invention, and also has a computer function, and by executing this communication relay program, Perform terminal identification processing and communication relay processing.

Further, the processor memory communicates with the terminal motion state determination unit 111 including the terminal position determination unit 111a, the target motion state determination unit 112 including the target position determination unit 112a, and the correspondence relationship determination unit 113 as functional components. It has a control unit 114, an approach target identification unit 121, a model construction unit 131, and a terminal radio wave information estimation unit 132. Note that these functional components can be regarded as the functions of the terminal identification program and the communication relay program stored in the processor memory, and arrows are drawn between the functional components of the base station 1 in the functional block diagram of FIG. The process flow shown in connection with is also understood as an embodiment of the terminal identification method and the communication relay method according to the present invention.

Similarly, in the functional block diagram of FIG. 1, the communication control unit 114 functions as a base station by controlling the wireless communication operation between the communication interface 101 and each terminal 2, and further communicates with each terminal 2. Various information related to the above is acquired and recorded to generate communication history information in which the information is arranged in chronological order, and the communication history information storage unit 102 stores and manages the information.

Here, in the present embodiment, the communication history information is communication radio wave information, which is information related to radio waves used in communication with the terminal 2, specifically,
(A) Time-series information on the frequency of radio waves received from terminal 2,
(B) Time-series information of the strength of the radio wave received from the terminal 2 (received signal power, RSSI (Received Signal Strength Indicator)),
Information that includes at least one of (c) time-series information on the radiation direction of radio waves related to communication with the terminal 2 and (d) time-series information on the round trip time (RTT) in communication with the terminal 2. ing.

The terminal movement state determination unit 111 acquires the above-mentioned communication radio wave information from the communication history information storage unit 102, and determines the terminal movement state information which is information related to the position change of the terminal 2 based on the communication radio wave information. Here, in the present embodiment, this terminal movement state information is
(A') Frequency shift of radio waves received from terminal 2,
(B') Fluctuation of radio wave intensity (received signal power, RSSI) received from terminal 2 per hour,
A terminal determined from at least one of (c') fluctuations in the radiation direction of radio waves related to communication with terminal 2 and (d') fluctuations in RTT per hour in communication with terminal 2. It is information related to the speed or angular velocity of 2.

First, for the above (a'), the terminal motion state determination unit 111 determines the deviation Δf_t from the frequency f0_t originally set for communication from the time series information of the radio wave frequency f_t of the above (a). By generating time-series information and applying an object velocity measurement method using the Doppler effect, which is well known in the field of radio wave engineering, to this deviation Δf_t, the time-series information of the velocity vector v_e of the terminal 2 can be obtained. It may be generated and used as terminal motion state information.

Further, regarding the above (b'), the terminal motion state determination unit 111 is based on the time series information of the received signal power (RSSI) E_t of the above (b) and the transmission signal strength from the terminal 2 set in advance. For example, using the Fris transmission formula, which is well known in the field of radio engineering, time-series information of the distance D_t to the terminal 2 is generated, and the time-series information of the velocity vector v_e of the terminal 2 is obtained from the time change of this distance D_t. It may be generated and used as terminal motion state information.

Further, regarding the above (c'), the terminal motion state determination unit 111 (base station) is based on the time-series information of the pair (φ_t ψ_t) of the elevation angle φ_t and the azimuth angle ψ_t indicating the radiation direction of the radio wave of the above (c). The time series information of the angular velocity vector w_e of the terminal 2 (centered on 1) may be generated, and this may be used as the terminal motion state information.

Furthermore, for the above (d'), the terminal motion state determination unit 111 sets the radio wave propagation velocity V and the coefficient 0.5 that divides the RTT into one way with respect to the RTTR_t from the time series information of the RTTR_t of the above (d). By multiplying, time-series information of communication distance (R_t * V / 2) is generated, and time-series information of velocity vector v_e of terminal 2 is generated from the time change of this communication distance, and this is used as terminal motion state information. May be.

The velocity vector v_e described above is, in some cases or more accurately, a projection vector of the velocity vector of the terminal 2 in a predetermined direction (for example, the direction connecting the base station 1 and the terminal 2).

Although the process of determining the terminal motion state information of the terminal 2 has been described above, the terminal position determination unit 111a of the terminal motion state determination unit 111 can be used for the acquired communication radio wave information, although it is in a preferred embodiment to be described later. Based on this, it is also preferable to determine the terminal position information, which is the information related to the position of the terminal 2. Here, specifically, not only the received signal power at the base station 1 but also
(A) Received signal power measurement values at a plurality of base stations existing around the base station 1 and
(B) To determine the terminal position information of the terminal 2 by using a well-known base station positioning technology that acquires and uses the received signal power measurement value of the terminal 2 that has received radio waves from a plurality of base stations, for example, by communication. Can be done.

Similarly, in the functional block diagram of FIG. 1, the camera 103 may be a photographing device compatible with visible light, near infrared rays, or infrared rays including a solid-state image sensor such as a CCD image sensor or a CMOS image sensor, and is a stereo camera. It can also be a spherical (omnidirectional) camera. Of course, instead of the camera 103, it is also possible to adopt a sensor that can sense the environment in which the terminal 2 can exist and generate environmental information, such as a LiDAR, a laser / infrared positioning device, a TOF camera, and a thermography device. is there.

Here, in the present embodiment, the camera 103 is an RGB-D camera capable of generating an RGB image and a depth image as environmental information, and the generated environmental information (RGB image data and depth image data) is a camera image. It is stored and managed by the storage unit 104.

Although the camera 103 is installed in the base station 1 in the present embodiment, for example, it is a camera installed at a position separated from the base station 1, for example, a surveillance camera in the city, and communicates with the base station 1. It may be connected. Further, as will be described later, it is also possible to take an embodiment in which a plurality of cameras 103 are provided at different positions regardless of whether the base station 1 is inside or outside.

The target motion state determination unit 112 acquires (RGB and depth) image data (environmental information) from the camera image storage unit 104, detects a "predetermined target" from the image data, and detects the velocity or acceleration of the target. (Target motion state information), in this embodiment, the time series information of the velocity vector v_m or the angular velocity vector w_m is determined. By the way, this velocity vector v_m is projected in a predetermined direction (for example, the direction connecting the base station 1 and the terminal 2) of a predetermined target velocity vector together with the velocity vector v_e determined by the terminal motion state determination unit 111. It may be determined as a vector.

Here, the "predetermined target" to be detected is, for example,
(A) The person (user) who owns the mobile terminal,
(B) An automobile equipped with a terminal 2 having a drive recorder function, a CAN information transfer function, an interface function for automatic driving control by a server, and the like, and further.
(C) A person who may include a terminal 2 which is a communication destination related to communication history information, such as an autonomous mobile robot or an autonomous flight drone equipped with a terminal 2 having an interface function for autonomous movement control by a server. It can be "or" moving body ". Further, in some cases, "equipment", "facility", "building", "fixed object", etc., which may include the terminal 2, may also be "predetermined objects".

In addition, the time series information (information related to velocity or angular velocity) of the velocity vector v_m or the angular velocity vector w_m in such a predetermined object is obtained in the present embodiment.
(A) A predetermined target is detected from the acquired time-series image data by the target position determination unit 112a of the target movement state determination unit 112, and the time-series information of the detection position of the detected target in the image space. Is then derived, and then the conversion process from the image space coordinate system to the real space coordinate system is performed for the detection position.
(B) It can be determined by finding the time-dependent change of the detection position in the derived real space coordinate system. Incidentally, the angular velocity vector w_m is calculated with the origin of the vector, that is, the center of rotation as the position of the base station 1 (camera 103).

Here, regarding the detection of a predetermined target from the (RGB and depth) image data in (a) above, for example, a model for detecting a predetermined target is constructed based on a machine learning algorithm well known for various image recognition, and the said. Using the model, a detection area (for example, a bounding box) in which a predetermined object is presumed to exist in the image data is specified, and a score indicating the certainty that the detection area is a predetermined object is also calculated. A detection region having a high score sufficient to satisfy a predetermined condition can be determined as the target region. Further, the detection position of a predetermined target can be a representative point (for example, the center of gravity or the midpoint of the lower end) of the determined target region.

By the way, when a stereo camera is adopted as the camera 103, for example, non-patent documents: Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg, “SSD: single shot multibox detector”, European Conference on Computer Vision, Computer Vision-ECCV 2016, pp. 21-37, by applying the object detector described in 2016. And it is possible to determine the detection position.

Further, when LiDAR is used as the sensor according to the present invention instead of the camera 103, for the point cloud (point cloud) which is the generated environmental information, for example, Charles R. Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation”, Journal of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77-85, using the object detector described in 2016 Thereby, the target area and the detection position can be determined.

By the way, when the target motion state determination unit 112 does not detect a predetermined target from the environmental information, that is, when one predetermined target is not measured (sensed), or when it is measured but is not detected due to the influence of noise or the like. Ends the target exercise state determination process for the environmental information.

Further, the target position determining unit 112a of the target motion state determining unit 112 can determine the detected predetermined target position (detection position) as described above, but in the present embodiment, the detected predetermined target position is further determined. For each target, time-series information (movement history information) of the position of the target is generated after associating the target with the detection position at each time point. This also makes it possible to individually track each predetermined target detected.

Here, for such a target tracking process, for example, a Kalman filter, which is a well-known state estimation method, is applied to the determined target area (bounding box), and the target detection area (from the state at the past time point to the current time point) ( The bounding box) is predicted, and the target area is determined based on the evaluation value using the overlapped area between the predicted detection area (bounding box) and the detection area (bounding box) detected at the present time as an evaluation value. It is also preferable. By the way, such target tracking processing is described in, for example, non-patent documents: Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft, “Simple Online and Realtime Tracking”, Journal of 2016 IEEE International Conference on Image Processing (ICIP), Pp. 3464-3468, 2016.

Further, as another embodiment, when a plurality of sensors other than the camera 103 or the above-mentioned camera are provided at different positions inside or outside the base station 1, the target motion state determination unit 112 may be used.
(A) Obtaining a plurality of environmental information from each of the plurality of cameras 103 (or sensors),
(B) Target motion state information (for example, information related to a predetermined target speed or angular velocity) is determined from each of these plurality of environmental information.
(C) Of the determined target exercise state information
(C1) For objects whose detection positions related to the target motion state information are close to a predetermined value or more (for example, those whose distances from each other are less than a predetermined threshold value), and / or
(C2) Image features of the target detection area (bounding box) related to the target motion state information are similar to each other more than a predetermined value (for example, distances from each other in the image feature space are less than a predetermined threshold). about,
It is also preferable that the target exercise state information relates to the same target. Further, in this case, the representative value (for example, the average value) of the detection position group for the target related to the target movement state information may be treated as the position of the same target.

As a result, it is possible to avoid a situation in which a certain terminal 2 is not included in the acquired environmental information, and as a result, the identification process of the terminal 2 fails. For example, environmental information from one camera 103 (or sensor) is not included in the environmental information because a certain predetermined object is located behind another object and is shielded by the other object (imaging). (Not done) There are quite a few situations. For example, from one camera 103, it is quite possible that the person holding the terminal 2 is obstructed by the stopped bus and cannot be seen.

On the other hand, if environmental information from a plurality of cameras 103 (or sensors) having different installation positions is used, an object that is not included in one environmental information is included in other environmental information that has a different sensing (measurement) viewpoint. As a result, it is fully possible to avoid the above-mentioned concerns.

By the way, as a typical example of using the target motion state information related to the plurality of cameras 103 (or sensors) as described above, the base station 1 is one or more base stations 1 existing around itself. It is possible to receive and acquire the target motion state information generated by the camera 103 (or sensor) installed in the base station 1 from each of them, and perform the same target determination process as described above. In this case, it is possible for a plurality of base stations 1 to cooperate with each other to more preferably carry out the terminal identification process according to the present invention.

Similarly, in the functional block diagram of FIG. 1, the correspondence relationship determination unit 113
(A) Terminal motion state information determined by the terminal motion state determination unit 111, information related to the speed or angular velocity of the terminal 2 in the present embodiment, and
(B) The "correspondence degree", which is the degree of correspondence between the target movement state information determined by the target movement state determination unit 112 and the information related to the detected speed or angular velocity of the predetermined target in the present embodiment, is calculated. Based on this "correspondence degree", it is determined whether or not the terminal 2 related to the terminal movement state information of the above (a) and the predetermined target related to the target movement state information of the above (b) have a correspondence relationship. , The terminal target correspondence information which is the information related to the correspondence relationship between the terminal 2 and a predetermined target is determined.

Here, in the present embodiment, the above-mentioned "correspondence degree" includes the information related to the speed or the angular velocity of the terminal 2 in the above (a) and the information related to the detected predetermined target speed or the angular velocity in the above (b). Is a value indicating the degree of matching, for example,
(A) Regarding the reciprocal (1 / | v_e−v_m |) of the magnitude of the difference between the velocity vector v_e of the terminal 2 at each time point in the predetermined time interval and the velocity vector v_m of the predetermined target detected at the same point. The representative value (for example, the average value) in the predetermined time section may be set as the correspondence degree C in the predetermined time section.
(B) Regarding the reciprocal (1 / | w_e−w_m |) of the magnitude of the difference between the angular velocity vector w_e of the terminal 2 at each time point in the predetermined time interval and the angular velocity vector w_m of the predetermined target detected at the same point. The representative value (for example, the average value) in the predetermined time interval may be the correspondence degree C in the predetermined time interval.

Next, when the calculated correspondence degree C exceeds the predetermined threshold value C_th (when C> C_th), the correspondence relationship determination unit 113 causes the terminal 2 and the detected predetermined target (in the predetermined time interval) to move to each other. It is determined that there is a correspondence relationship, and for example, the terminal target correspondence information recorded by associating the terminal ID of the terminal 2 with the target ID of the predetermined target is determined. Note that this threshold C_th may be set to an empirically suitable value by, for example, an administrator.

Further, as a modification mode, the correspondence relationship determination unit 113 calculates the correspondence degree C for all the pairs generated between all the terminals 2 and all the detected predetermined targets, and the optimum result among them. It is also preferable to determine the correspondence relationship between the terminal 2 and the predetermined target by the greedy algorithm that selects the set that gives the (correspondence relationship) as the solution, and determine the terminal target correspondence information.

FIG. 2 is a schematic diagram for explaining an embodiment of processing by the terminal movement state determination unit 111, the target movement state determination unit 112, and the correspondence relationship determination unit 113.

In the embodiment shown in FIG. 2, the moving terminal 2 mounted on the automobile transmits a radio wave having a frequency f0_t, and the base station 1 communicating with the terminal 2 has a frequency (from this terminal 2). Receives radio waves of f0_t + Δf_t). The terminal motion state determination unit 111 of the base station 1 acquires communication radio wave information (communication history) in which the terminal ID of the terminal 2 is associated with the received radio wave frequency (f0_t + Δf_t), and from the time series information of the frequency deviation amount Δf_t. , The time-series information (terminal motion state information) of the velocity vector v_e of the terminal 2 is generated by using the well-known Doppler effect object velocity measurement method.

On the other hand, the automobile equipped with the terminal 2 is photographed by the camera 103 including its moving state, and the target motion state determining unit 112 of the base station 1 captures image data of the automobile as environmental information including the automobile. Is acquired, the image data is subjected to the detection process of the "automobile" preset as a predetermined target, and the time-series information of the detection position of the automobile equipped with the terminal 2 is generated. Next, the time-series information of the displacement (position change) vector Δl_m is generated from the time-series information of the detection position, and the time-series information of the velocity vector v_m corresponding to the displacement vector per unit time (target motion) is generated from the information. State information) is generated.

After that, the correspondence relationship determination unit 113 of the base station 1 determines from the time-series information of the generated speed vector v_e of the terminal 2 and the time-series information of the speed vector v_m of the detected vehicle (predetermined target). The inverse number (1 / | v_e−v_m |) of the magnitude of the vector difference at each time point in the time interval is calculated, and the degree of correspondence C (v_e, v_m) is determined from these values. Next, when the determined correspondence degree C (v_e, v_m) exceeds the predetermined threshold value C_th (when C> C_th), the correspondence relationship determination unit 113 corresponds between the terminal 2 and the detected automobile (predetermined target). It is determined that there is a relationship, and finally, a terminal target correspondence table (terminal target correspondence information) recorded by associating the terminal ID of the terminal 2 with the detected target ID of the automobile is generated.

As described above, according to the base station 1 of the present embodiment, the correspondence relationship between the terminal 2 and the predetermined target (automobile in FIG. 2) existing in the environment in which the terminal 2 can exist is determined by the terminal 2 and the predetermined target. It is possible to determine by comparing the speeds (movement states) of each other. That is, the terminal target correspondence information can be determined from the comparison of the respective speeds (exercise states) without depending on the positional relationship between the terminal 2 and the predetermined object, and in particular, the terminal 2 can move (exercise). Even if it is a terminal, the identification process can be performed more reliably.

Although one embodiment of the terminal identification process has been described above with reference to FIG. 2, in this embodiment, information related to the speed is generated from the frequency of the received radio wave. On the other hand, as another preferred embodiment, the terminal motion state information is provided.
(A') Frequency shift of radio waves received from terminal 2,
(B') Fluctuation of radio wave intensity (received signal power, RSSI) received from terminal 2 per hour,
At least two (for example, all four) of (c') fluctuations in the radiation direction of radio waves related to communication with the terminal 2 and (d') fluctuations in the RTT per hour in communication with the terminal 2). It is also preferable to use the information relating to the speed or the angular velocity of the plurality of terminals 2 determined from each of the above.

In this case, the degree of correspondence C is the degree to which each of the information related to the speeds or angular velocities of the plurality of terminals 2 and the information related to the detected speeds or angular velocities of a predetermined target match (for example, the magnitude of the difference between the two vectors). It can be the reciprocal of), and multiple calculations will be made. Here, the correspondence relationship determination unit 113 is, for example,
(A) When the ratio of those having a predetermined threshold value C_th or more among these plurality of correspondence degrees C is a predetermined value (for example, 0.5) or more, or
(B) When the sum of these plurality of correspondence degrees C or the sum of weighting is equal to or greater than a predetermined value,
It is also preferable to determine that the terminal 2 and the detected predetermined target have a corresponding relationship.

In any case, in the embodiment in which the identification process of the terminal 2 is carried out by integrating the plurality of correspondence degrees C in this way, the information related to the velocity or the angular velocity is estimated from various methods by different methods, so that various environments are available. Even under the above circumstances, the accuracy of the identification process of the terminal 2 is maintained or improved.

Furthermore, as another preferred embodiment, the correspondence relationship determination unit 113 (FIG. 1) is
(A) Terminal position information determined by the terminal position determination unit 111a (FIG. 1) of the terminal movement state determination unit 111, and
(B) The position correspondence degree (position matching degree) LC, which is the degree of matching with the target position information determined by the target position determining unit 112a (FIG. 1) of the target exercise state determining unit 112, is calculated, and the above-mentioned exercise state is calculated. It is also preferable to determine whether or not the terminal 2 and the detected predetermined target have a correspondence relationship based on not only the correspondence degree C but also the position correspondence degree LC.

Here, the position correspondence degree LC is, for example, a representative value in the predetermined time interval for the reciprocal of the absolute value (distance) of the difference between the position of the terminal 2 and the position of the predetermined target at each time point in the predetermined time interval. (For example, the average value) can be set. Further, in the correspondence relationship determination unit 113, for example, the correspondence degree C of the exercise state exceeds a predetermined threshold value C_th (C> C_th), and the position correspondence degree LC exceeds the predetermined threshold value LC_th (LC> LC_th). ), It is also preferable to determine that the terminal 2 and the detected predetermined target have a corresponding relationship.

In this way, by carrying out the identification process of the terminal 2 in consideration of the position correspondence degree LC, it is possible to determine a more accurate correspondence relationship (between the terminal 2 and a predetermined target).

Further, another preferred embodiment regarding the degree of correspondence C of the exercise state will be described. In this embodiment, at least one base station 1 exists around the base station 1, and each base station 1 determines the target motion state information from the environmental information obtained by its own camera 103, and further determines the target motion state information by itself. It is possible to exchange the target movement state information with each other.

Here, the target motion state determination unit 111 (FIG. 1) acquires the target motion state information determined by the surrounding base station 1, and further, the correspondence relationship determination unit 113 (FIG. 1) is acquired in this way. The correspondence degree C'between the target exercise state information and the determined terminal exercise state information is also calculated, and the terminal is based not only on the correspondence degree C of the target exercise state information determined by oneself but also on this correspondence degree C'. It is also preferable to determine whether or not 2 and a predetermined target have a corresponding relationship. In this case, it is possible to carry out the terminal identification process similar to the terminal identification process performed in the embodiment in which the base station 1 is connected to the plurality of cameras 103 as described above. Further, this makes it possible to avoid a situation in which a certain terminal 2 is not included in the environmental information by its own camera 103, and as a result, the base station 1 fails in the identification process of the terminal 2. It becomes.

[Application example of terminal identification processing]
Hereinafter, an application example of the terminal identification process described above will be described.

In the functional block diagram of FIG. 1, the approach target identification unit 121 was detected by the target movement state determination unit 112 in a situation where a plurality of movable terminals 2 exist in the communication area environment (other than a predetermined target). Based on the detection positions of a plurality of targets (including the target), a plurality of targets that are estimated to be closer to each other later than a predetermined value are identified.

For example, the approach target identification unit 121 is each at a future time point after a predetermined time elapses, based on the current detection position of each detected target and the velocity vector calculated from the past time change of the detection position. It is also preferable to estimate the position of the target and specify the target whose distance from each other at the future time point is equal to or less than a predetermined threshold as the approach target group.

In this case, the communication control unit 114 receives the information of the plurality of targets specified by the approach target identification unit 121, and the correspondence relationship determination unit 113 determines that the communication control unit 114 has a correspondence relationship with any of these plurality of targets. (At least one) terminal 2 is specified, and at least one of the specified terminals 2 is addressed with information related to the presumed approach to a predetermined value or more, such as an approach / collision alarm, an encounter / encounter forecast, and the like. It is transmitted (from the communication interface 101).

FIG. 3 is a schematic diagram for explaining an application example of the terminal identification method according to the present invention.

According to the application example of FIG. 3, the base station 1 remotely monitors an automobile (or a robot or the like) connected by communication with 5G. Specifically, the base station 1 is in communication connection with the terminal 2_001 having the terminal ID 001 and the terminal 2_002 having the terminal ID 002, and further, the correspondence relationship determination unit 113 of the base station 1 is
(A) The terminal 2_001 and the detected automobile 3_002 having the target ID 002 are in a corresponding relationship (that is, the terminal 2_001 is mounted on the automobile 3_002).
(B) The terminal 2_002 and the detected automobile 3_003 having the target ID of 003 are in a corresponding relationship (that is, the terminal 2_002 is mounted on the automobile 3_003).
It is decided that this is done, and a terminal target correspondence table that records that fact is generated.

On the other hand, the approach target identification unit 121 specifies the automobile 3_003 and the driver 4_001 whose target ID is 001 as one element of the approach target group, and the automobile 3_003 and the driver 4_001 are one approach target group. The proximity target group information, which is the information to be included in, is output to the communication control unit 114. In this application example, the driver 4_001 is in a situation where the building standing along the road becomes an obstacle and cannot be recognized by the sensor mounted on the automobile 3_003.

In such a situation, the communication control unit 114
(A) Based on the approach target group information, the terminal 2_002 having a correspondence relationship with the automobile 3_003 included in the approach target group is determined as the approach alarm notification destination from the acquired terminal target correspondence table.
(B) The alarm information that "the vehicle 4_001 (also included in the approach target group) is approaching" is transmitted to the terminal 2_002 determined as the notification destination.

In addition, the approach target identification unit 121 also transmits the detection position information of the pedestrian 4_001 to the terminal 2_002 in accordance with this alarm information, and sets the current position (and its flow line) of the pedestrian 4_001 to the road on the screen of the terminal 2. It is also preferable to highlight it on the map.

By carrying out the processing as described above, for example, it is possible to promote safe driving by the driver and safe walking / running of a person, and to prevent a collision accident caused by a moving object such as an automobile. In particular, a dangerous situation for a certain automobile, however, a dangerous situation that cannot be detected or predicted by the in-vehicle sensor (camera) can be notified to this automobile, which is the very party concerned, via the identified terminal 2. It will be possible.

[Communication relay processing, communication relay program]
Although various embodiments related to the identification process of the terminal 2 have been described above, the communication relay in the base station 1 realized by using the functional block diagram of FIG. 1 and FIG. 4 and the identification process is described below. An embodiment of the process will be described. Note that FIG. 4 is a schematic diagram for explaining an embodiment of processing in the model building unit 131 and the terminal radio wave information estimation unit 132, which will be described later.

In the functional block diagram of FIG. 1, the model building unit 131 builds a learning model for estimating terminal radio wave information. Specifically, in the present embodiment, the model building unit 131 is as shown in FIG. 4 (A).
(A) Time-series image data (environmental information) generated by the camera 103 (which may include an object related to the terminal 2 and an object which may be an electromagnetic interference obstacle) and
(B) Learning data including information related to a predetermined target detected from this time-series image data, for example, information related to a detection position of the target (target position information).
(C) Received radio wave information (at each time point or representative time point of the environmental information in (a) above) related to the terminal 2 determined by the correspondence relationship determination unit 113 to have a correspondence relationship with this predetermined target, for example, received radio wave. Training data using power as correct data is generated, and the training data is used to generate machine learning algorithms well-known in the field of image recognition (for example, a plurality of convolutional layers in a neural network algorithm and their outputs). The terminal radio wave information estimation model is constructed by using the combination with the Fully-Connected Layers).

Here, as already described in detail, in the conventional machine learning model as disclosed in Non-Patent Document 1, for example, an object detected from time-series environmental information generated by a camera is the machine learning model. It is not possible to determine whether or not it corresponds to the terminal related to the received radio wave power predicted by.

On the other hand, the terminal radio wave information estimation model constructed by the model building unit 131 is information related to the target (determined to have a corresponding relationship) that is exactly associated with the received radio wave information as correct answer data, for example, the target position. The information is included in the learning data and constructed. Therefore, the received radio wave information predicted by this terminal radio wave information estimation model, for example, the received radio wave power, can be interpreted as a predicted value for a predetermined target determined as the identification destination of the terminal 2 related to the received radio wave information. .. That is, the estimation process of the received radio wave information by the terminal radio wave information estimation model surely solves the unsolved problem of surely identifying the corresponding target in the estimation process by the conventional machine learning model.

Next, the terminal radio wave information estimation unit 132 estimates the received radio wave information of the terminal 2, for example, the received radio wave power, using the terminal radio wave information estimation model constructed by the model construction unit 131. Specifically, in the present embodiment, the terminal radio wave information estimation unit 132 has the terminal radio wave information estimation unit 132 as shown in FIG. 4 (B).
(A) Time-series image data (environmental information) generated by the camera 103, which includes the terminal 2 which is the estimation target and may include an object which may be an electromagnetic interference.
(B) The target position information of the predetermined target detected from the time-series image data and determined to have a corresponding relationship with the terminal 2 which is the estimation target is applied to the terminal radio wave information estimation model. As an input, based on the received radio power (as estimated information) output from the terminal radio information estimation model by this input, the predicted value of the received radio power (at the estimated time point or time interval) of the terminal 2 to be estimated is calculated. decide.

Finally, the communication control unit 114 (FIG. 1), which has acquired the predicted value of the received radio wave power of the terminal 2 (which has high estimation accuracy as described above), bases the terminal 2 (or the terminal 2) on the basis of the predicted value. It is determined whether or not to switch the communication path of (a predetermined target corresponding to). For example, when the predicted value in a predetermined future time point or time interval is equal to or less than a predetermined power threshold value, communication with the terminal 2 may be handed over to another adjacent base station 1.

Here, also in another adjacent base station 1, the predicted value of the received radio wave power in the same terminal 2 is determined, and the predicted value is shared among the base stations 1 to specify the largest predicted value. It is also preferable to switch the communication path to the base station 1 that has determined the specified maximum predicted value.

As described above, according to the base station 1 of the present embodiment, by using the terminal radio wave information estimation model constructed by using the terminal identification processing result, the communication failure due to the communication path obstruction which becomes a big problem in 5G. In particular, it is possible to carry out communication relay processing that can surely solve the problem of temporary communication failure due to a moving obstruction.

As described in detail above, according to the present invention, the correspondence between the terminal and the target existing in the environment in which the terminal can exist is derived from the communication radio wave information and the change in the target detection position, respectively. It is possible to determine by comparing the "states" with each other. That is, the "terminal target correspondence information" can be determined from the comparison of each "exercise state" without relying on the mutual positional relationship between the terminal and the target or only the positional relationship.

Further, the terminal identification process according to the present invention solves the problem of communication failure due to the communication path obstruction in the coming 5G, and notifies / warns the vehicle equipped with the terminal of the approach of a person or another vehicle. Furthermore, it is applied in various situations and fields, such as determining the relationship between the browsing page of a user who is determined to have a corresponding relationship with a certain terminal and the flow line of the user and utilizing it for marketing and management. It is possible.

In the various embodiments of the present invention described above, various changes, modifications and omissions in the technical idea and the scope of the present invention can be easily made by those skilled in the art. The above explanation is just an example and does not attempt to restrict anything. The present invention is limited only to the scope of claims and their equivalents.

1 Base station (terminal identification device, communication relay device)
101 Communication interface 102 Communication history information storage unit 103 Camera 104 Camera image storage unit 111 Terminal motion state determination unit 111a Terminal position determination unit 112 Target motion state determination unit 112a Target position determination unit 113 Correspondence relationship determination unit 114 Communication control unit 121 Approach target Specific part 131 Model construction part 132 Terminal radio wave information estimation part 2, 2_001, 2_002 Terminal 3_002, 3_003 Automobile 4_001 Traveler

Claims (11)

A terminal that acquires communication radio wave information, which is information related to communication radio waves, from a communication means that communicates with the terminal, and determines terminal motion state information, which is information related to a change in the position of the terminal, based on the communication radio wave information. Exercise state determination means and
An object that acquires environmental information that is information related to the environment from a sensor that senses the environment in which the terminal can exist, detects a predetermined object from the environmental information, and is information related to a change in the detection position of the object. Target exercise state determination means for determining exercise state information,
The degree of correspondence between the determined terminal movement state information and the determined target movement state information is calculated, and based on the corresponding degree, the terminal related to the terminal movement state information and the target movement state information are related. A terminal identification device comprising a correspondence relationship determining means for determining whether or not a target has a correspondence relationship and determining information relating to the correspondence between the terminal and the target. The communication radio wave information includes time-series information of the frequency of the received radio wave, time-series information of the intensity of the received radio wave, time-series information of the radiation direction of the radio wave related to communication, and time-series information of the round trip time in communication. Including at least one of
The terminal motion state information is determined from at least one of the frequency shift, the hourly variation of the intensity, the hourly variation of the radiation direction, and the hourly variation of the round trip time. Information related to the speed or angular velocity of the terminal.
The target motion state information is information related to the speed or angular velocity of the target determined from the change in the detection position of the target per hour.
The corresponding degree is the degree to which the information related to the speed or the angular velocity of the terminal and the information related to the speed or the angular velocity of the target match.
The terminal identification device according to claim 1, wherein the correspondence relationship determining means determines whether or not the terminal and the target have a correspondence relationship based on the degree of matching. The terminal motion state information is determined from at least two of the frequency shift, the hourly variation of the intensity, the hourly variation of the radiation direction, and the hourly variation of the round trip time. Information related to the speed or angular velocity of multiple terminals.
The degree of agreement is the degree to which each of the information related to the speed or the angular velocity of the plurality of terminals matches the information related to the speed or the angular velocity of the target, and is calculated in a plurality.
The correspondence relationship determining means has a correspondence relationship between the terminal and the target based on the ratio of the plurality of matching degrees that are equal to or higher than a predetermined value, or the sum of the plurality of matching degrees or the sum of the weights. The terminal identification device according to claim 2, wherein it determines whether or not. The terminal motion state means also determines terminal position information, which is information related to the position of the terminal, based on the communication radio wave information.
The target movement state determining means also determines the target position information, which is the information related to the detection position of the target detected from the environmental information.
The correspondence relationship determining means calculates the degree of position matching, which is the degree to which the determined terminal position information and the determined target position information match, and based on the degree of position matching, the terminal and the target The terminal identification device according to any one of claims 1 to 3, wherein the terminal identification device determines whether or not there is a correspondence relationship. The sensors are a plurality of sensors provided at different positions from each other.
The target motion state determining means acquires a plurality of environmental information from each of the plurality of sensors, determines the target motion state information from each of the plurality of environmental information, and among the determined target motion state information. , For those whose detection positions of the objects related to the target exercise state information are close to a predetermined value or more, and / or for those whose feature amounts of the target detection areas related to the target exercise state information are similar to each other more than a predetermined value. The terminal identification device according to any one of claims 1 to 4, wherein the target exercise state information relates to the same target. The target motion state determining means acquires target motion state information determined from environmental information acquired from a sensor other than the sensor in the other terminal identification device from other terminal identification devices existing in the vicinity. ,
The correspondence relationship determining means also calculates the degree of correspondence between the acquired target movement state information and the determined terminal movement state information, and based on the corresponding degree, the terminal and the target have a correspondence relationship. The terminal identification device according to any one of claims 1 to 5, wherein it is determined whether or not the terminal is present in the terminal identification device. The terminal includes a plurality of terminals installed or carried on a mobile body.
The terminal identification device is
An approach target identification means for identifying a plurality of targets that are presumed to approach each other later than a predetermined value based on the detected positions of the plurality of detected objects.
It is presumed that the correspondence relationship determination means approaches at least one of at least one terminal determined to have a correspondence relationship with any of the specified objects and approaches the predetermined value or more. The terminal identification device according to any one of claims 1 to 6, further comprising a communication control means for transmitting information. Learning data including the environmental information and information related to the target detected from the environmental information, which is in a corresponding relationship with the target by the terminal identification device according to any one of claims 1 to 7. A learning model for estimating terminal radio wave information, which is constructed by learning data in which information related to the radio wave of the terminal determined to be the correct answer data is used as correct answer data. Data including the environmental information and information related to the target detected from the environmental information is input to the learning model according to claim 8, and based on the information related to the radio wave of the terminal output by the learning model. As a terminal radio wave information estimating means for estimating information related to the target radio wave as a terminal,
A communication relay device comprising a communication control means for determining whether or not to switch the target communication path as a terminal based on the information related to the estimated radio wave. A program that functions a computer that can determine the correspondence between a terminal and a predetermined target.
Communication radio wave information, which is information related to communication radio waves, is acquired from a communication means that communicates with the terminal, and terminal motion state information, which is information related to a position change of the terminal, is determined based on the communication radio wave information. Terminal movement state determination means and
Environmental information, which is information related to the environment, is acquired from a sensor that senses the environment in which the terminal can exist, and the target is detected from the environmental information, and the target movement, which is information related to the change in the detection position of the target. Target exercise state determination means for determining state information,
The degree of correspondence between the determined terminal movement state information and the determined target movement state information is calculated, and based on the corresponding degree, the terminal related to the terminal movement state information and the target movement state information are related. A terminal identification program characterized in that a computer functions as a correspondence relationship determining means for determining whether or not a target has a correspondence relationship and determining information related to the correspondence relationship between the terminal and the target. A computer-based terminal identification method that can determine the correspondence between a terminal and a predetermined target.
Communication radio wave information, which is information related to communication radio waves, is acquired from a communication means that communicates with the terminal, and terminal motion state information, which is information related to a position change of the terminal, is determined based on the communication radio wave information. On the other hand, environmental information, which is information related to the environment, is acquired from a sensor that senses the environment in which the terminal can exist, and the predetermined target is detected from the environmental information to relate to a change in the detection position of the target. Steps to determine the target movement state information, which is information,
The degree of correspondence between the determined terminal movement state information and the determined target movement state information is calculated, and based on the corresponding degree, the terminal related to the terminal movement state information and the target movement state information are related. A terminal identification method comprising a step of determining whether or not a target has a correspondence relationship and determining information relating to the correspondence between the terminal and the target.

Images