JP7105659B2 - Image information synthesizing device - Google Patents

Description

The present invention relates to an image information synthesizing device for synthesizing map information with automatic travel route information or actual travel route information of a work vehicle.

Patent Document 1 discloses a positioning device that calculates position information of a work vehicle using a positioning satellite, a storage device that stores map information, and plots the current position of the work vehicle positioned by the positioning device on the map information. and a control device for displaying on a display device.
Patent Document 2 discloses a work vehicle system in which work is performed by an unmanned automatically traveling work vehicle that automatically travels and a manned accompanying traveling vehicle that travels side by side with the automatically traveling work vehicle. A high-precision position detection device is mounted on an automatically traveling work vehicle because it is necessary to automatically travel the automatically traveling work vehicle along a preset automatic travel route in a field. On the other hand, the companion vehicle is equipped with a position detection device having a position detection accuracy lower than that of the position detection device mounted on the autonomous working vehicle.

JP 2016-74372 A JP 2015-194981 A

When the work vehicle is to automatically travel, an automatic travel route is created in advance. In order to create an automatic driving route, it is necessary to specify the position and shape of the field. Therefore, normally, a position detection unit mounted on a work vehicle is used to generate and store field identification information for identifying the position and shape of the field. Then, an automatic travel route is generated for the field identified by the field identification information.

It is conceivable that the automatic driving route information generated in this way and the actual driving route information (actual driving route information) detected by the position detection unit are combined with the map information and displayed on the display.
However, the positional accuracy of the existing map information provided on the Internet is generally lower than the positional information detected by the position detection unit mounted on the work vehicle. Therefore, if the automatic driving route information and the actual driving route information are directly combined with the map information, a composite image is generated in which the position of the route information and the position of the map information do not match.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image information synthesizing device capable of obtaining a synthesized image in which the position of automatic travel route information or actual travel route information and the position of map information are matched.

An embodiment of the present invention includes a first storage unit that stores first field identification information that is generated based on map information and that identifies a predetermined field area; a second storage section for storing second field identification information generated using a position detection section for positioning and for identifying the area of the field; and an actual travel route of the work vehicle detected by the position detection section. an image information synthesizing unit for synthesizing route information composed of automatic traveling route information created based on information or the second field specifying information with the map information, wherein the image information synthesizing unit is configured to combine the map information with the first field position; Based on the coordinate difference between the position of the field specified by the information and the position of the field specified by the second field specifying information, each position information constituting the route information corresponds to a point in the map information. An image information synthesizing device configured to synthesize the route information with the map information after the route information is corrected so as to be converted into the position information of the map.

With this configuration, a composite image is obtained in which the position of the automatic driving route information or the actual driving route information and the position of the map information are matched.
In one embodiment of the present invention, the second agricultural field identification information is created based on the position of the working vehicle detected by the position detection unit while the working vehicle is traveling along the contour of the agricultural field. be done.

In one embodiment of the present invention, the coordinate difference is coordinates between a barycentric position of the farm field identified by the first field position information and a barycentric position of the farm field identified by the second field identifying information. difference.

FIG. 1 is a schematic diagram showing the configuration of an image information synthesizing system to which an image information synthesizing apparatus according to an embodiment of the present invention is applied. FIG. 2 is a schematic diagram showing an example of an automatic driving route. FIG. 3 is an electrical block diagram showing electrical configurations of the work vehicle and the control terminal. FIG. 4 is an electrical block diagram showing the electrical configuration of the server. FIG. 5 is a flowchart for explaining the operation of the image information synthesizing section. FIG. 6 is a schematic diagram for explaining the processing of steps S3 to S6 in FIG. FIG. 7 is a schematic diagram showing an example of the contour FA of the farm field _F after synthesis and the actual travel route information Q. As shown in FIG.

Embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
FIG. 1 is a schematic diagram showing the configuration of an image information synthesizing system to which an image information synthesizing apparatus according to an embodiment of the present invention is applied.
The image information synthesizing system 1 includes a work vehicle 2 equipped with a communication terminal 3, a control terminal 4, a user terminal 5, and a server 6 as an image information synthesizing device. In this embodiment, work vehicle 2 is a tractor.

The working vehicle 2 includes a traveling machine body 8 as a vehicle body that travels in the field. Various working machines 9 such as a cultivator, a plow, a fertilizer applicator, a lawn mower, and a sowing machine can be selectively attached to the traveling body 8 . In this embodiment, a cultivator is attached as the working machine 9 .
The work vehicle 2 can travel automatically and manually. Automatic travel means that the steering mechanism of the work vehicle 2 is controlled by the control unit 22 and the work vehicle control device 21 (see FIG. 3) provided in the work vehicle 2, and work is performed along a preset automatic travel route. It means that the vehicle 2 runs. On the other hand, manual travel means that the work vehicle 2 travels by operating each mechanism provided in the work vehicle 2 by the user.

FIG. 2 is a schematic diagram showing an example of an automatic driving route.
The automatic travel route R is generated so as to connect the work start position S and the work end position E in the field F. As shown in FIG. The automatic travel route R is composed of a plurality of linear work paths R1 and a connection path R2 that alternately connects the ends of two work paths R1 that are adjacent to each other. A plurality of work paths R1 are set in a work area W within the farm field F, and a plurality of connection paths R2 are set in a non-work area N within the farm field F. The work area W is an area where work is performed, and the non-work area N is an area where no work is performed.

The work path R1 is a route along which work is performed by the work machine 9 . The connection path R2 is a turning path including an arc-shaped portion where turning and turning operations are performed. In the automatic traveling route R created in this way, since the direction is changed by 180° at each connection route R2, the traveling direction of the work vehicle 2 is one work route R1 and one work route R1 adjacent thereto. are in opposite directions to each other.

Returning to FIG. 1, the communication terminal 3 includes a position detector 31 (see FIG. 3) that measures the position of the work vehicle 2 using the reference station 11 and the positioning satellites 12. FIG. Communication terminal 3 can communicate with server 6 via telephone network 13 . Also, the communication terminal 3 can wirelessly communicate with the control terminal 4 .
The control terminal 4 is a mobile terminal such as a smart phone, a tablet personal computer (tablet PC), or the like. The control terminal 4 remotely controls the work vehicle 2 and can wirelessly communicate with the communication terminal 3 . The control terminal 4 can communicate with the server 6 via the Internet line 15 by being connected to an access point 14 such as a wireless LAN router installed in the user's home or office.

The user terminal 5 is installed in the user's home or office. The user terminal 5 includes a personal computer (PC) 5A, a display 5B, and an operating device 5C such as a keyboard. The personal computer 5A includes a CPU, memory, hard disk, etc., although not shown. The hard disk stores an operating system (OS), a program such as a browser for browsing web pages, and other necessary data. The user terminal 5 can communicate with the server 6 via the Internet line 15 by being communicatively connected to the access point 14 .

The personal computer 5</b>A registers in the server 6 first field identification information for identifying a predetermined field owned by the user (hereinafter referred to as “display target field F”). Further, the personal computer 5A acquires from the server 6 a composite image obtained by combining the map image with the automatic travel route or the actual travel route (actual travel route) of the work vehicle 2 for the display target farm field F, and displays it on the display 5B. .

The first field identification information is information for identifying the position and shape of the display target field F, and is registered based on the web page for field registration provided by the server 6 . Specifically, the Web page for field registration includes a map image (map information). When the user designates the area of the display target field F on the map image, the server 6 specifies the first field. Information is generated and registered with the server. The first field identification information is composed of, for example, positional information of a plurality of feature points on the outline of the field F to be displayed. When the shape of the display target field F is rectangular, for example, the first field identification information is composed of position coordinates of four vertices.

In this embodiment, the position of the map image is defined in a plane coordinate system (hereinafter referred to as "plane coordinate system") having a predetermined position as the origin, a straight line passing through the origin and extending in the east-west direction as the X-axis, and a straight line passing through the origin and extending in the north-south direction as the Y-axis. , “map plane coordinate system”). In this map plane coordinate system, the positive direction of the Y-axis is assumed to be north, and the positive direction of the X-axis is assumed to be east. In this embodiment, it is assumed that the position information forming the first agricultural field identification information generated based on the map image is also represented by the coordinate values of the map plane coordinate system.

FIG. 3 is an electrical block diagram showing electrical configurations of the work vehicle 2 and the control terminal 4. As shown in FIG.
The work vehicle 2 includes a work vehicle control device 21 and a communication terminal 3 mounted on the work vehicle 2 . The work vehicle control device 21 controls the operation of the traveling body 8 (moving forward, backward, stopping, turning, etc.) and the operation of the working machine 9 (moving up and down, driving, stopping, etc.). A plurality of controllers (not shown) for controlling each part of the work vehicle 2 are electrically connected to the work vehicle control device 21 .

The plurality of controllers include an engine controller that controls the number of revolutions of the engine, a vehicle speed controller that controls the vehicle speed of the work vehicle, a steering controller that controls the steering angle of the front wheels of the work vehicle 2, and controls the elevation of the work implement 9. It includes a lift controller, a PTO controller that controls the rotation of the PTO shaft, and the like.
The communication terminal 3 has a control section 22 . A position detection unit 31, an inertial measurement unit (IMU) 32, a communication unit 33, a wireless communication unit 34, an operation display unit 35, an operation unit 36, a storage unit 37, and the like are connected to the control unit 22. .

The position detection unit 31 calculates position information of the work vehicle 2 based on the satellite positioning system. The satellite positioning system is, for example, RTK (Real Time Kinematic)-GNSS (Global Navigation Satellite System). RTK-GNSS (real-time kinematic GNSS) uses a reference station 11 (see FIG. 1) installed at a predetermined position. At predetermined time intervals, the reference station 11 acquires positioning information of the reference station 11 calculated based on GNSS satellite signals received from a plurality of positioning satellites 12 (see FIG. 1), and its pre-recognized position (recognition self-position), and transmits the difference information as a positioning correction signal.

The satellite signal receiving antenna 16 receives satellite signals from the positioning satellites 12 . A reference station signal receiving antenna 17 receives positioning correction information from the reference station 11 . The position detector 31 acquires satellite signals from a plurality of GNSS satellites via the satellite signal reception antenna 16 . The position detection unit 31 also acquires positioning correction information from a predetermined reference station 11 via the reference station signal reception antenna 17 . The position detection unit 31 calculates positioning information of the work vehicle 2 based on satellite signals acquired from a plurality of positioning satellites, and combines the obtained positioning information of the work vehicle 2 with the positioning correction information acquired from the reference station 11. Position information representing the position of the work vehicle 2 is calculated by correcting using the position information. Position information consists of longitude, latitude and altitude information, for example. In this embodiment, the location information shall consist of longitude and latitude information.

The inertial measurement device 32 is a measurement unit capable of measuring the posture (orientation), acceleration, and the like of the work vehicle 2 .
The communication section 33 is a communication interface for the control section 22 to communicate with the server 6 via the telephone line network 13 .
The wireless communication unit 34 is a communication interface for wireless communication between the control unit 22 and the control terminal 4 . The wireless communication unit 34 is composed of, for example, a wireless LAN router. A wireless communication antenna 18 is connected to the wireless communication unit 34 .

The operation display unit 35 is composed of, for example, a touch panel display. The operation unit 36 includes, for example, one or more operation buttons.
The storage unit 37 is composed of a storage device such as a nonvolatile memory. The storage unit 37 is provided with a position information storage unit 38, an automatic travel route storage unit 39, and the like. The position information storage unit 38 stores the position information for each time calculated by the position detection unit 31 together with the time information. Automatic driving route information generated by the control terminal 4 is stored in the automatic driving route storage unit 39 .

The control unit 22 includes a microcomputer having a CPU and memory (ROM, RAM, nonvolatile memory, etc.).
The control unit 22 reads the position information stored in the position information storage unit 37 in real time or at a predetermined timing, and transmits it to the control terminal 4 . Further, the control unit 22 controls the work vehicle control device 21 to cause the work vehicle 2 to automatically travel along a pre-generated automatic travel route or stop the automatic travel.

The control terminal 4 has a control section 40 . A wireless communication unit 51 , an operation display unit 52 , an operation unit 53 , a storage unit 54 and the like are connected to the control unit 40 .
The wireless communication unit 51 is a communication interface for the control terminal 4 to communicate with the communication terminal 3 of the work vehicle 2 . The wireless communication unit 51 is composed of, for example, a wireless LAN adapter. A wireless communication antenna 19 is connected to the wireless communication unit 51 .

The operation display unit 52 displays various data and accepts user's operations. The operation display unit 52 is composed of, for example, a touch panel display. The operation unit 53 includes, for example, one or more operation buttons.
The storage unit 54 is composed of a storage device such as a nonvolatile memory. The storage unit 54 is provided with a position information storage unit 55, a second agricultural field identification information storage unit 56, an automatic travel route storage unit 57, and the like.

The location information storage unit 55 stores location information for each time received from the communication terminal 3 . The location information for each time is composed of, for example, time information and location information included in the received location information. When the control unit 22 of the communication terminal 3 transmits the position information stored in the storage unit 37 to the control terminal 4 in real time, the position information for each time is obtained by adding the time information on the control terminal 4 side to the received position information. It may consist of additional information.

The second agricultural field identification information storage section 56 stores the second agricultural field identification information generated by the control section 40 . The automatic traveling route storage unit 57 stores automatic traveling route information for the display target farm field F generated by the control terminal 4 .
The control unit 40 includes a microcomputer having a CPU and memory (ROM, RAM, nonvolatile memory, etc.) 41 .

The control unit 40 includes a second agricultural field identification information generation unit 42, an automatic travel route generation unit 43, and the like.
The second field identification information generation unit 42 uses the position detection unit 31 of the communication terminal 3 to generate second field identification information for identifying the position and shape of the display target field F. FIG. The second field identification information is composed of, for example, positional information of a plurality of feature points on the outline of the field F to be displayed. When the shape of the display target field F is rectangular, for example, the second field identification information is composed of position coordinates of four vertices.

The second field identification information is generated, for example, as follows. That is, the user manually drives the work vehicle 2 to circle the work vehicle 2 along the outer circumference of the field F to be displayed. The second agricultural field identification information generation unit 42 acquires position information for each time detected by the position detection unit 31 of the communication terminal 3 during this period. Then, the second field identification information generation unit 42 generates the second field identification information based on the acquired position information for each time. The second agricultural field identification information generated in this manner is stored in the second agricultural field identification information storage unit 56 .

The automatic driving route generation unit 43 generates an automatic driving route based on the second field identification information stored in the second farm field identification information storage unit 56 and the information necessary for automatic driving route generation input by the user. to generate The automatic driving route information generated in this manner is stored in the automatic driving route storage unit 57 .
The user operates the control terminal 4 to transfer the automatic driving route information generated by the automatic driving route generating section 43 to the automatic driving route storage section 39 in the communication terminal 3 . Thereafter, the user manually drives the work vehicle 2 to move the work vehicle 2 to the start position of the automatic travel route. Then, the user operates the control terminal 4 to start automatic running. Thereby, the work vehicle 2 is controlled to travel along the automatic travel route.

Position information for each time in the position information storage unit 55, automatic driving start time information when automatic driving started, automatic driving end time information when automatic driving ended, second field identification in the second field identification information storage unit 56 The information, the automatic driving route information in the automatic driving route storage unit 57, and the like are transmitted to the server 6 at arbitrary timing. For example, when the user inputs an information transmission command while the control unit 40 is connected to the access point 14 (see FIG. 1) installed in the user's home or office, It is transmitted to the server 6 via the Internet line 15 .

FIG. 4 is an electrical block diagram showing the electrical configuration of the server 6. As shown in FIG.
The server 6 has a control section 60 . A communication unit 71 , an operation display unit 72 , an operation unit 73 , a storage unit 74 and the like are connected to the control unit 60 .
The communication unit 71 is a communication interface for the control unit 60 to communicate with the communication terminal 3 of the work vehicle 2, the control terminal 4, the user terminal 5, and the like.

The operation display unit 72 displays various data and receives operations by the person in charge. The operation display unit 72 is composed of, for example, a touch panel display. The operation unit 73 includes, for example, one or more operation buttons.
The storage unit 74 is composed of a storage device such as a hard disk or nonvolatile memory. The storage unit 74 is provided with a position information storage unit 75, a first farm field identification information storage unit 76, a second farm field identification information storage unit 77, an automatic travel route storage unit 78, and the like. The first agricultural field identification information storage section 76 is an example of the first storage section of the present invention. The second agricultural field identification information storage section 77 is an example of the second storage section of the present invention.

The position information storage unit 75 stores the position information for each time received from the control terminal 4, the automatic travel start time, the automatic travel end time, and the like.
The first agricultural field identification information registered by the user terminal 5 is stored in the first agricultural field identification information storage unit 76 .
The second agricultural field identification information storage unit 77 stores the second agricultural field identification information received from the control terminal 4 .

Automatic driving route information received from the control terminal 4 is stored in the automatic driving route storage unit 78 .
The control unit 60 includes a microcomputer having a CPU and memory (ROM, RAM, nonvolatile memory, etc.) 61 . The control unit 60 includes an image information synthesizing unit 62 that synthesizes route information, which is automatic travel route information or actual travel route information for the display target field F, with map information.

This map information is the same map information as the map information presented to the user terminal 5 to generate the first field identification information. The actual travel route information of the work vehicle 2 may be the actual travel route information detected by the position detection unit 31 while the work vehicle 2 is automatically traveling in the display target farm field F, or the display target farm field F. Actual travel route information detected by the position detection unit 31 when the work vehicle 2 is manually traveling in the interior may be used.

The image information synthesizing unit 62 generates route information based on the coordinate difference between the position of the display target field F represented by the first field position information and the position of the display target field F represented by the second field specifying information. After correcting the route information so that each position information constituting the map information is converted into the position information of the corresponding point of the map information, the route information is combined with the map information.
FIG. 5 is a flowchart for explaining the operation of the image information synthesizing section 62. As shown in FIG.

Here, a case will be described in which the actual travel route information detected by the position detection unit 31 while the work vehicle 2 is automatically traveling in the display target farm field F is combined with the map information.
The image information synthesizing unit 62 reads the first and second field identification information stored in the first and second field identification information storage units 76 and 77 into the memory (work memory) 61 . The image information synthesizing unit 62 also selects the time calculated by the position detecting unit 31 when the work vehicle 2 is automatically traveling in the display target farm field F, among the position information for each time in the position information storage unit 75 . Actual travel route information (time-series information) consisting of position information for each vehicle is read into the memory (work memory) 61 (step S1).

Next, a plurality of positional information (longitude, latitude) that constitute the second field specifying information are converted into positional information (x, y) in the map plane coordinate system, and a plurality of positional information that constitute the actual traveling route information (Longitude, latitude) is converted into position information (x, y) in the map plane coordinate system (step S2).
The XY coordinate system in FIG. 6 indicates map plane coordinates. _A solid line FA in FIG. 6 indicates an example of the contour of the display target field F specified by the first field specifying information. A dashed-dotted line FB in FIG. 6 shows an example of the contour of the display target field _F specified by the second field specifying information after the coordinate conversion in step S2. Curve Q in FIG. 6 indicates the actual travel route represented by the actual travel route information after the coordinate conversion in step S2. In FIG. 6, the shape of the display target farm field F is rectangular for convenience of explanation, but the display target farm field F may have any shape other than the rectangle.

Next, the image information synthesizing unit 62 calculates a first barycentric position _{G A (x A , y A ) , which} is the barycentric position of the display target field FA specified by the first field specifying information (step S3).
Next, the image information synthesizing unit 62 calculates the second barycentric position G _B (x _B , y _B ), which is the barycentric position of the display target field F _B specified by the second field specifying information after the coordinate conversion in step S2. Calculate (step S4).

G _A in FIG. 6 indicates the first center-of-gravity position G _A (x _A , y _A ), and GB in FIG. 6 indicates the second center _- of _{-gravity position GB (x B ,} y _B ).
Next, the image information synthesizing unit 62 _calculates the coordinate differences Δx and Δy between the first gravity center position G _A (x _A , y _A ) and the second gravity center position GB (x _B , y _B ) using the following equation (1). (step S5).
Δx=x _A −x _B
Δy=yA _{- yB} (1)
Next, using the coordinate differences Δx and Δy, the image information synthesizing section 62 converts each position information (x, y) constituting the actual travel route information after the coordinate conversion in step S2 into a corresponding point on the map image. (step S6). Specifically, the image information synthesizing unit 62 adds the coordinate differences Δx and Δy to each position information (x, y) constituting the actual traveling route information after the coordinate conversion in step S2, thereby obtaining the actual traveling route. Each positional information (x, y) included in the information is corrected.

Finally, the image information synthesizing unit 62 synthesizes the actual travel route information after correction in step S6 with the map information (step S7). As a result, a composite image is obtained in which the positions of the actual travel route information and the map information are matched. FIG. 7 shows an example of the outline FA of the farm field _F after synthesis and the actual travel route information Q. In FIG.
As a result, it is possible to provide the user terminal 5 with a web page including a composite image in which the position of the actual travel route information and the position of the map information match at any timing. As a result, the user can cause the display 5B of the user terminal 5 to display a composite image in which the position of the actual travel route information and the position of the map information match.

FIG. 5 describes the operation of the image information synthesizing unit 62 when synthesizing the actual traveling route information detected by the position detecting unit 31 with the map information while the work vehicle 2 is automatically traveling in the display target farm field F. did. However, by a similar operation, the actual traveling route information detected by the position detection unit 31 while the working vehicle 2 is manually traveling in the display target field F is combined with the map information, or the map information is generated by the control terminal 4. It is also possible to synthesize the automatic travel route information for the display target farm field F with the map information.

In the above-described embodiment, the positions of the map information and the first field identification information are expressed by the coordinate values of the plane coordinate system, but the positions of the map information and the first field identification information are expressed by the coordinate values of the latitude and longitude coordinate system. may be represented. If the positions of the map information and the first field identification information are represented by coordinate values in the latitude/longitude coordinate system, the process of step S2 in FIG. 5 is unnecessary.
In the above-described embodiment, synthesized image information in which the position of the actual travel route information and the position of the map information are matched is obtained. Also, it is possible to obtain synthesized image information in which the position of the automatic driving route and the position of the map information are matched.

In the above-described embodiments, the work vehicle is a tractor, but the work vehicle may be a rice planter, a combine harvester, a civil engineering/construction work device, a snowplow, a riding work machine, a walking work machine, or the like.
In addition, various design changes can be made within the scope of the matters described in the claims.

1 image information synthesis system 2 work vehicle 3 communication terminal 4 control terminal 5 user terminal 6 server 11 reference station 12 positioning satellite 21 work vehicle control unit 22 control unit 31 position detection unit 38 position information storage unit 39 automatic traveling route storage unit 40 control unit 42 second field identification information generating unit 43 automatic driving route generating unit 55 position information storage unit 56 second field identification information storage unit 57 automatic driving route storage unit 60 control unit 61 memory 62 image information synthesizing unit 71 communication unit 75 position information storage Section 76 First farm field identification information storage section 77 Second farm field identification information storage section 78 Automatic travel route storage section

Claims (3)

a first storage unit for storing first field identification information generated based on map information and for identifying a predetermined field area;
a second storage unit for storing second field identification information for identifying the area of the field, which is generated using a position detection unit that measures the position of the work vehicle using a positioning satellite;
Route information including automatic travel route information created based on the actual travel route information of the work vehicle in the farm field detected by the position detection unit or the second farm field identification information is combined with the map information. and an image information synthesizing unit,
The image information synthesizing unit, based on a coordinate difference between the center of gravity of the farm field specified by the first farm field position information and the center of gravity of the farm field specified by the second farm field specifying information, calculates the route information. image information, after correcting the route information so that each position information constituting the synthesizer. 2. The second agricultural field identification information according to claim 1, wherein the second agricultural field identification information is created based on the position of the working vehicle detected by the position detection unit while the working vehicle is traveling along the contour of the agricultural field. image information synthesizing device. The first field position information and the second field identification information are composed of position information of a plurality of feature points on the contour line of the field. 3. The image information synthesizing device according to claim 1 or 2.

Images