JP7492369B2 - Parking assistance device and parking assistance method - Google Patents

Description

The present invention relates to a parking assistance device and a parking assistance method.

Parking assistance devices that assist in parking a vehicle are known. For example, Patent Document 1 discloses a parking assistance device that "provides assistance for parking a vehicle in a set target parking position, and includes a remote control means that is located away from a display device that displays an image and that remotely operates a pointer superimposed on the image, a direct movement means that directly moves the target parking frame to a coordinate position input by a user using the remote control means, a direction movement means that moves the target parking frame in a direction associated with a direction button selected by the user using the remote control means by selecting a direction button superimposed on the image, a switching means that switches between a direct movement mode by the direct movement means and a direction movement mode by the direction movement means by operating the remote control means, and a control means that automatically guides the vehicle to a target parking position that is set according to the position of the target parking frame."

JP 2010-95027 A

In some cases, automatic parking may not be possible if there are no white lines or other demarcation lines in the parking lot, or if the demarcation lines cannot be detected due to weather conditions, etc. In such cases, a method of setting a target parking frame by remotely operating a pointer superimposed on a captured image using a remote control means as in Patent Document 1 may be considered, but it requires time to get used to the operation before parking.
The present invention aims to provide a parking assistance device and a parking assistance method that enable a parking position to be set by intuitive operation even when the dividing lines separating a parking area cannot be detected, and that enables automatic parking at the set parking position.

In order to achieve the above object, the parking assistance device of the present invention is characterized by comprising: a situation detection unit that detects a situation around a vehicle; a display control unit that causes a display unit to display the situation around the vehicle detected by the situation detection unit and an icon; a reception unit that receives touch operations on a touch panel provided on the display unit, the touch operations being a first touch operation of touching the icon, a second touch operation of touching a position on the icon different from the touch position of the first touch operation, and a movement operation of moving the icon to a parking position where the vehicle is desired to be parked by the first touch operation and the second touch operation while keeping the icon touched by the first touch operation and the second touch operation; a determination unit that determines a parking position of the vehicle based on a position of the icon after it has been moved by the movement operation, and determines a parking direction based on a positional relationship between the first touch operation and the second touch operation in a direction in which the icon moves by the movement operation; a route generation unit that generates a parking route for the vehicle based on the parking position and the parking direction determined by the determination unit; and a control information generation unit that generates control information for causing the vehicle to run along the parking route generated by the route generation unit.

According to the present invention, even if the parking area demarcating lines cannot be detected, the parking position can be set by intuitive operation, and the vehicle can be automatically parked at the set parking position.

FIG. 1 is a diagram showing a configuration of a parking assistance device. FIG. 13 is a diagram showing an example of a display image. FIG. 13 is a diagram showing an example of a display image. FIG. 11 is a diagram showing a method of operating a vehicle icon. FIG. 11 is a diagram showing a method of operating a vehicle icon. FIG. 11 is a diagram showing a method of operating a vehicle icon. FIG. 13 is a diagram showing a method of operating a graphic icon. 4 is a flowchart showing the operation of the parking assistance device.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.
1 is a diagram showing a configuration of an on-vehicle device 3 mounted on a vehicle. The vehicle mounting the on-vehicle device 3 is referred to as a host vehicle 1A. In addition, a vehicle other than the host vehicle 1A is referred to as a different vehicle 1B.
The in-vehicle device 3 includes a position detection unit 10 , a sensor section 20 , an operation section 50 , a display section 60 , a vehicle control unit 70 , a steering device 71 , a drive device 73 , a braking device 75 , a transmission 77 , and a parking assistance device 100 .

The position detection unit 10 detects the current position of the vehicle 1A. The position detection unit 10 includes a GNSS (Global Navigation Satellite System) receiver that receives GNSS signals, and a processor that calculates the latitude and longitude, which are the current position of the vehicle 1A, based on the GNSS signals received by the GNSS receiver. The GNSS receiver and processor are not shown. The position detection unit 10 outputs the latitude and longitude indicating the current position of the vehicle 1A to the parking assistance device 100 as position information.

The sensor unit 20 includes multiple sensors. In this embodiment, the sensor unit 20 includes a photographing unit 30 having multiple cameras, and a sonar unit 40. In this embodiment, the sensor unit 20 includes a camera and a sonar, but the sensor of the sensor unit 20 is not limited to a camera and a sonar. For example, the sensor unit 20 may be equipped with a radar or a LiDAR (Laser Imaging Detection and Ranging) that can measure the distance to an obstacle using radio waves, light, etc. The sensor unit 20 outputs the captured image of the photographing unit 30 and the sensor data of the sonar unit 40 to the parking assistance device 100 as detection information.

The photographing unit 30 includes a front camera 31 that photographs the front of the vehicle 1A, a rear camera 32 that photographs the rear of the vehicle 1A, a left side camera 33 that photographs the left side of the vehicle 1A, and a right side camera 34 that photographs the right side of the vehicle 1A. Each of these cameras includes an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and a data processing circuit that generates an image from the light receiving state of the image sensor. The photographing unit 30 has an angle of view adjusted so that the four cameras can photograph a 360° range centered on the vehicle 1A. The front camera 31, the rear camera 32, the left side camera 33, and the right side camera 34 photograph their respective photographing ranges at a predetermined frame rate to generate a photographed image. The image generated by the front camera 31 is called a forward captured image, the image generated by the rear camera 32 is called a rearward captured image, the image generated by the left side camera 33 is called a left side captured image, and the image generated by the right side camera 34 is called a right side captured image. The image capturing unit 30 outputs these generated captured images to the parking assistance device 100. The parking assistance device 100 temporarily stores the input captured images in the memory 110.

The sonar units 40 are mounted in multiple locations on the vehicle 1A, such as the front, rear, left and right sides, and use ultrasonic waves to detect the size, position and distance to obstacles present around the vehicle 1A.

The operation unit 50 is a reception unit that receives operations from a passenger in the vehicle 1A. The operation unit 50 outputs operation information corresponding to the received operation to the parking assistance device 100. The operations received by the operation unit 50 include, for example, an operation to start parking assistance and an operation to end parking assistance.

The display unit 60 corresponds to the reception unit of the present invention. The display unit 60 includes a touch panel 65, and displays an image based on image data generated by the parking assistance device 100 on the touch panel 65. The touch panel 65 includes a display panel such as a liquid crystal display or an organic EL display, and a touch sensor that detects a touch operation on the display panel. The touch sensor may be a commonly known type of sensor such as a resistive film type or a capacitive type. The touch panel 65 detects a touch operation on the display panel using the touch sensor, and generates a position signal that indicates the operation position of the detected touch operation. The touch panel 65 outputs operation information including the generated position signal to the parking assistance device 100.

The vehicle control unit 70 is a computer device such as an ECU (Electronic Control Unit), and is a unit that controls the steering device 71, drive device 73, braking device 75, and transmission device 77 mounted on the host vehicle 1A. The vehicle control unit 70 is connected to the steering device 71, drive device 73, braking device 75, and transmission device 77 via an in-vehicle network such as a CAN (Controller Area Network). The vehicle control unit 70 controls the steering device 71, drive device 73, braking device 75, and transmission device 77 according to control information input from the parking assistance device 100.

The steering device 71 is a device including an actuator for steering the steering wheels of the host vehicle 1A.
The drive device 73 is a device including an actuator that adjusts the drive force of the drive wheels of the host vehicle 1 A. If the power source of the host vehicle 1 A is an engine, this actuator corresponds to a throttle actuator, and if the power source is a motor, this actuator corresponds to the motor.
The braking device 75 is a device that includes an actuator that controls a brake system provided in the host vehicle 1A based on information from the parking assistance device 100 and controls the braking force applied to the wheels of the host vehicle 1A.
The transmission 77 is a device including a transmission and an actuator. The transmission 77 drives the actuator to control the shift position of the transmission, and switches the gear ratio of the transmission and the forward and reverse motion of the host vehicle 1A.

The parking assistance device 100 is a computer device that includes a processor 130 such as a CPU (Central Processing Unit) or MPU (Microprocessor Unit), and a memory 110 such as a ROM (Read Only Memory) or RAM (Random Access Memory). In addition to these devices, the parking assistance device 100 also includes a storage device such as a HDD (Hard Disk Drive) or SSD (Solid State Drive), an interface circuit for connecting sensors and peripheral devices, and an in-vehicle network communication circuit for communicating with other in-vehicle devices via an in-vehicle network. The parking assistance device 100 realizes various functional configurations by having the processor execute computer programs stored in the memory 110 or the storage device.

The memory 110 stores computer programs executed by the processor 130, data processed when the processor 130 executes the computer programs, and data resulting from the processing. The memory 110 also stores conversion parameters for converting images captured by the image capture unit 30 into overhead images. The memory 110 also stores images captured by the image capture unit 30 and sensor values of the sonar unit 40. The memory 110 also stores trajectory data indicating the display position and orientation trajectory of the vehicle icon 5.

The parking assistance device 100 has, as its functional configuration, a position acquisition unit 131, a situation detection unit 132, an overhead image generation unit 133, a reception unit 134, a display control unit 135, a determination unit 136, a path generation unit 137, and a control information generation unit 138.

The latitude and longitude indicating the current position of the vehicle 1A calculated by the position detection unit 10 are input to the position acquisition unit 131 as position information. The position acquisition unit 131 estimates the current position of the vehicle 1A using a publicly known or well-known dead reckoning method, and corrects the current position of the vehicle input from the position detection unit 10 based on the estimation result.

The situation detection unit 132 receives, as detection information, images of the surroundings of the vehicle 1A captured by the image capture unit 30. The situation detection unit 132 temporarily stores the input captured images in the memory 110. The situation detection unit 132 also receives, as detection information, sensor values of the sonar unit 40. The situation detection unit 132 temporarily stores the input sensor values in the memory 110.

The situation detection unit 132 detects the situation around the vehicle 1A. The situation detection unit 132 detects the situation around the vehicle 1A based on the captured images stored in the memory 110 and the sensor values.
The surrounding conditions detected by the situation detection unit 132 include the presence or absence of obstacles, a parking area, a driving route along which the vehicle 1A is to be driven, etc. Obstacles include, for example, people, other vehicles 1B, structures such as pillars in a parking lot, shopping carts, etc. The situation detection unit 132 also executes a detection process to detect demarcation lines drawn on the road surface from the captured image and detect a parking area identified by the detected demarcation lines. The demarcation lines are lines such as white lines that demarcate the parking area of the vehicle.

The overhead image generating unit 133 converts the captured images acquired by the situation detecting unit 132 using the conversion parameters stored in the memory 110, and synthesizes the converted captured images to generate an overhead image.
The overhead image generating unit 133 converts the front, rear, right side, and left side images of the vehicle 1A captured by the cameras 31 to 34 into a front overhead image, a rear overhead image, a right side overhead image, and a left side overhead image obtained by looking down in a plan view from above the vehicle 1A. The overhead image generating unit 133 synthesizes the front overhead image, rear overhead image, right side overhead image, and left side overhead image obtained by the conversion into a single overhead image that is connected all around the vehicle 1A.

Operation information is input to the reception unit 134 from the touch panel 65 or the operation unit 50. The reception unit 134 functions as a reception unit that receives input of the position of a vehicle icon 5 (described later) and the orientation of the icon. The reception unit 134 outputs the received operation information to the display control unit 135.

The display control unit 135 generates a display image to be displayed on the touch panel 65 based on the input operation information, the overhead image generated by the overhead image generation unit 133, and the execution result of the detection process executed by the situation detection unit 132. The display control unit 135 causes the generated display image to be displayed on the touch panel 65.

2 and 3 are diagrams showing examples of a display image generated by the display control unit 135. FIG.
The display image is an image in which a vehicle icon 5 representing the vehicle 1A is superimposed on the overhead image generated by the overhead image generation unit 133. The display control unit 135 generates the display image by displaying the vehicle icon 5 at a position of the overhead image corresponding to the current position of the vehicle 1A. Furthermore, when a partition line dividing the parking area is detected by the situation detection unit 132, the display control unit 135 superimposes a candidate frame 200 representing the detected partition line on the display image. Fig. 2 shows a display image when a partition line is detected by the detection process executed by the situation detection unit 132, and Fig. 3 shows a display image when a partition line is not detected by the detection process executed by the situation detection unit 132.

In addition, when the operation information input from the reception unit 134 is information on an operation on the vehicle icon 5, the display image unit 135 changes the position of the vehicle icon 5 on the display image, i.e., the position on the overhead image, according to the operation information. Furthermore, when the operation information is information on an operation on the vehicle icon 5, the display control unit 135 acquires position information corresponding to the changed display position of the vehicle icon 5. In this embodiment, the case where this position information is a position on the overhead image is described, but the position information may be latitude and longitude indicating an actual position corresponding to the position on the overhead image, information on the distance and direction from the vehicle 1A, or the amount of movement from the position before the change. The display control unit 135 associates the acquired position information with the orientation of the vehicle icon 5 at this position, and stores the associated data in the memory 110 as trajectory data.

Furthermore, the display control unit 135 determines whether the operation information input from the reception unit 134 is a confirmation operation. The confirmation operation is an operation to confirm the parking position of the vehicle 1A. The confirmation operation may be an operation of a button provided on the operation unit 50, or a touch operation on the touch panel 65. If the operation on the vehicle icon 5 has not continued for a predetermined time or more, the display control unit 135 displays a display such as "Parking the vehicle 1A at a position corresponding to the display position of the vehicle icon 5", and determines that a confirmation operation has been performed when a positive operation is input by the operator. In addition, the display control unit 135 may display a display such as "Do you want to fine-tune the parking position?" on the touch panel 65, and adjust the parking position by moving it slightly toward the passenger seat of the vehicle 1A when an operation to fine-tune is input.

When a confirmation operation is input, the determination unit 136 determines a parking position where the vehicle 1A is to be parked and a parking direction when parking the vehicle 1A at this parking position, based on the display position of the vehicle icon 5 displayed on the touch panel 65 and the orientation of the vehicle icon 5. The determination unit 136 obtains an actual position corresponding to the position on the overhead image where the vehicle icon 5 is displayed, based on the latitude and longitude information obtained by the position acquisition unit 131, and determines the obtained position as the parking position.

The parking direction is the traveling direction of the host vehicle 1A when the host vehicle 1A is moved to the parking position, and includes forward parking, in which the host vehicle 1A is parked by moving forward, and backward parking, in which the host vehicle 1A is parked by moving backward. The determination unit 136 determines the parking direction based on the trajectory data generated by the display control unit 135. The trajectory data includes position information corresponding to the display position of the vehicle icon 5 and the orientation of the vehicle icon 5 at that position. If the vehicle icon 5 moves forward into the parking position, the determination unit 136 determines the parking direction to be forward parking. If the vehicle icon 5 moves backward into the parking position, the determination unit 136 determines the parking direction to be backward parking.

The route generating unit 137 generates a parking route for moving the host vehicle 1A to a parking position based on the trajectory data generated by the display control unit 135 and the situation around the host vehicle 1A detected by the situation detection unit 132. The route generating unit 137 generates a route that follows the trajectory data and that has the shortest travel distance for the host vehicle 1A. In addition, if an obstacle is detected by the situation detection unit 132 along the generated route, the route generating unit 137 corrects the parking route based on the position of the detected obstacle.

The control information generating unit 138 generates control information for controlling the driving of the host vehicle 1A based on the parking path generated by the path generating unit 137. This control information is information for driving the host vehicle 1A along the parking path, and includes information for controlling steering, driving, braking, and gear shifting.

Next, a method for changing the position and orientation of the vehicle icon 5 by a touch operation on the touch panel 65 will be described with reference to FIGS.
4 shows three vehicle icons 5A, 5B, and 5C. Vehicle icon 5A shows an icon displayed at a position corresponding to the current position of vehicle 1A. Vehicle icon 5C shows an icon displayed at a parking position where vehicle 1A is desired to be parked. Vehicle icon 5B shows an icon moving from the position of vehicle icon 5A to the position of vehicle icon 5C.

The operator performs a touch operation on the vehicle icon 5 displayed on the touch panel 65, and moves the vehicle icon 5 to the parking position where the operator wants to park the host vehicle 1A. The operator touches the vehicle icon 5 with his/her finger and moves the finger to the parking position without removing it from the touch panel 65. The display control unit 135 changes the display position of the vehicle icon 5 according to the operation information input from the touch panel 65. As a result, the display position of the vehicle icon 5 moves following the movement of the operator's finger. When the vehicle icon 5 moves to the position where the operator wants to park the host vehicle 1A, the operator removes his/her finger from the touch panel 65.

FIG. 4 shows a method of operating the vehicle icon 5 when backing into parking the host vehicle 1A.
The operator touches the rear side of the vehicle icon 5 with his/her finger and moves the touching finger to the parking position. When the operator parks the vehicle 1A forward, the operator touches the front side of the vehicle icon 5 with his/her finger and moves the touching finger to the parking position.

FIG. 5 is a diagram showing a method of performing a rotation operation for rotating the orientation of the vehicle icon 5. As shown in FIG.
When the operator wants to change the orientation of the vehicle icon 5, the operator touches an arbitrary position on the vehicle icon 5 and stops the touching finger without moving it for a certain period of time. When the input operation information indicates the same position on the touch panel 65 for a certain period of time or more, the display control unit 135 determines that a rotation operation for rotating the vehicle icon 5 has been selected, and displays a rotation operator 201 capable of rotating the vehicle icon 5 as shown in FIG. 5. The rotation operator 201 shown in FIG. 5 indicates that the vehicle icon 5 can be rotated clockwise. When the rotation operator 201 is displayed on the vehicle icon 5, the operator moves the finger touching the touch panel 65 in accordance with the rotation operator 201. The display control unit 135 rotates the orientation of the vehicle icon 5 in accordance with the input operation information.

FIG. 6 is a diagram showing another method of rotating the vehicle icon 5. In FIG.
For example, when a predetermined operation is received, the display control unit 135 displays the rotation operator 202 shown in Fig. 6. Examples of the predetermined operation include a touch operation of touching the vehicle icon 5 twice in succession, and an operation of rotating a touching finger near the vehicle icon 5. The predetermined operation may also be a button operation provided on the operation unit 50. When the rotation operator 202 is displayed on the touch panel 65, the operator touches the display position of the rotation operator 202 with a finger and moves the touching finger in a direction in which the operator wants to rotate the vehicle icon 5. The display control unit 135 rotates the orientation of the vehicle icon 5 according to the input operation information.

FIG. 7 is an explanatory diagram for explaining an operation method for an icon.
In the above description, the icon is the vehicle icon 5 in the shape of a vehicle, but the icon displayed on the touch panel 65 is not limited to the vehicle icon 5. Fig. 7 shows a case where a rectangular shape is displayed as an icon. This icon will be referred to as a graphic icon 7 below. In the case of the graphic icon 7 shown in Fig. 7, the orientation of the vehicle cannot be determined. In such a case, the operator operates the graphic icon 7 using multiple fingers to input the parking position and parking direction.

For example, the operator touches the end of the graphic icon 7 that the operator wishes to set on the front side of the vehicle 1A with a first finger. For example, the operator touches one end of the graphic icon 7 in the longitudinal direction with the operator's index finger. Next, the operator touches the end of the graphic icon 7 that the operator wishes to set on the rear side of the vehicle 1A with a second finger. For example, the operator touches the other end of the graphic icon 7 in the longitudinal direction with the operator's thumb.

The operator, with two fingers touching the touch panel 65, moves the graphic icon 7 to the parking position. At this time, when the operator is parking the vehicle 1A forward, the operator moves the graphic icon 7 with the first finger at the front. When the operator is parking the vehicle 1A backward, the operator moves the graphic icon 7 with the second finger at the front. The parking assistance device 100 determines the parking direction by determining whether the finger at the front of the movement direction of the graphic icon 7 is the touch position of the first finger or the touch of the second finger.

FIG. 8 is a flowchart showing the operation of the parking assistance device 100.
The operation of the parking assistance device 100 will be described with reference to the flow chart shown in FIG.
First, the parking assistance device 100 judges whether or not a start operation for instructing the start of parking assistance has been received (step S1). If the start operation has not been received (step S1/NO), the parking assistance device 100 does not start processing until the start operation is received.

When the parking assistance device 100 receives a start operation (step S1/YES), it acquires detection information from the sensor unit 20. Here, it acquires an image captured by the image capture unit 30 as the detection information (step S2).

When the parking assistance device 100 acquires the captured image, it converts the acquired captured image into an overhead image using the conversion parameters stored in the memory 110 (step S3). Next, the parking assistance device 100 executes a process of detecting the lane markings on the overhead image or the captured image before it is converted into an overhead image (step S4).

When the parking assistance device 100 is unable to detect a partition line by the detection process (step S5/NO), the parking assistance device 100 generates a display image by superimposing a vehicle icon 5 at a position on the overhead image corresponding to the current position of the vehicle 1A (step S6). When the parking assistance device 100 is able to detect a candidate parking frame line by the detection process (step S4/YES), the parking assistance device 100 generates a display image by superimposing a vehicle icon 5 and a candidate frame 200 indicating the position of the detected candidate parking frame line on the overhead image (step S7). The vehicle icon 5 is superimposed at a position on the overhead image corresponding to the current position of the vehicle 1A.

Next, the parking assistance device 100 displays the generated display image on the touch panel 65 together with a guidance display (step S8). For example, the guidance display may display a message such as "Please move the vehicle icon 5 to the parking position where you want to park your vehicle 1A."

Next, the parking assistance device 100 judges whether or not operation information of a touch operation on the touch panel 65 has been input (step S9). The operation information is, for example, information indicating the position (coordinates) of the touch panel 65 touched by the touch operation. If no operation information has been input (step S9/NO), the parking assistance device 100 proceeds to the judgment of step S13.

When operation information is input (step S9/YES), the parking assistance device 100 changes the display position of the vehicle icon 5 in the display image based on the position signal included in the input operation information (step S10). Next, the parking assistance device 100 acquires position information indicating the changed position of the vehicle icon 5 (step S11). This position information may be a position on the overhead image, latitude and longitude, distance and direction information from the vehicle 1A, or the amount of movement from the previous position.

Next, the parking assistance device 100 associates the acquired position information with the orientation of the vehicle icon 5 at this position and stores it in the memory 110 (step S12). The parking assistance device 100 additionally registers the position information as part of the trajectory data stored in the memory 110.

Next, the parking assistance device 100 judges whether or not a confirmation operation for confirming the operation has been received (step S13). A confirmation operation is an operation indicating that the movement of the vehicle icon 5 to the parking position has been completed. If the parking assistance device 100 has not received a confirmation operation (step S13/NO), it returns to the judgment of step S9.

When the parking assistance device 100 receives a confirmation operation (step S13/YES), the parking assistance device 100 determines the parking position and parking direction of the vehicle 1A based on the position and orientation of the vehicle icon 5 when the confirmation operation is received (step S14). The parking assistance device 100 determines the actual position corresponding to the position of the vehicle icon 5 on the overhead image when the confirmation operation is input as the parking position. The parking assistance device 100 also determines the parking direction based on the orientation of the vehicle icon 5 in the parking position and the trajectory data.

Next, the parking assistance device 100 generates a parking route for moving the vehicle 1A to a parking position based on the generated trajectory data and the situation around the vehicle 1A detected by the situation detection unit 132 (step S15). The route generation unit 137 generates a route that follows the trajectory data and that has the shortest travel distance for the vehicle 1A. In addition, if an obstacle is detected by the situation detection unit 132 along the generated route, the route generation unit 137 corrects the parking route based on the position of the detected obstacle.

Next, the parking assistance device 100 generates control information based on the generated parking path (step S16). The control information is information for driving the host vehicle 1A along the set parking path, and includes information for controlling steering, driving, braking, and gear shifting.
When the parking assistance device 100 generates the control information, it outputs the generated control information to the vehicle control unit 70. The vehicle control unit 70 controls the driving of the steering device 71, the driving device 73, the braking device 75, and the transmission device 77 in accordance with the control information input from the parking assistance device 100. As a result, the host vehicle 1 starts traveling along the parking path set by the parking assistance device 100, and parks in the parking position set by the operator's operation, in the parking direction set by the operator's operation.

As described above, the parking assistance device 100 of this embodiment displays the situation around the vehicle 1A detected by the situation detection unit 132 and the vehicle icon 5 on the display unit 60, and accepts input of the position of the vehicle icon 5 and the orientation of the vehicle icon 5. In the parking assistance device 100, the reception unit 134 determines the parking position for parking the vehicle 1A and the parking direction when parking the vehicle 1A in the parking position based on the accepted position and orientation of the vehicle icon 5. Furthermore, the parking assistance device 100 generates a parking path for the vehicle 1A based on the determined parking position and parking direction, and the path generation unit 137 generates control information for driving the vehicle 1A according to the generated parking path.
Therefore, a parking position for parking the vehicle 1A and a parking direction for parking the vehicle 1A in the parking position can be determined by inputting the position of the vehicle icon 5 and the orientation of the vehicle icon 5. Therefore, even if the partition lines that divide the parking area cannot be detected from the image captured by the camera, the parking position and parking direction can be specified by an intuitive operation, and the vehicle 1 can be automatically parked in the specified parking position and parking direction.

In addition, the parking assistance device 100 receives operation information of a touch operation on the touch panel 65 of the display unit 60.
The display control unit 135 changes the position of the vehicle icon 5 in accordance with operation information of a touch operation performed on the vehicle icon 5 .
The determination unit 136 determines the parking direction based on the trajectory of the position of the vehicle icon 5 that has been changed in accordance with the operation information.
Therefore, the parking direction of the vehicle 1A can be specified by touching the vehicle icon 5 displayed on the touch panel 65, and the parking position and parking direction can be set by intuitive operations.

In addition, the route generating unit 137 generates a parking route based on the trajectory of the position of the vehicle icon 5 that has been changed in accordance with the operation information.
Therefore, a parking route for the vehicle 1A can be set by a touch operation on the vehicle icon 5 displayed on the touch panel 65.

In addition, the route generating unit 137 generates a parking route based on the trajectory of the position of the vehicle icon 5 changed according to the operation information and the situation around the host vehicle 1A detected by the situation detecting unit.
Therefore, a parking path can be generated that reflects the situation around the vehicle 1A detected by the situation detection unit, making it possible to avoid collision between the vehicle 1A and an obstacle.

In addition, the vehicle icon 5 is a vehicle icon 5 having the shape of the host vehicle 1A, and the determination unit 136 determines the parking direction based on the orientation of the vehicle icon 5.
Therefore, the parking direction can be set by an intuitive operation.

In addition, based on the operation information received by the receiving unit 134, the determination unit 136 detects a first touch operation on the graphical icon 7 and a second touch operation touched at a position on the graphical icon 7 different from the first touch operation.
The determination unit 136 determines the parking direction based on the positions of the graphic icon 7 touched by the first touch operation and the second touch operation.
Therefore, even if the icon is not the vehicle icon 5 having the shape of a vehicle, the parking direction can be specified by operating the icon.

When a long press operation on the vehicle icon 5 is detected from the operation information, the display control unit 135 displays a rotation operator 201 for rotating the vehicle icon 5 , and changes the orientation of the vehicle icon 5 according to the operation information for the rotation operator 201 .
Therefore, it becomes possible to change the orientation of the vehicle icon 5 by a touch operation, thereby improving operability.

In addition, the situation detection unit 132 detects the partition lines that separate the parking area from images captured by a camera mounted on the vehicle 1A.
The display control unit 135 causes the touch panel 65 to display, as a candidate frame 200, a parking area candidate that includes the demarcation line detected by the situation detection unit 132.
Therefore, it is possible to simplify the designation of a parking position by operating the vehicle icon 5, and to prevent the occurrence of a case where an incorrect position where parking is not possible is designated as a parking position.

The above-described embodiment is merely an example of one aspect of the present invention, and any modifications and applications are possible without departing from the spirit and scope of the present invention.
For example, the blocks shown in Figure 1 are schematic diagrams in which components are classified according to the main processing contents in order to facilitate understanding of the present invention, and the components can be further classified into more components according to the processing contents, or each component can be classified to perform more processes.

Also, in FIG. 1, the parking assistance device 100 may be configured to include at least one of the position detection unit 10 and the sensor unit 20 as an integrated unit.

When the parking assistance method of the present invention is implemented using a computer, the program executed by the computer can be configured in the form of a recording medium or a transmission medium for transmitting the program. The recording medium can be a magnetic or optical recording medium or a semiconductor memory device. Specific examples include portable or fixed recording media such as a flexible disk, HDD (Hard Disk Drive), CD-ROM (Compact Disk Read Only Memory), DVD, Blu-ray (registered trademark) Disc, magneto-optical disk, flash memory, and card-type recording medium. The recording medium may also be a non-volatile storage device such as a ROM or HDD provided in the parking assistance device 100.

For example, the processing units in the flowchart shown in FIG. 8 are divided according to the main processing content in order to make the processing of the parking assistance device 100 easier to understand, and the present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing of the parking assistance device 100 may be divided into even more processing units according to the processing content. Furthermore, the processing of the parking assistance device 100 may be divided so that one processing unit includes even more processes.

REFERENCE SIGNS LIST 1 Vehicle 1A Vehicle 1B Other vehicle 3 In-vehicle device 5, 5A, 5B, 5C Vehicle icon 10 Position detection unit 20 Sensor section 30 Photography section 31 Front camera 32 Rear camera 33 Left side camera 34 Right side camera 40 Sonar unit 50 Operation section 65 Touch panel 70 Vehicle control unit 71 Steering device 73 Drive device 75 Braking device 77 Transmission device 100 Parking assistance device 110 Memory 130 Processor 131 Position acquisition section 132 Situation detection section 133 Overhead image generation section 134 Reception section 136 Decision section 137 Route generation section 138 Control information generation section 200 Candidate frame 201 Rotation operator 202 Rotation operator

Claims (8)

A situation detection unit that detects a situation around the vehicle;
a display control unit that causes a display unit to display the surrounding situation of the vehicle detected by the situation detection unit and an icon;
a reception unit that receives touch operations on a touch panel included in the display unit, the touch operations including a first touch operation of touching the icon, a second touch operation of touching a position on the icon different from the touch position of the first touch operation, and a movement operation of moving the icon to a parking position where the vehicle is desired to be parked while keeping the icon touched by the first touch operation and the second touch operation ;
a determination unit that determines a parking position of the vehicle based on a position of the icon after the icon is moved by the moving operation, and determines a parking direction based on a positional relationship between the first touch operation and the second touch operation in a direction in which the icon is moved by the moving operation ;
A route generating unit that generates a parking route for the vehicle based on the parking position and the parking direction determined by the determination unit;
a control information generating unit that generates control information for causing the vehicle to travel along the parking path generated by the path generating unit;
A parking assistance device comprising: The reception unit outputs operation information corresponding to the received touch operation to the path generation unit;
2. The parking assistance device according to claim 1 , wherein the route generation unit acquires a trajectory of the positions of the icon based on the operation information, and generates a parking route based on the acquired trajectory of the positions of the icon. 3. The parking assistance device according to claim 2, wherein the route generation unit acquires a trajectory of the positions of the icon based on the operation information, and generates the parking route based on the acquired trajectory of the positions of the icon and a situation around the vehicle detected by the situation detection unit. 4. The parking assistance device according to claim 2, wherein when the display control unit detects a long press operation of the icon based on the operation information, the display control unit displays a rotation operator for rotating the icon, and changes an orientation of the icon in accordance with the operation information for the rotation operator. The situation detection unit detects a parking area dividing line from an image captured by a camera mounted on the vehicle,
5. The parking assistance device according to claim 1 , wherein the display control unit causes the touch panel to display a candidate frame indicating a parking area specified by the detected demarcation line. A situation detection unit that detects a situation around the vehicle;
a display control unit that causes a display unit to display the surrounding situation of the vehicle detected by the situation detection unit and an icon;
A receiving unit that receives an input of a position of the icon and a direction of the icon;
A determination unit that determines a parking position at which the vehicle is to be parked and a parking direction when the vehicle is to be parked at the parking position based on the position of the icon and the orientation of the icon received by the reception unit;
A route generating unit that generates a parking route for the vehicle based on the parking position and the parking direction determined by the determination unit;
a control information generating unit that generates control information for causing the vehicle to travel along the parking path generated by the path generating unit,
the reception unit receives operation information of a touch operation on a touch panel included in the display unit;
The display control unit changes a position of the icon according to operation information of a touch operation performed on the icon;
The determination unit determines a parking direction based on a trajectory of a position of the icon changed in accordance with the operation information,
When a long press operation of the icon is detected from the operation information, the display control unit displays a rotation operator for rotating the icon, and changes an orientation of the icon in accordance with the operation information for the rotation operator.
A parking assistance device comprising: A situation detection unit that detects a situation around the vehicle;
a display control unit that causes a display unit to display the surrounding situation of the vehicle detected by the situation detection unit and an icon;
A receiving unit that receives an input of a position of the icon and a direction of the icon;
A determination unit that determines a parking position at which the vehicle is to be parked and a parking direction when the vehicle is to be parked at the parking position based on the position of the icon and the orientation of the icon received by the reception unit;
A route generating unit that generates a parking route for the vehicle based on the parking position and the parking direction determined by the determination unit;
a control information generating unit that generates control information for causing the vehicle to travel along the parking path generated by the path generating unit,
the reception unit receives operation information of a touch operation on a touch panel included in the display unit;
The display control unit changes a position of the icon according to operation information of a touch operation performed on the icon;
The determination unit determines a parking direction based on a trajectory of a position of the icon changed in accordance with the operation information,
The situation detection unit detects a parking area dividing line from an image captured by a camera mounted on the vehicle,
The display control unit causes the touch panel to display a candidate frame indicating a parking area specified by the detected demarcation line.
A parking assistance device characterized by: A processor of a parking assistance device mounted on a vehicle,
Detecting a situation around the vehicle;
displaying the detected surroundings of the vehicle and an icon on a display unit ;
receiving touch operations on a touch panel included in the display unit, the touch operations including a first touch operation of touching the icon, a second touch operation of touching a position on the icon different from the touch position of the first touch operation, and a moving operation of moving the icon to a parking position where the vehicle is desired to be parked while keeping the icon touched by the first touch operation and the second touch operation ;
determining a parking position of the vehicle based on a position of the icon after the icon is moved by the moving operation, and determining a parking direction based on a positional relationship between the first touch operation and the second touch operation in a direction in which the icon is moved by the moving operation;
generating a parking path for the vehicle based on the determined parking position and the determined parking direction;
generating control information for driving the vehicle according to the generated parking path;
A parking assistance method comprising:

Images