JP6979614B2 - A display control system, a display system with a display control system, a display control method, a program, and a mobile body with a display system. - Google Patents

Description

The present disclosure generally relates to a display control system, a display system including a display control system, a display control method, a program, and a mobile body including the display system. More specifically, the present disclosure relates to a display control system for displaying the state of automatic operation of the moving body on a display device, a display system including a display control system, a display control method, a program, and a moving body including a display system.

Conventionally, a display control device for controlling the display of information provided to an occupant (user) when a vehicle (moving body) can travel in an automatic driving mode is known, and is disclosed in, for example, Patent Document 1. ing. In the display control device described in Patent Document 1, the control unit controls the image generation unit so as to generate a predetermined image to be displayed on the display medium (display device) according to the automatic driving level of the vehicle. The predetermined image has a highlighting area that indicates a specific direction that the driver needs to be aware of when an object is present around the vehicle.

Japanese Unexamined Patent Publication No. 2017-71284

In the display control device described in Patent Document 1, when the state of automatic driving changes during automatic driving of a moving body, a user of the moving body (for example, a driver) changes the driving state (why the state of automatic driving changes). There was a problem that it was difficult to grasp.

The present disclosure is made in view of the above points, and includes a display control system, a display system including a display control system, a display control method, a program, and a display system in which a moving body user can easily grasp the status of automatic driving. The purpose is to provide a moving object.

The display control system according to one aspect of the present disclosure includes an acquisition unit and a control unit. The acquisition unit acquires driving information indicating a state of automatic operation that automatically controls the movement of the moving body, and object information about an object around the moving body. The control unit controls the display of the image to be displayed on the display device. The control unit causes the display device to display a specific image when the state of the automatic operation of the moving body changes due to an object around the moving body. The specific image includes a first marker circled arrow indicating the traveling direction of the moving body, and the second marker indicating the state of the automatic operation of the movable body, and a third marker surrounding the object on the image , A fourth marker that links the first marker and the third marker. The fourth marker is a figure that connects the first marker and the third marker.

The display system according to one aspect of the present disclosure includes the above-mentioned display control system and a display device. The display device displays an image around the moving body and the image.

The display control method according to one aspect of the present disclosure includes an acquisition step and a control step. The acquisition step is a step of acquiring operation information indicating a state of automatic operation for automatically controlling the movement of a moving body and object information regarding an object around the moving body. The control step is a step of controlling the display of an image to be displayed on the display device. In the control step, when the state of the automatic operation of the moving body is changed due to an object around the moving body, a specific image is displayed on the display device. The specific image includes a first marker circled arrow indicating the traveling direction of the moving body, and the second marker indicating the state of the automatic operation of the movable body, and a third marker surrounding the object on the image , A fourth marker that links the first marker and the third marker. The fourth marker is a figure that connects the first marker and the third marker.

The program according to one aspect of the present disclosure is a program for causing a computer system to execute the above-mentioned display control method.

The mobile body according to one aspect of the present disclosure includes the above display system. The mobile body includes a detection system and an operation control system. The detection system detects an object around the moving body. The operation control system executes the automatic operation of the moving body based on the detection result of the detection system.

The present disclosure has an advantage that the user of the mobile body can easily grasp the situation of automatic driving.

FIG. 1 is a conceptual diagram showing a display control system and a configuration of a display system according to an embodiment of the present disclosure. FIG. 2 is a conceptual diagram showing an internal configuration of an automobile equipped with the same display system. FIG. 3 is a flowchart showing the operation of the same display control system. FIG. 4 is a conceptual diagram showing an example of a specific image displayed on a display device by the same display control system. FIG. 5 is a conceptual diagram showing another example of the specific image displayed on the display device by the display control system of the same.

(1) Configuration Hereinafter, the display control system 1, the display system 10, and the mobile body (here, the automobile 100) according to the embodiment of the present disclosure will be described with reference to FIG. The display control system 1 of the present embodiment includes an acquisition unit 11 and a control unit 12. Further, the display system 10 of the present embodiment includes a display control system 1 and a display device 3. Further, the automobile 100 of the present embodiment includes a display system 10, a detection system 4, and a driving control system 2.

The acquisition unit 11 acquires operation information from the operation control system 2. The driving information is information representing a state of automatic driving that automatically controls the running of the automobile 100. The driving information includes, for example, information such as an accelerator pedal depression amount (accelerator opening degree), a brake pedal depression amount, and a steering control amount. Further, the acquisition unit 11 acquires object information regarding the object B1 around the automobile 100 from the detection system 4. The object information is information that can be detected by a camera, a sonar sensor, a radar, a LiDAR (Light Detection and Ranging), or the like.

In the present embodiment, the acquisition unit 11 acquires object information from the detection system 4 including the image pickup device 41 and the laser radar 42. The image pickup device 41 takes an image of the space around the automobile 100. In the present embodiment, the image pickup device 41 is an omnidirectional camera. The laser radar 42 measures the distance from the object B1 existing in the space around the automobile 100 to the automobile 100, the properties of the object B1, and the like. Specific examples of the object information include the distance from the vehicle 100 to the vehicle moving around the vehicle 100, the relative coordinates of this vehicle with respect to the vehicle 100, the inter-vehicle distance between a plurality of vehicles, the relative speed of these vehicles, and the like. be. Here, the object B1 around the automobile 100 in the object information includes a vehicle running or stopped around the automobile 100, a structure such as a guardrail, a pedestrian, a small animal, and the like.

The control unit 12 is composed of, for example, a CPU (Central Processing Unit) and a microcomputer having a memory as a main configuration. In other words, the control unit 12 is realized by a computer having a CPU and a memory, and the computer functions as the control unit 12 by executing a program stored in the memory by the CPU. Although the program is recorded in advance in the memory of the control unit 12 here, the program may be provided by being recorded in a telecommunication line such as the Internet or recorded in a recording medium such as a memory card. The same applies to the operation control system 2.

The control unit 12 controls the display of the image 5 to be displayed on the display device 3. In the present embodiment, the control unit 12 causes the display device 3 (first display 31 (described later)) to display an image 5 showing the state of automatic driving of the automobile 100, except in a specific case. As shown in FIG. 2, the image 5 is displayed in the lower center region of the first display 31 as a figure having a width along the traveling direction of the automobile 100. The image 5 is displayed as a figure in which an arrow indicating the traveling direction of the automobile 100 is surrounded by a circle.

Further, the control unit 12 causes the display device 3 (first display 31) to display the specific image 6 in a specific case. The specific case is a case where the state of automatic driving of the automobile 100 changes due to the object B1 around the automobile 100. That is, even if the state of automatic driving of the automobile 100 changes, it does not correspond to a specific case unless the change is caused by the object B1 around the automobile 100. The specific image 6 is an image in which the object B1 based on the object information acquired by the acquisition unit 11 and the state of automatic driving of the automobile 100 based on the driving information acquired by the acquisition unit 11 are linked. The specific image 6 will be described in detail in "(2) Operation" described later.

The driving control system 2 is a system that automatically controls the traveling of the automobile 100 based on the position information, the vehicle information, and the like, in addition to the object information acquired from the detection system 4. The position information is information based on the position of the automobile 100, and is information that can be detected by using a positioning system such as GPS (Global Positioning System) such as road information at the position of the own vehicle. Specific examples of location information include the number of lanes on the road at the position of the own vehicle, whether it is an intersection, whether it is a junction, whether it is a one-way street, the width of the road, the presence or absence of a sidewalk, the slope, and the curvature of a curve. ..

The vehicle information is information representing the local state of the automobile 100 itself, and is information that can be detected by a sensor mounted on the automobile 100. Specific examples of vehicle information include the moving speed (traveling speed) of the automobile 100, the acceleration applied to the automobile 100, the amount of depression of the accelerator pedal, the amount of depression of the brake pedal, the steering angle, the inclination of the own vehicle, and the seat in the vehicle. There is a tilt etc. In addition, data specific to the automobile 100, such as vehicle width, vehicle height, and total length, are also included in the vehicle information.

In the present embodiment, the automobile 100 is automatically moved to the destination set by the car navigation system by the driving control system 2 without requiring a driving operation (for example, acceleration / deceleration operation, steering operation, etc.) by the user. It is assumed that the vehicle runs in a targeted manner. The "user" here includes not only the driver of the automobile 100 but also the occupants other than the driver who is on board the automobile 100.

The display device 3 displays the image around the automobile 100 and the image 5 acquired from the detection system 4. The display device 3 is provided at a position inside the automobile 100 so that the user can easily see it. In this embodiment, the display device 3 is arranged in the dashboard 101. Further, in the present embodiment, the display device 3 is composed of a plurality of (here, four) displays 31 to 34. Each of the plurality of displays 31 to 34 is, for example, a liquid crystal display. Hereinafter, the display 31 is referred to as a “first display 31”, the display 32 is referred to as a “second display 32”, the display 33 is referred to as a “third display 33”, and the display 34 is referred to as a “fourth display 34”.

The first display 31 displays an image of the space in front of the automobile 100 among the images captured by the image pickup device 41 of the detection system 4. Further, as described above, the image 5 or the specific image 6 is also displayed on the first display 31. The image 5 or the specific image 6 is displayed so as to be superimposed on the image of the space in front of the automobile 100.

The second display 32 displays an image of the space on the left side of the automobile 100 among the images captured by the image pickup device 41 of the detection system 4. The image of the space to the right of the automobile 100 among the images captured by the image pickup device 41 of the detection system 4 is displayed on the third display 33. In the present embodiment, the first displays 31 to the third displays 33 are arranged side by side in one direction (left-right direction in FIG. 2). Then, on the first display 31 to the third display 33, an image in a range of 180 degrees forward when viewed from the center of the automobile 100 is displayed.

The fourth display 34 is arranged below the first display 31. The fourth display 34 displays an image image representing a range of images displayed on the first display 31 to the third display 33. Specifically, the first icon A1 and the circular second icon A2 surrounding the first icon A1 are displayed on the fourth display 34. The first icon A1 is a figure representing a state in which the automobile 100 is viewed from above. The second icon A2 represents a range of images displayed on the first display 31 to the third display 33. Here, the upper semicircular ring area of the second icon A2 is highlighted.

(2) Operation Hereinafter, an example of the operation of the display control system 1 of the present embodiment will be described with reference to FIG. The process from "START" to "END" in FIG. 3 is repeated according to the acquisition cycle (for example, 30 times per second) by the acquisition unit 11. First, the control unit 12 acquires the operation information and the object information from the acquisition unit 11 (step S101). Then, the control unit 12 determines whether or not the state of automatic driving of the automobile 100 has changed by comparing the acquired driving information with the previously acquired driving information (step S102).

For example, as shown in FIG. 4, when the driving control system 2 detects a pedestrian (object B1) in a direction crossing the road by the detection system 4, the road is a green light and the automobile 100 can move in the traveling direction. Also performs brake control. In this case, the driving state of the automobile 100 has changed from the state in which the brake control is not executed to the state in which the brake control is executed.

If the state of automatic driving of the automobile 100 has not changed (step S102: No), the control unit 12 does not perform the subsequent processing and executes step S101 again. On the other hand, when the state of automatic driving of the automobile 100 changes (step S102: Yes), the control unit 12 determines the object B1 based on the acquired object information, the state of automatic driving of the automobile 100 based on the acquired driving information, and the state of automatic driving of the automobile 100. Is generated (step S103). Then, the control unit 12 displays the generated specific image 6 on the first display 31 in a form of superimposing it on the image displayed on the first display 31 (step S104).

As shown in FIG. 4, the specific image 6 has a first marker 61, a second marker 62, a third marker 63, and a fourth marker 64. Similar to the image 5, the first marker 61 is an image having a width along the traveling direction of the automobile 100, and is represented by a figure in which an arrow indicating the traveling direction of the automobile 100 is surrounded by a circle. However, the color of the first marker 61 is different from the color of the image 5 before the automatic driving state of the automobile 100 changes (for example, before the brake control is executed). For example, in the control unit 12, the color of the image 5 when the operation control system 2 is not executing the brake control is green, and the color of the specific image 6 when the operation control system 2 is executing the brake control is yellow. There may be. Further, for example, the color of the specific image 6 when the automobile 100 is stopped by the brake control of the driving control system 2 may be red. That is, the colors of the image 5 and the specific image 6 change according to the state of automatic driving of the automobile 100.

The second marker 62 is an image formed on a virtual surface substantially perpendicular to the road, and is represented by a figure including a character string indicating a state of automatic driving of the automobile 100. The second marker 62 is displayed above the first marker 61. Here, since the brake control is executed by the operation control system 2, the second marker 62 includes the character string "BRAKE".

Like the second marker 62, the third marker 63 is an image formed on a virtual surface substantially perpendicular to the road, and is represented by a frame-shaped figure surrounding the object B1 on the image. Here, when there are a plurality of pedestrians (object B1) around the automobile 100, the control unit 12 sets the third marker 63 only to the pedestrians that have caused the change in the automatic driving state of the automobile 100. Display.

Like the first marker 61, the fourth marker 64 is an image having a width along the traveling direction of the automobile 100, and is represented by an arrow-shaped figure connecting the first marker 61 and the third marker 63. There is. That is, the fourth marker 64 links the third marker 63, which represents the object B1, with the first marker 61, which represents the state of automatic driving of the automobile 100.

Therefore, the user sees the specific image 6 displayed on the display device 3 (here, the first display 31), and the automatic driving state of the automobile 100 changes due to the pedestrian (object B1). Is possible to perceive. In other words, the user can easily grasp the state of automatic driving of the automobile 100 by looking at the specific image 6.

(Modification example)
The above embodiment is only one of the various embodiments of the present disclosure. The above embodiment can be variously modified according to the design and the like as long as the object of the present disclosure can be achieved. Further, the same function as that of the display control system 1 may be realized by a display control method, a computer program, a recording medium on which the program is recorded, or the like.

The display control method according to one aspect includes an acquisition step and a control step. The acquisition step is a step of acquiring driving information indicating an automatic driving state for automatically controlling the movement of the moving body (vehicle 100) and object information regarding the object B1 around the moving body. The control step is a step of controlling the display of the image 5 to be displayed on the display device 3. In the control step, when the state of the automatic operation of the moving body changes due to the object B1 around the moving body, the specific image 6 is displayed on the display device 3. The specific image 6 is an image in which the object B1 based on the object information and the state of the automatic operation of the moving body based on the driving information are linked.

Further, the program according to one aspect is a program for causing a computer system to execute the above-mentioned display control method.

The execution subject of the display control system 1 or the display control method in the present disclosure includes a computer system. The computer system mainly consists of a processor and a memory as hardware. By the processor executing the program recorded in the memory of the computer system, the function as the execution subject of the system or the method in the present disclosure is realized. The program may be pre-recorded in the memory of the computer system, but may be provided through a telecommunication line. Further, the program may be recorded and provided on a non-temporary recording medium such as a memory card, an optical disk, or a hard disk drive that can be read by a computer system. The processor of a computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large scale integrated circuit (LSI). A plurality of electronic circuits may be integrated on one chip, or may be distributed on a plurality of chips. A plurality of chips may be integrated in one device, or may be distributed in a plurality of devices.

Further, the functions of the control unit 12 of the display control system 1 may be distributed to a plurality of systems (devices). At least a part of the functions of the control unit 12 may be realized by, for example, the cloud (cloud computing).

Further, the display control system 1 directly communicates between vehicles (between vehicles) or between vehicles and infrastructure such as traffic lights and road signs (between roads), so-called V2X (between vehicles). Vehicle to Everything) communication technology may be used. According to the communication technology of V2X, for example, the vehicle 100 can acquire object information from a surrounding vehicle or infrastructure. Further, the contents of the image 5 and the specific image 6 to be displayed on the display device 3 may be determined by the infrastructure. In this case, at least a part of the control unit 12 may not be mounted on the automobile 100.

Further, the moving body to which the display control system 1 is applied is not limited to the automobile 100, but can be applied to moving objects other than the automobile 100, such as a two-wheeled vehicle, a train, an aircraft, a construction machine, and a ship.

In the present embodiment, even when the driving state of the automobile 100 changes due to the object B1 existing in the space to the right or left of the automobile 100, the control unit 12 displays the specific image 6 on the display device 3. You may let me. For example, as shown in FIG. 5, it is assumed that the driving control system 2 executes the brake control due to a pedestrian (object B1) existing in the space on the left side of the automobile 100. In this case, the control unit 12 causes the first marker 61 and the second marker 62 of the specific image 6 to be displayed on the first display 31, and causes the pedestrian displayed on the second display 32 to display the third marker 63. .. Further, the control unit 12 displays the fourth marker 64 so as to straddle the first display 31 and the second display 32. Also in this case, the user sees the specific image 6 displayed on the display device 3 (here, the first display 31 and the second display 32), so that the user automatically causes the automobile 100 to be caused by the pedestrian (object B1). It is possible to perceive that the driving condition has changed.

In the present embodiment, the control unit 12 causes the pedestrian (object B1) to display the third marker 63 only when the specific image 6 is displayed on the display device 3, but the present invention is not limited to this. For example, the control unit 12 may display the third marker 63 on one or more objects B1 existing in the vicinity of the automobile 100 even when the image 5 is displayed without displaying the specific image 6. In this case, since the one or more objects B1 do not cause a change in the automatic driving state of the automobile 100, the fourth marker 64 is not displayed on the display device 3.

In the present embodiment, the specific image 6 shown in FIGS. 4 and 5 is an example, and the first markers 61 to the fourth markers 64 may be represented by figures other than the figures shown in FIGS. 4 and 5. For example, the fourth marker 64 may be represented by a figure other than an arrow, such as a straight line.

In the present embodiment, the first display 31 may be configured by, for example, a touch panel display. In this case, the display device 3 is configured to change the range of the image displayed on the first display 31 to the third display 33 according to the swipe operation performed by the user on the first display 31. May be good.

In the present embodiment, the displays 31 to 34 may be composed of an organic EL (electroluminescence) display in addition to the liquid crystal display. Further, in the present embodiment, the display device 3 is composed of four displays 31 to 34, but the present invention is not limited to this. For example, the display device 3 may be composed of three or less displays, or may be composed of five or more displays. Further, the display device 3 may be configured by, for example, a head-up display (HUD: Head-Up Display).

In the present embodiment, the automobile 100 may be provided with another display device capable of displaying an image behind the automobile 100. Examples of other display devices include an inner rear-view mirror, an electronic mirror used in an automobile 100 instead of an outer rear-view mirror, and the like. In this case, even when the driving state of the automobile 100 changes due to the object B1 existing in the space behind the automobile 100, the control unit 12 causes the display device 3 and other display devices to display the specific image 6. It is possible.

(summary)
As described above, the display control system (1) according to the first aspect includes an acquisition unit (11) and a control unit (12). The acquisition unit (11) acquires driving information indicating the state of automatic driving that automatically controls the movement of the moving body (vehicle (100)) and object information regarding an object (B1) around the moving body. The control unit (12) controls the display of the image (5) to be displayed on the display device (3). The control unit (12) causes the display device (3) to display the specific image (6) when the state of the automatic operation of the moving body changes due to the object (B1) around the moving body. The specific image (6) is an image in which an object (B1) based on object information and a state of automatic driving of a moving body based on driving information are linked.

In the display control system (1) according to the second aspect, in the first aspect, the control unit (12) displays an image (5) on the display device (3) before the state of the automatic operation of the moving body changes. The specific image (6) is displayed on the display device (3) in a color different from the color of).

The display system (10) according to the third aspect includes the display control system (1) of the first or second aspect and the display device (3). The display device (3) displays an image and an image (5) around the moving body (automobile (100)).

The display control method according to the fourth aspect includes an acquisition step and a control step. The acquisition step is a step of acquiring driving information indicating a state of automatic driving that automatically controls the movement of a moving body (automobile (100)) and object information related to an object (B1) around the moving body. The control step is a step of controlling the display of the image (5) to be displayed on the display device (3). In the control step, when the state of the automatic operation of the moving body is changed due to the object (B1) around the moving body, the specific image (6) is displayed on the display device (3). The specific image (6) is an image in which an object (B1) based on object information and a state of automatic driving of a moving body based on driving information are linked.

The program according to the fifth aspect is a program for causing a computer system to execute the above-mentioned display control method.

The mobile body (vehicle (100)) according to the sixth aspect includes the display system (10) of the third aspect. The mobile body includes a detection system (4) and an operation control system (2). The detection system (4) detects an object (B1) around the moving body. The operation control system (2) executes automatic operation of the moving body based on the detection result of the detection system (4).

According to the first to sixth aspects, when the state of automatic driving changes during the automatic driving of a moving body, a specific image linking the object that caused the change with the state of automatic driving ( 6) is displayed on the display device (3). For this reason, the user of the moving body compares with the case of simply viewing the image around the moving body at the time when the state of automatic driving changes by looking at the specific image (6) displayed on the display device (3). Then, it is easy to understand why the operating condition has changed. That is, according to these aspects, there is an advantage that the user of the mobile body can easily grasp the situation of automatic driving.

The configuration according to the second aspect is not an essential configuration for the display control system (1) and can be omitted as appropriate.

1 Display control system 10 Display system 11 Acquisition unit 12 Control unit 2 Operation control system 3 Display device 4 Detection system 5 Image 6 Specific image 100 Automobile (moving body)
B1 object

Claims (6)

An acquisition unit that acquires driving information indicating the state of automatic driving that automatically controls the movement of a moving body, and object information about an object around the moving body.
It is equipped with a control unit that controls the display of the image to be displayed on the display device.
The control unit includes a first marker circled with an arrow indicating the traveling direction of the moving body when the state of the automatic operation of the moving body changes due to an object around the moving body, and the control unit. a second marker indicating the state of the automatic operation of the moving body, and a third marker surrounding the object on the image, and the fourth marker attaching strap and said third marker and said first marker, a specific image including the is displayed on the display device,
The fourth marker is a display control system having a figure connecting the first marker and the third marker. The control unit displays the specific image in a color different from the color of the image showing the state of the automatic operation of the moving body to be displayed on the display device before the state of the automatic operation of the moving body changes. The display control system according to claim 1, which is displayed on a display device. The display control system according to claim 1 or 2,
A display system including an image around the moving body and a display device for displaying the image. The operation information indicating the state of the automatic operation that automatically controls the movement of the moving body, the acquisition step of acquiring the object information about the object around the moving body, and the acquisition step.
It has a control step that controls the display of the image to be displayed on the display device.
In the control step, when the state of the automatic operation of the moving body is changed due to an object around the moving body, the first marker circled with an arrow indicating the traveling direction of the moving body and the said. a second marker indicating the state of the automatic operation of the moving body, and a third marker surrounding the object on the image, and the fourth marker attaching strap and said third marker and said first marker, a specific image including the is displayed on the display device,
The fourth marker is a display control method in which the first marker and the third marker are connected to each other. For computer systems
A program for executing the display control method according to claim 4. A mobile body provided with the display system according to claim 3.
A detection system that detects objects around the moving object,
A mobile body including an operation control system that executes the automatic operation of the mobile body based on the detection result of the detection system.

Images