CN113596103A

CN113596103A - Information processing method and related device for vehicle, vehicle and storage medium

Info

Publication number: CN113596103A
Application number: CN202110758779.3A
Authority: CN
Inventors: 刘刚; 王子彬; 李炳泽; 朱赟
Original assignee: Zhejiang Shangtang Technology Development Co Ltd
Current assignee: Zhejiang Shangtang Technology Development Co Ltd
Priority date: 2021-07-05
Filing date: 2021-07-05
Publication date: 2021-11-02

Abstract

The application discloses an information processing method of a vehicle, a related device, the vehicle and a storage medium, wherein the information processing method of the vehicle comprises the following steps: acquiring the current position of the vehicle; determining a corresponding track point of the current position on a preset route; and executing preset processing matched with the position condition of the track point on the preset route by utilizing the vehicle-mounted component of the vehicle. By means of the scheme, intelligent processing in the vehicle moving process is achieved, and user experience is improved.

Description

Information processing method and related device for vehicle, vehicle and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a vehicle information processing method, a related apparatus, a vehicle, and a storage medium.

Background

At present, in each scenic spot, the only function of the guide vehicle is manned, and other functions except the manned cannot be realized, for example, functions of explaining information of each scenic spot and the like. When the tourist is on the bus, the tourist can only know the relevant conditions of the scenic spots through internet searching and other modes, or manually explain through tour guide. It is very inconvenient to search through internet or manually explain tour guides.

Disclosure of Invention

The application at least provides an information processing method of a vehicle, a related device, the vehicle and a storage medium.

The application provides an information processing method of a vehicle, which comprises the following steps: acquiring the current position of the vehicle; determining a corresponding track point of the current position on a preset route; and executing preset processing matched with the position condition of the track point on the preset route by utilizing the vehicle-mounted component of the vehicle.

Therefore, through presetting the corresponding preset treatment of each position on the line in advance, follow-up alright carry out the corresponding preset treatment according to the current position of vehicle, realize the intelligent processing of vehicle removal in-process, and then improve personnel's in the vehicle experience, bring certain convenience.

Wherein, the preset route consists of a plurality of line segments; determining the corresponding track point of the current position on the preset route, including: determining a route matched with the current position in the preset routes as a target line segment; and determining track points by using the position relation between the current position and the target line segment.

Therefore, the target line segment matched with the current position is determined, and the track point is determined according to the position relation between the current position and the target line segment, so that the obtained track point is determined more accurately.

Wherein, confirm in the preset route with the route of current position matching, include: finding out line segments meeting a first condition between a first vector and a second vector from the plurality of line segments, wherein the first vector consists of a starting point to a current position of the line segment, and the second vector consists of a starting point to an end point of the line segment; or finding out the line segments with the distance from the current position meeting the second condition from the plurality of line segments.

Therefore, because the vectors have not only magnitude but also direction, the matching route is determined by the relationship between the vectors, so that the determined route is more accurate. In addition, the matched route is determined according to the distance between the current position, the reference factor is less, and the time consumption of the whole process is less.

The step of finding out the line segment between the first vector and the second vector, which satisfies the first condition, from the plurality of line segments is performed in response to the length of the first vector being less than or equal to a preset length.

Therefore, the accuracy of the subsequent matching result can be improved by judging the size of the first vector before determining the line segment matched with the current position.

The first condition is that the length of the first vector is smaller than that of the second vector, and an included angle between the first vector and the second vector is smaller than a preset angle.

Therefore, the accuracy of the matching result can be improved by comprehensively determining the line segment matched with the vector from the length of the vector and the included angle relationship between the vector and the vector.

The second condition is that the distance between the multi-segment line segment and the current position is the shortest.

Therefore, by selecting the line segment closest to the current position as the line segment to be matched with, the matching accuracy can be improved.

Wherein, utilize the position relation between current position and the target line segment, confirm the track point, include: and taking the intersection point of the perpendicular line from the current position to the target line segment and the target line segment as a track point.

Therefore, since the shortest distance from a point to a line is a perpendicular line segment, it is more accurate to serve as a track point by the intersection of the perpendicular line from the current position to the target line segment and the target line segment.

Before determining the corresponding track point of the current position on the preset route, the method further comprises the following steps: acquiring a plurality of frames of images obtained by shooting the environment where the planned route is located and positioning information respectively corresponding to the plurality of frames of images; obtaining position information of a plurality of inflection points on the planned route based on the multi-frame image and the positioning information, and connecting the inflection points to form a preset route by using the position information of the inflection points; or transmitting the multi-frame image and the positioning information to the local equipment, and acquiring a preset route from the local equipment.

Therefore, the route formed by the line segments is obtained by determining the inflection points and then connecting the inflection points, and compared with the route established by other methods, the route established by the method has higher speed and is simpler. In addition, the resource consumption of the vehicle can be reduced by transmitting the multi-frame image and the positioning information to the local equipment and then receiving the preset route from the local equipment.

The multi-frame images are obtained by shooting through a shooting component of the vehicle or a shooting device in communication connection with the vehicle.

Therefore, the environment image can be acquired in various modes, and the flexibility of equipment selection in the image acquisition process is improved.

The method for obtaining the position information of the inflection points on the planned route based on the multi-frame images and the positioning information comprises the following steps: analyzing the multi-frame images to determine an image related to the inflection point, or displaying the multi-frame images and determining the image related to the inflection point in response to a selection of a user; and obtaining the position information of the inflection point based on the positioning information of the image related to the inflection point.

Therefore, the position information of each inflection point can be obtained by analyzing the multi-frame images, or the inflection point information can be determined by receiving the selection of the user, so that the resource consumption of the execution equipment can be reduced.

Wherein, utilize the on-vehicle subassembly of vehicle, carry out the preset processing of the position condition matching on presetting the route with the track point, include: detecting whether the track point is located in a preset interval of a preset station in a preset route, wherein the preset interval comprises at least one of an inbound interval and an outbound interval; and responding to the track point located in the preset interval of the preset station, and executing preset processing matched with the preset interval of the preset station by using the vehicle-mounted assembly.

Therefore, when the track points are judged to be in the preset interval, corresponding preset processing can be executed, and the whole process is convenient and fast.

The preset processing comprises the step of carrying out augmented reality processing on preset virtual information of the environment object matched with the position condition, wherein the preset virtual information is from target map data; before performing a preset process matching the position of the track point on the preset route by using the vehicle-mounted component of the vehicle, the method further comprises: acquiring initial map data, wherein the initial map data is constructed on the basis of a plurality of frames of images which are obtained by shooting the environment where the planned route is located in advance and positioning information respectively corresponding to the plurality of frames of images; determining setting information of at least one environmental object in the initial map data, wherein the setting information comprises preset virtual information; and associating the setting information of at least one environment object to the initial map data to obtain target map data.

Therefore, the setting information of the environment object is associated to the initial map, so that the corresponding virtual information can be acquired according to the position of the vehicle.

The application provides an information processing apparatus of a vehicle, including: the first positioning module is used for acquiring the current position of the vehicle; the second positioning module is used for determining a corresponding track point of the current position on a preset route; and the processing module is used for executing preset processing matched with the position condition of the track point on the preset route by utilizing the vehicle-mounted component of the vehicle.

The application provides an on-board device applied to a vehicle, which comprises a memory and a processor, wherein the processor is used for executing program instructions stored in the memory so as to realize the information processing method of the vehicle.

The system further comprises a vehicle-mounted component used for carrying out augmented reality processing on the preset virtual information.

The present application provides a vehicle, comprising: the vehicle theme and the vehicle-mounted device who locates the vehicle main part, wherein, the vehicle-mounted device is above-mentioned vehicle-mounted device.

The present application provides a computer-readable storage medium having stored thereon program instructions that, when executed by a processor, implement the information processing method of the above-described vehicle.

According to the scheme, the corresponding preset processing of each position on the line is preset, and the corresponding preset processing can be executed subsequently according to the current position of the vehicle, so that the intelligent processing in the moving process of the vehicle is realized, the experience of personnel in the vehicle is further improved, and certain convenience is brought.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

FIG. 1 is a schematic flow chart diagram illustrating an embodiment of a method for processing information of a vehicle according to the present application;

FIG. 2 is a schematic diagram illustrating a preset route according to an embodiment of the information processing method of the vehicle of the present application;

FIG. 3 is a schematic flowchart illustrating step S12 according to an embodiment of the vehicle information processing method;

FIG. 4 is a schematic structural diagram of an embodiment of an information processing apparatus of a vehicle according to the present application;

FIG. 5 is a schematic structural diagram of an embodiment of an in-vehicle apparatus of the present application applied to a vehicle;

FIG. 6 is a schematic structural diagram of an embodiment of the subject vehicle;

FIG. 7 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application.

Detailed Description

The following describes in detail the embodiments of the present application with reference to the drawings attached hereto.

In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present application.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship. Further, the term "plurality" herein means two or more than two. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an embodiment of an information processing method for a vehicle according to the present application. Specifically, the method may include the steps of:

step S11: the current position of the vehicle is determined.

In the disclosed embodiment, the current position of the vehicle can be determined in various ways, for example, directly by a Global Positioning System (GPS) device or by visual inertial positioning. Of course, if the camera of the vehicle is a depth camera or a three-dimensional (3D) camera, the current position of the vehicle can be directly determined according to the visual positioning. There are various ways to obtain the current position of the vehicle, and the above-mentioned several ways are only examples, and in other embodiments, any other way that can achieve the positioning can be fully adopted.

Step S12: and determining the corresponding track point of the current position on the preset route.

The preset route is a preset vehicle action route. Of course, the vehicle may not travel exactly along the preset route. That is, in some application scenarios, the current position may not be on the preset route. If the preset route may be a line segment, and the actual road often has a certain width, therefore, a certain distance exists between the current position of the vehicle and the preset route, and at this time, the track point of the current position on the preset route needs to be determined. Of course, the most ideal case is that the current position overlaps the preset route, and the current position can be directly used as the corresponding track point.

Step S13: and executing preset processing matched with the position condition of the track point on the preset route by utilizing the vehicle-mounted component of the vehicle.

The onboard components of the vehicle include, but are not limited to, a display, an audio player, and the like. The display may be a real-existing display frequency, or a projector device, etc.

The preset processing corresponding to the position condition of the track point on the preset route can be preset. In some application scenarios, if the vehicle is a guiding vehicle in a scenic spot, the preset processing for matching the position situation on the preset route may be a voice explanation of the scenic spot or a special effect of an environmental object. For example, a display may be used to display a corresponding special effect, and/or an audio player may be used to play corresponding audio, and so on.

Before executing the preset processing matched with the position condition of the track point on the preset route, the following steps can be further executed: initial map data is acquired. The initial map data is constructed based on a plurality of frames of images which are obtained by shooting the environment where the planned route is located in advance and positioning information which corresponds to the plurality of frames of images respectively. The multi-frame image can be shot by a shooting component of the vehicle or shot by a shooting device in communication connection with the vehicle. The multiframe is at least two frames and more. The greater the number of images taken, the more accurate the initial map data is constructed, but the more time it takes to construct the initial map relatively.

The method for obtaining the initial map data may be based on a multi-frame image, or may send the multi-frame image to the local device, and receive the initial map data fed back by the local device. Wherein the initial map data is three-dimensional map data. By constructing the three-dimensional map data, the environmental object is also three-dimensional, so that the three-dimensional preset virtual information about the environmental object can be constructed. The three-dimensional map data can be constructed by sparse three-dimensional point clouds or dense three-dimensional point clouds. Specifically, the manner of constructing the initial map data based on the multi-frame image may be: and extracting and matching feature points of adjacent images, triangularizing each matched feature point based on the pose corresponding to each frame of image to obtain a three-dimensional point corresponding to each matched feature point, and obtaining a three-dimensional point cloud map. Of course, the three-dimensional point cloud map can be further constructed by a structured plane. The so-called structured plane construction may in particular be the composition of three-dimensional points belonging to a plane into a plane. The pose of each frame of image may be acquired by using a high-precision GPS device, which may acquire the position and orientation of the second image. Each frame of image and the corresponding pose can be synchronized in a timestamp mode. Each frame of image and the corresponding pose are taken as offline data, the offline data can be transmitted to other equipment to construct initial map data, and corresponding preset routes and debugging of conditions such as triggering of preset processing are determined. Then, the finally obtained target map data, the preset route, the preset processing and other related contents can be sent to the vehicle so as to carry out final debugging on the vehicle. The initial map data is constructed by acquiring the second image obtained by shooting the vehicle or the shooting device in communication connection with the vehicle, so that the constructed map is more practical. In addition, if the initial map is constructed using the local device, it is possible to reduce resource consumption of the device that executes the information processing method of the vehicle.

And determining setting information of at least one environment object in the initial map data, wherein the setting information of the environment object comprises preset virtual information.

In the embodiment of the present disclosure, the preset virtual information is a virtual three-dimensional model, and is used for Augmented Reality (AR) display. In particular, the virtual three-dimensional model may be edited using development engine software. Wherein, the development engine software can be Unity or UnrealEngine, etc. Specifically, in response to a second preset operation by the user, an editing mode for the initial map data is entered. In the edit mode, setting information of at least one environmental object in the initial map data is determined based on input information of a user. The second preset operation of the user may be a start instruction of a program of the development engine software, or a start instruction of an editing mode, or the like. The initial map data can be edited in real time according to the second preset operation of the user, so that the user can adjust the setting information corresponding to each environment object at any time. By setting the editing mode, the user can set the setting information of the environment object in the editing mode, and the convenience of the user in the using process can be improved.

And associating the setting information of at least one environment object to the initial map data to obtain target map data. The setting information of the environment object is associated to the initial map, so that the corresponding virtual information can be acquired according to the position of the vehicle.

The preset virtual information in the setting information of the environmental object may be associated with the initial map data in a manner that the setting information of the environmental object is superimposed on the corresponding environmental object in the three-dimensional map scene. For example, when the installation information of the environmental object is the building appearance of a certain building, the building appearance of the building may be changed by superimposing the building appearance on the initial map data of the corresponding building, and associating the information with the initial map data. If only one side of the building needs to be changed and the original appearance of the other side is kept, the setting information can be determined only for the side needing to be changed, or the transparent setting information can be set for the original appearance of the other side needing to be kept, and if the setting information is overlapped, the original appearance can be kept.

The setting information of the environment object further comprises interactive virtual information and first preset operation corresponding to the interactive virtual information. The interactive virtual information is used for responding to a first preset operation of a user and replacing preset virtual information as preset virtual information. The interactive virtual information can also be used for responding to a first preset operation of a user and carrying out an enhanced display operation together with the preset virtual information. For example, the interactive virtual information is not the same type of information as the preset virtual information, for example, the preset virtual information may be an appearance of an environment, and the interactive virtual information may be a board, a character model, audio information, and the like that explains information related to the environment. In some application scenarios, the interactive virtual information may be hidden and rendered on the environment object, and is not displayed or played when the first preset operation of the user is not received, and is displayed or played when the first preset operation of the user is received. The interactive virtual information and the first preset operation corresponding to the interactive virtual information are further set in the setting information, so that interaction can be achieved according to the first preset operation of the user in the following process, and interestingness of augmented reality processing is improved.

Before determining the corresponding track point of the current position on the preset route, the following steps can be further executed:

acquiring a plurality of frame images obtained by shooting the environment where the planned route is located and positioning information respectively corresponding to the plurality of frame images. The obtaining manner of the positioning information corresponding to the multiple frames of images is as described above, and is not described herein again.

And obtaining the position information of a plurality of inflection points on the planned route based on the multi-frame image and the positioning information. And connecting a plurality of inflection points to form a preset route by using the position information of the inflection points. The route formed by a plurality of line segments is obtained by determining inflection points and connecting the inflection points, and compared with the route established by other methods, the route established by the method has higher speed and is simpler. Or transmitting the multi-frame image and the positioning information to the local equipment, and acquiring a preset route from the local equipment. The local device may be any device other than the vehicle that acquires the plurality of frames of images. That is, multiple frames of images and corresponding positioning information may generate offline data. By generating the off-line data, the navigation experience on the route can be simulated on the local equipment, the development and the test are convenient, and the debugging and the development on the vehicle are not required all the time. By transmitting the multi-frame image and the positioning information to the local equipment and then receiving the preset route from the local equipment, the resource consumption of the vehicle can be reduced.

The multi-frame images are obtained by shooting through a shooting component of the vehicle or a shooting device in communication connection with the vehicle. The shooting device in communication connection with the vehicle can be an unmanned aerial vehicle, a mobile phone and the like. The multi-frame image used for determining the preset route and the multi-frame image used for constructing the initial map data can be the same or different. But both images are taken in the same environment. Therefore, even if the images used by the two are different, the preset route can be associated with the initial map data in a building manner, namely, the position of the track point on the preset route in the initial map data is known. Therefore, the environment image can be acquired in various modes, and the flexibility of equipment selection in the image acquisition process is improved.

The mode of obtaining the position information of the inflection points on the planned route based on the multi-frame image and the positioning information may specifically be: the multi-frame images are analyzed to determine an image associated with the inflection point. For example, parallax analysis is performed on adjacent frame images, and when the parallax satisfies a preset condition, it is determined that the position of the previously photographed image is an inflection point. Specifically, the determination is made in combination with the first parallax of the adjacent frame images, the traveling speed, and the rotational speed of the apparatus that captured the images. For example, a next image frame is predicted from a previously photographed image frame by a traveling speed and a rotating speed of the apparatus, and a second parallax between the previously photographed image frame and a preset next image frame is acquired. And judging whether the first parallax is larger than the second parallax, and if so, considering the position corresponding to the previously shot image frame as an inflection point.

Alternatively, a plurality of frame images are displayed, and an image associated with an inflection point is determined in response to a selection of a user. For example, the user empirically selects an image corresponding to an inflection point from a plurality of images. And recording the positioning information corresponding to the image selected by the user.

Then, based on the positioning information of the image related to the inflection point, the position information of the inflection point is obtained. The location information of the inflection point may be location information of the inflection point in the target map. Of course, the location information of the image related to the inflection point may be directly used as the location information of the inflection point. The position information of each inflection point can be obtained by analyzing the multi-frame images, or the inflection point information is determined by receiving the selection of a user, so that the resource consumption of the execution equipment can be reduced.

And connecting the inflection points to form a preset route by using the position information of the inflection points. In some application scenes, the offline data are loaded, each frame of image is rendered, and the corresponding pose is updated to the virtual camera. Here, the virtual camera refers to a camera assembly on a virtual vehicle that moves in accordance with the pose in the target map. And the virtual vehicle acquires a corresponding scene according to the pose so as to carry out inflection point analysis.

Wherein each inflection point is recorded in a preset course configuration for subsequent use.

After the preset route is acquired, the site location is further analyzed. The site location refers to a location corresponding to a more important environmental object in the environment. The site location is typically an area for reasons such as environmental footprint. Further, inbound and outbound intervals are determined for the station location. In the embodiment of the present disclosure, the number of inbound intervals and outbound intervals is the same, and each inbound interval and each outbound interval are alternately arranged. For a station, at least one inbound interval and at least one outbound interval are included. And responding to the track point located in the preset interval of the preset station, and executing preset processing matched with the preset interval of the preset station by using the vehicle-mounted assembly. The length of the interval between the inbound interval and the outbound interval may be determined according to specific situations, for example, according to the time consumed by the corresponding preset processing. In some application scenarios, the distance between the inbound interval and the outbound interval at least ensures that the predetermined processing can be performed at a predetermined ratio, which may be eighty percent, but this is merely an example, and in other embodiments, other ratios may be adopted. And the sizes of the single inbound interval and the single outbound interval may also be determined on a case-by-case basis.

The inbound interval is used for starting corresponding operation, and the outbound interval is used for closing corresponding operation. For example, the inbound zone is used to turn on the audio for explanation of the attraction, and the outbound zone is used to turn off the audio for explanation of the attraction.

And recording each inbound interval and outbound interval and the corresponding preset processing in the preset route configuration so as to search the preset route configuration, determine the corresponding preset processing according to the preset route configuration after acquiring the track points of the vehicle on the preset route, and then execute the preset processing matched with the position condition of the track points on the preset route.

The preset route is composed of a plurality of line segments. Wherein, the multiple sections can be two or more sections. Referring to fig. 2, fig. 2 is a schematic diagram illustrating a preset route according to an embodiment of the information processing method for a vehicle of the present application. As shown in fig. 2, the predetermined route is composed of segments connected by four inflection points, which are N0, N1, N2 and N3. The line segments respectively comprise a line segment composed of N0 and N1, a line segment composed of N1 and N2, a line segment composed of N2 and N3, and a line segment composed of N3 and N0.

The inbound interval is the interval composed of the line segment AB, and the outbound interval is the interval composed of the line segment CD. Of course, fig. 2 is only an example, and in a specific application scenario, a plurality of inflection points, an inbound interval, and an outbound interval may be included.

Referring to fig. 3, fig. 3 is a schematic flowchart illustrating step S12 according to an embodiment of the information processing method for a vehicle of the present application. As shown in fig. 3, the step S12 includes the following sub-steps:

step S121: and determining a line segment matched with the current position in the preset route as a target line segment.

Specifically, a segment matched with the current position is found from the multiple segments as a target segment. The method for determining the line segment matched with the current position from the multiple line segments can be various:

1. and finding out line segments between the first vector and the second vector, which meet a first condition, from the plurality of line segments. Wherein the first vector consists of the start of the line segment to the current position. That is, the first vector points from the start of the line segment to the end of the current position. The second vector consists of the start to end point of the line segment. Specifically, the second vector points from the start of a line segment to the end of the same line segment. Wherein the line segment start point and end point may be set in advance. For example, the preset route is set in advance to set a start point and an end point for each line segment in the clockwise order. In other disclosed embodiments, the start and end points of each line segment may be determined based on the direction of travel of the vehicle. Specifically, the angle between the line connecting the starting point to the end point of the line segment and the vehicle traveling direction may be smaller than 90 °. For example, if the line segment is a horizontal line and the direction of travel of the vehicle is from left to right, the end of the line segment on the left sideThe point is the starting point of the line segment, and the end point on the right side of the line segment is the end point of the line segment. Of course, in other embodiments, the angle between the line connecting the start point to the end point of the line segment and the direction of travel of the vehicle is greater than or equal to 90 °. The first condition may be that the length of the first vector is smaller than the length of the second vector, and an included angle between the first vector and the second vector is smaller than a preset angle. Optionally, the starting points of the first and second vectors are the same. As shown in FIG. 2, point P is the current position and the first vector may be

The second vector may be

Because the vectors have both magnitude and direction, the matched route is determined through the relationship between the vectors, so that the determined route is more accurate. In some disclosed embodiments, the step of finding a line segment between the first vector and the second vector satisfying the first condition from the plurality of line segments is performed in response to the length of the first vector being less than or equal to a preset length. For example, if there are 5 first vectors, it is determined whether the lengths of the 5 vectors are less than or equal to a predetermined length, and if only two of the first vectors are less than or equal to the predetermined length, the second vector corresponding to the first vector is retained, and it is determined whether the first condition is satisfied between the first vector and the second vector. Of course, if the lengths of all the first vectors are greater than the preset length, the subsequent steps are not executed. The size of the first vector is judged before the line segment matched with the current position is determined, so that the accuracy of the subsequent matching result can be improved.

2. And finding out line segments with the distance from the current position meeting a second condition from the plurality of line segments. Wherein the second condition may be that the distance from the current position in the multi-segment line segment is the closest. The method for obtaining the line segment closest to the current position may be to draw a circle at the current position to find a tangent line, and use the line segment corresponding to the minimum radius as the target line segment, or to make a perpendicular line from the current position to each line segment, and use the line segment corresponding to the shortest perpendicular line as the target line segment. And determining a matched route according to the distance between the current position and the current position, wherein the reference factor is less, and the time consumption of the whole process is less. And the line segment closest to the current position is selected as the line segment matched with the line segment, so that the matching accuracy can be improved.

Step S122: and determining track points by using the position relation between the current position and the target line segment.

Specifically, the intersection point of the perpendicular line from the current position to the target line segment and the target line segment is used as the track point. Of course, the first vector can also be orthographically projected onto the target line segment, and the projection point of the current position is the track point. Assuming that the target segment is the segment N0N1, as shown in fig. 2, the locus point of the current position P on the target segment is the point P1. The target line segment matched with the current position is determined, and then the track point is determined according to the position relation between the current position and the target line segment, so that the obtained track point is determined more accurately.

And because the shortest distance from a point to a line is a perpendicular line segment, the intersection point of the perpendicular line from the current position to the target line segment and the target line segment is more accurate as a track point.

In some disclosed embodiments, the way of executing the preset processing matched with the position condition of the track point on the preset route by using the vehicle-mounted component of the vehicle may be: and detecting whether the track point is located in a preset interval of a preset station in a preset route. The preset interval comprises at least one of an inbound interval and an outbound interval. And responding to the track point being located in the preset section of the preset station, and executing preset processing matched with the preset section by using the vehicle-mounted assembly. As described above, the preset process matched with the preset section may be stored in the configuration of the preset route, matched using the trajectory point and the configuration of the preset route, determined whether the preset section is located in the preset section, and executed as the corresponding preset process. In the above example, if it is determined that the track point P1 is located in the inbound zone AB, the predetermined process matching the inbound zone AB is executed. When the track points are judged to be in the preset interval, corresponding preset processing can be executed, and the whole process is convenient and fast.

In some disclosed embodiments, the preset processing may be augmented reality processing of preset virtual information of the environmental object matching the location situation. The preset virtual information is from target map data. The environmental object may be an object whose position is relatively fixed in the target map data. Such as buildings, trees, etc. The preset virtual information of the environmental object may be any model constructed for the environmental object in advance, such as a three-dimensional visualization model or a sound effect model.

For the preset virtual information of the same environmental object, the number of the constructed models is not limited, and may be 1, or 2 or more. The target map data may include preset virtual information of a plurality of environmental objects. For example, preset virtual information may be set for all the environmental objects, or preset virtual information may be set for a part of the environmental objects in the target map data.

The mode of performing augmented reality processing on the preset virtual information by using the vehicle-mounted component of the vehicle may be to combine the preset virtual information with an image actually shot by the vehicle to obtain the image after augmented reality processing, and display the image on a display, or play corresponding audio by using an audio player. For example, the preset virtual information is combined with a real environment obtained by shooting the vehicle, and the combination of the preset virtual information and the real environment is displayed on a display of the vehicle.

In some disclosed embodiments, the preset virtual information may be display-type information and/or audio-type information. The display type information comprises at least one of a display special effect, prompt information and a three-dimensional model arranged outside the building. The display special effect can be firework blooming, virtual character model, particle special effect and the like. Wherein, the prompt message can be at least one of characters and images. For example, the prompt may be in the form of a pop-up window. The prompting information of the text can be used for introducing relevant information or safety protection knowledge of the environmental object and the like. The image-like guidance information may be used to convey information like traffic signs (stop, left turn, right turn, slow running, etc.), security signs (no smoke, carefully sliding, etc.), and the like. The three-dimensional model provided outside the building may be a three-dimensional model created for the appearance of the building. The audio information may be a description of the history of the environmental object, or an introduction of the security protection. For example, when a vehicle passes through a place which may cause certain threats to the body or property of a human body, audio corresponding to safety protection can be played, so that the safety of passengers in the driving process of the vehicle is guaranteed. Through setting up the virtual information of multiple classification, can improve the interest in follow-up augmented reality processing procedure.

The method for augmented reality processing of the preset virtual information by using the vehicle-mounted component of the vehicle may be that, in response to the preset virtual information including the display-type information, the display-type information is rendered at a preset position of the environmental object in the first image. The preset position of the environment object can be a preset position on the environment object body, and can also be a preset position relatively independent from the environment object body. The former, for example, renders display type information on an outer wall of a building body, and the latter, for example, renders display type information on a floor beside the building body. The embodiment of the present disclosure takes the preset position on the body of the environmental object as an example. And, the rendered first image may be displayed with an in-vehicle component of the vehicle. In the embodiment of the disclosure, the first image may be captured by a capturing component of the vehicle, or may be captured by a capturing device in communication connection with the vehicle. The shooting device in communication connection with the vehicle can be specifically an unmanned aerial vehicle, a mobile phone, a computer, an intelligent camera and the like. The display effect in the first image can be improved by obtaining the preset position of the display type information rendered in the first image corresponding to the environment object.

The method for performing augmented reality processing on the preset virtual information by using the vehicle-mounted component of the vehicle may be that in response to the preset virtual information including audio information, the audio information is played by using the vehicle-mounted component of the vehicle. The vehicle-mounted component playing the audio information can be a vehicle-mounted sound. Corresponding audio information is played according to the positioning information of the vehicle, so that the interestingness of the vehicle in the driving process can be improved. When the preset virtual information of the environment object includes both display type information and audio type information, the display of the rendered first image by the vehicle-mounted component of the vehicle and the playing of the audio type information by the vehicle-mounted component of the vehicle can be executed according to the current positioning information of the vehicle.

After the preset virtual information is augmented reality processed by utilizing the vehicle-mounted component of the vehicle, the following steps can be further executed:

and responding to a first preset operation of a user, and acquiring interactive virtual information matched with the preset operation from a preset environment. And replacing the preset virtual information with the interactive virtual information to update the preset virtual information of the environment object. Wherein, the logical response relationship between the first preset operation and the interactive virtual information can be preset. Alternatively, the first preset operation may be performed on the displayed preset virtual information or the environment object on the display interface of the vehicle-mounted component. For example, the first preset operation instruction may be a single click, a double click, a drag, a slide, or the like instruction on the environmental object or the displayed preset virtual information. Further, different interactive virtual information can be set according to different sliding directions of the user. Of course, the first preset operation may also be a voice operation. For example, when the preset voice information is recognized, the user is considered to have performed the first preset operation. For example, when the preset virtual information includes display-type information, the displayed information may be clicked, and other preset virtual information may be used as new preset virtual information. Through operating preset virtual information or environment objects or operating according to voice, the whole interaction process is convenient and fast. Of course, if the environment object has only one preset virtual information, the preset operation will not result in the replacement of the preset virtual information. For example, the passenger may see a picture of a real building after real-time special effect processing on the display screen, and interact with preset virtual information superimposed on the building through the display screen, such as changing the appearance of the building, adding or switching real-time visual special effects for the building, and the like.

And after the interactive virtual information is replaced by the preset virtual information, the vehicle-mounted component of the vehicle is used for carrying out augmented reality processing on the preset virtual information again. Specifically, augmented reality processing is performed on a first image currently shot by the vehicle by using the changed preset virtual information. Through responding to the first preset operation of the user, the preset virtual information of the environment object is replaced, and the step of utilizing the vehicle-mounted component of the vehicle to perform augmented reality processing on the preset virtual information is executed again, so that the interaction between the user and the equipment can be realized, and the interestingness of the vehicle in the driving process is further improved.

The information processing method of the vehicle provided by the embodiment of the disclosure can be applied to the tourist car. The tourist guide vehicle belongs to one of regional electric vehicles and is a tourist sightseeing and regional vehicle developed in tourist attractions, parks, scientific and technological parks, large amusement parks, closed communities, campuses, resorts, urban pedestrian streets and other regions. The vehicle can be an environment-friendly electric riding vehicle special for riding instead of walking, and can also be used for connection of large-scale enterprises such as industrial parks, scientific and technical parks and government industrial parks. The vehicle may be an AR vehicle, an autonomous vehicle, or the like.

Through the technical scheme provided by the embodiment of the disclosure, the manned connection and guide explanation demands of scenes such as scenic spots, industrial parks and the like can be met, and contents such as industrial development, economic planning, scientific and technological culture and the like can be displayed in a visual and intelligent mode.

The main body of the vehicle information processing method may be an information processing apparatus of the vehicle, for example, the vehicle information processing method may be executed by a terminal device or a server or other processing device, where the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like. In some possible implementations, the information processing method of the vehicle may be implemented by a processor calling a computer readable instruction stored in a memory.

Referring to fig. 4, fig. 4 is a schematic structural diagram of an embodiment of an information processing device of a vehicle according to the present application. The information processing apparatus 100 of the vehicle includes a first positioning module 110, a second positioning module 120, and a processing module 130. A first positioning module 110, configured to obtain a current position of a vehicle; the second positioning module 120 is configured to determine a corresponding track point of the current position on the preset route; and the processing module 130 is configured to execute preset processing matched with the position condition of the track point on the preset route by using the vehicle-mounted component of the vehicle.

In some disclosed embodiments, the predetermined route is composed of a plurality of segments; the second positioning module 120 determines a track point corresponding to the current position on the preset route, including: determining a line segment matched with the current position in the preset route as a target line segment; and determining track points by using the position relation between the current position and the target line segment.

According to the scheme, the target line segment matched with the current position is determined, and the track point is determined according to the position relation between the current position and the target line segment, so that the obtained track point is determined more accurately.

In some disclosed embodiments, the determining, by the second positioning module 120, a line segment in the preset route matching the current position includes: finding out line segments meeting a first condition between a first vector and a second vector from the plurality of line segments, wherein the first vector consists of a starting point to a current position of the line segment, and the second vector consists of a starting point to an end point of the line segment; or finding out the line segments with the distance from the current position meeting the second condition from the plurality of line segments.

According to the scheme, the vectors not only have the size, but also have the direction, so that the matched route is determined through the relation among the vectors, and the determined route is more accurate. In addition, the matched route is determined according to the distance between the current position, the reference factor is less, and the time consumption of the whole process is less.

In some disclosed embodiments, the step of finding a line segment between the first vector and the second vector, which satisfies the first condition, from the plurality of line segments is performed in response to the length of the first vector being less than or equal to a preset length.

According to the scheme, the size of the first vector is judged before the line segment matched with the current position is determined, so that the accuracy of a subsequent matching result can be improved.

In some disclosed embodiments, the first condition is that the length of the first vector is less than the length of the second vector, and the included angle between the first vector and the second vector is less than a preset angle.

According to the scheme, the accuracy of the matching result can be improved by comprehensively determining the line segment matched with the vector according to the length of the vector and the included angle relationship between the vector and the vector.

In some disclosed embodiments, the second condition is that the distance from the current location in the multi-segment line segment is closest.

According to the scheme, the line segment closest to the current position is selected as the line segment matched with the line segment, and the matching accuracy can be improved.

In some disclosed embodiments, determining the track point by using the position relationship between the current position and the target line segment includes: and taking the intersection point of the perpendicular line from the current position to the target line segment and the target line segment as a track point.

In some disclosed embodiments, before the second positioning module 120 determines the corresponding track point of the current position on the preset route, the second positioning module 120 is further configured to: acquiring a plurality of frames of images obtained by shooting the environment where the planned route is located and positioning information respectively corresponding to the plurality of frames of images; obtaining position information of a plurality of inflection points on the planned route based on the multi-frame image and the positioning information, and connecting the inflection points to form a preset route by using the position information of the inflection points; or transmitting the multi-frame image and the positioning information to the local equipment, and acquiring a preset route from the local equipment.

According to the scheme, the inflection points are determined firstly, and then the inflection points are connected to obtain the route formed by the line segments, compared with the route established in other modes, the route established by the method is faster, and the established route is simpler.

In some disclosed embodiments, the second positioning module 120 captures a plurality of images from a camera assembly of the vehicle or a camera communicatively coupled to the vehicle; and/or obtaining position information of a plurality of inflection points on the planned route based on the multi-frame image and the positioning information, wherein the position information comprises: analyzing the multi-frame images to determine an image related to the inflection point, or displaying the multi-frame images and determining the image related to the inflection point in response to a selection of a user; and obtaining the position information of the inflection point based on the positioning information of the image related to the inflection point.

According to the scheme, the position information of each inflection point can be obtained by analyzing the multi-frame image, or the inflection point information is determined by receiving the selection of a user, so that the resource consumption of the execution equipment can be reduced.

In some disclosed embodiments, the processing module 130 executes a preset process matched with the position condition of the track point on the preset route by using the vehicle-mounted component of the vehicle, including: detecting whether the track point is located in a preset interval of a preset station in a preset route, wherein the preset interval comprises at least one of an inbound interval and an outbound interval; and responding to the track point located in the preset interval of the preset station, and executing preset processing matched with the preset interval of the preset station by using the vehicle-mounted assembly.

According to the scheme, when the track points are judged to be in the preset interval, corresponding preset processing can be executed, and the whole process is convenient and fast.

In some disclosed embodiments, the pre-setting process includes augmented reality processing of pre-setting virtual information of the environmental object matched with the location condition, the pre-setting virtual information being from target map data; before executing the preset processing matched with the position situation of the track point on the preset route by using the vehicle-mounted component of the vehicle, the second positioning module 120 is further configured to: acquiring initial map data, wherein the initial map data is constructed on the basis of a plurality of frames of images which are obtained by shooting the environment where the planned route is located in advance and positioning information respectively corresponding to the plurality of frames of images; determining setting information of at least one environmental object in the initial map data, wherein the setting information comprises preset virtual information; and associating the setting information of at least one environment object to the initial map data to obtain target map data.

According to the scheme, the setting information of the environment object is associated to the initial map, so that the corresponding virtual information can be acquired according to the position of the vehicle.

Referring to fig. 5, fig. 5 is a schematic structural diagram of an embodiment of an on-board device applied to a vehicle according to the present application. As shown in fig. 5, the in-vehicle apparatus 10 includes a memory 200 and a processor 300, and the processor 300 is configured to execute program instructions stored in the memory 200 to implement the steps in any of the above-described embodiments of the information processing method for a vehicle.

Specifically, the processor 300 is configured to control itself and the memory 200 to implement the steps in any of the above-described embodiments of the information processing method of the vehicle. Processor 300 may also be referred to as a CPU (Central Processing Unit). Processor 300 may be an integrated circuit chip having signal processing capabilities. The Processor 300 may also be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, processor 300 may be commonly implemented by integrated circuit chips.

In some disclosed embodiments, the in-vehicle apparatus 10 further includes an in-vehicle component 100 for augmented reality processing of the preset virtual information. The vehicle-mounted assembly 100 may further include a display (not shown), an audio player (not shown), and the like. Wherein, the vehicle-mounted component 100 is connected to the processor 300, and the processor 300 can control the vehicle-mounted component 100 to perform corresponding operations. For example, the processor 300 controls an audio player in the in-vehicle component 100 to perform an operation of playing audio.

Wherein the display may be an interactive display. Wherein the display may acquire interactive virtual information matching the preset operation from the target map data in response to the first preset operation of the user, and replace the interactive virtual information with non-preset virtual information to update the preset virtual information of the environmental object, so that the onboard apparatus 10 re-performs augmented reality processing of the preset virtual information using the onboard component 100 of the vehicle.

Referring to fig. 6, fig. 6 is a schematic structural diagram of an embodiment of a vehicle according to the present application. As shown in fig. 6, the vehicle 1 includes a vehicle body 20 and an in-vehicle device 10 provided on the vehicle body 20. Here, the vehicle-mounted device 10 please refer to the above-mentioned embodiment of the vehicle-mounted device 10, and details thereof are omitted.

The in-vehicle device 10 includes a display screen (not shown). Wherein the display screen is arranged inside the vehicle 1. Specifically, the display screen is provided on the side of the vehicle 1. The display surface of the display screen is parallel to the traveling direction of the vehicle 1. Alternatively, the remaining seats, except for the seat in the driving seat, are aligned in the direction in which the vehicle 1 travels. Wherein, when a human body sits on the seat, the human body faces the display surface of the display screen.

The vehicle 1 provided by the embodiment of the present disclosure may be a tourist guide. The tourist guide vehicle belongs to one of regional electric vehicles and is a tourist sightseeing and regional vehicle developed in tourist attractions, parks, scientific and technological parks, large amusement parks, closed communities, campuses, resorts, urban pedestrian streets and other regions. The vehicle can be an environment-friendly electric riding vehicle special for riding instead of walking, and can also be used for connection of large-scale enterprises such as industrial parks, scientific and technical parks and government industrial parks.

Referring to fig. 7, fig. 7 is a schematic structural diagram of an embodiment of a computer-readable storage medium according to the present application. The computer readable storage medium 2 stores program instructions 21 executable by the processor, the program instructions 21 being for implementing the steps in any of the vehicle information processing method embodiments described above.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

The foregoing description of the various embodiments is intended to highlight various differences between the embodiments, and the same or similar parts may be referred to each other, and for brevity, will not be described again herein.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely one type of logical division, and an actual implementation may have another division, for example, a unit or a component may be combined or integrated with another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some interfaces, and may be in an electrical, mechanical or other form.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Claims

1. An information processing method of a vehicle, characterized by comprising:

acquiring the current position of the vehicle;

determining a corresponding track point of the current position on a preset route;

and executing preset processing matched with the position condition of the track point on the preset route by utilizing the vehicle-mounted component of the vehicle.

2. The method of claim 1, wherein the predetermined route is comprised of a plurality of line segments; the determining of the corresponding track point of the current position on the preset route includes:

determining the line segment matched with the current position in the preset route as a target line segment;

and determining the track points by using the position relation between the current position and the target line segment.

3. The method of claim 2, wherein said determining the line segment of the preset route that matches the current location comprises:

finding out the line segments meeting a first condition between a first vector and a second vector from the plurality of line segments, wherein the first vector consists of a starting point to the current position of the line segment, and the second vector consists of a starting point to an end point of the line segment; or,

and finding out the line segments with the distance from the current position meeting a second condition from the line segments.

4. The method of claim 3, wherein the step of finding the line segment between the first vector and the second vector that satisfies the first condition from the plurality of line segments is performed in response to the length of the first vector being less than or equal to a preset length;

and/or the first condition is that the length of the first vector is smaller than that of the second vector, and an included angle between the first vector and the second vector is smaller than a preset angle;

and/or the second condition is that the distance between the multi-segment line segment and the current position is the nearest.

5. The method of claim 2, wherein determining the trajectory point using the positional relationship between the current position and the target line segment comprises:

and taking the intersection point of the perpendicular line from the current position to the target line segment and the target line segment as the track point.

6. The method according to any one of claims 1 to 5, wherein before said determining the corresponding track point of the current position on the preset route, the method further comprises:

acquiring a plurality of frames of images obtained by shooting the environment where the planned route is located and positioning information respectively corresponding to the plurality of frames of images;

obtaining position information of a plurality of inflection points on the planned route based on the multi-frame image and the positioning information, and connecting the inflection points to form the preset route by using the position information of the inflection points; or transmitting the multi-frame image and the positioning information to local equipment, and acquiring a preset route from the local equipment.

7. The method of claim 6, wherein the plurality of frames of images are captured by a capture component of the vehicle or a capture device communicatively coupled to the vehicle;

and/or obtaining position information of a plurality of inflection points on the planned route based on the multi-frame image and the positioning information, wherein the obtaining of the position information comprises:

analyzing the multi-frame image to determine the image related to the inflection point, or displaying the multi-frame image and determining the image related to the inflection point in response to a selection of a user;

and obtaining the position information of the inflection point based on the positioning information of the image related to the inflection point.

8. The method according to any one of claims 1 to 7, wherein the performing, by means of an on-board component of the vehicle, a preset process matching the position situation of the track point on the preset route includes:

detecting whether the track point is located in a preset interval of a preset station in the preset route, wherein the preset interval comprises at least one of an inbound interval and an outbound interval;

and responding to the track point located in the preset interval of the preset station, and executing preset processing matched with the preset interval of the preset station by using the vehicle-mounted component.

9. The method according to any one of claims 1 to 8, wherein the preset processing comprises augmented reality processing of preset virtual information of the environmental object matched with the position situation, wherein the preset virtual information is derived from target map data;

before the executing, by the vehicle-mounted component of the vehicle, the preset processing matched with the position condition of the track point on the preset route, the method further includes:

acquiring initial map data, wherein the initial map data is constructed on the basis of a plurality of frames of images which are obtained by shooting the environment where the planned route is located in advance and positioning information respectively corresponding to the plurality of frames of images;

determining setting information of at least one environmental object in the initial map data, wherein the setting information comprises the preset virtual information;

and associating the setting information of the at least one environmental object to the initial map data to obtain target map data.

10. An information processing apparatus of a vehicle, characterized by comprising:

the first positioning module is used for acquiring the current position of the vehicle;

the second positioning module is used for determining a corresponding track point of the current position on a preset route;

and the processing module is used for executing preset processing matched with the position condition of the track point on the preset route by utilizing the vehicle-mounted component of the vehicle.

11. An in-vehicle apparatus applied to a vehicle, characterized by comprising a memory and a processor for executing program instructions stored in the memory to implement the method of any one of claims 1 to 9.

12. The apparatus of claim 11, further comprising an in-vehicle component for augmented reality processing of preset virtual information.

13. A vehicle characterized by comprising a vehicle body and an in-vehicle device provided to the vehicle body, wherein the in-vehicle device is the in-vehicle device according to claim 11 or 12.

14. A computer readable storage medium having stored thereon program instructions, characterized in that the program instructions, when executed by a processor, implement the method of any of claims 1 to 9.