WO2024027112A1 - Vehicle control methods, apparatus and system, automated parking assist controller, and terminal - Google Patents

Abstract

Vehicle control methods, apparatus and system, an automated parking assist controller, and a terminal. A vehicle control method is applied to the automated parking assist (APA) controller and comprises: sending to a terminal images collected by a plurality of cameras arranged on a vehicle and camera calibration data, so that the terminal, according to the images and the camera calibration data, determines parking place information and constructs an electronic map (S110); receiving the parking place information and electronic map information returned by the terminal (S120); and determining a parking trajectory according to the parking place information, the electronic map information and radar data, and controlling, according to the parking trajectory, the vehicle to park in a target parking place (S130).

Description

Vehicle control method, device, automatic parking assist controller, terminal and system

This application claims priority to the Chinese patent application with application number 202210916224.1, which was submitted to the China Patent Office on August 1, 2022. The entire content of the above application is incorporated into this application by reference.

Technical field

Embodiments of the present application relate to the field of vehicle control technology, for example, to a vehicle control method, device, automatic parking assistance controller, terminal and system.

Background technique

With the development of vehicle control technology, automatic parking technology is gradually integrated into more models. Automatic parking technology can use visual algorithms to obtain images collected by radars installed on vehicles and perform image processing to determine parking space information and construct an electronic map. It can then determine parking trajectories based on planning and control algorithms to control vehicle parking.

Among related technologies, most of the visual algorithms of automatic parking technology are deployed on the automatic parking assist controller (Automatic Park Assist, APA) of the vehicle, making the hardware equipment on the vehicle more cumbersome. When the vision algorithm needs to be updated to the vehicle after improvements, it is necessary to consider whether the vehicle's hardware resources can support the updated vision algorithm, and when updating the vision algorithm, it is necessary to perform remote upgrade technology (Over The Air, OTA) multiple times. Upgrade, making the upgrade of visual algorithms cumbersome.

Contents of the invention

This application provides a vehicle control method, device, automatic parking assist controller, terminal and system.

In the first aspect, embodiments of the present application provide a vehicle control method applied to an automatic parking assist APA controller, including:

Send the images collected by multiple cameras installed on the vehicle and the camera calibration data to the terminal, so that the terminal determines the parking space information and constructs an electronic map based on the images and the camera calibration data;

Receive parking space information and electronic map information returned by the terminal;

A parking trajectory is determined based on the parking space information, the electronic map information and radar data, and the vehicle is controlled to park into a target parking space based on the parking trajectory.

In the second aspect, embodiments of the present application provide a vehicle control method, applied to a terminal, including:

Receive the image and camera calibration data sent by the automatic parking assist APA controller. The image Collected by multiple cameras installed on the vehicle;

Determine parking space information and construct an electronic map based on the image and the camera calibration data;

Return parking space information and electronic map information to the APA controller, so that the APA controller determines the parking trajectory according to the parking space information, the electronic map information and radar data and controls the vehicle according to the parking trajectory Park into the target parking space.

In a third aspect, embodiments of the present application provide a vehicle control device, including:

A sending module configured to send the images collected by multiple cameras installed on the vehicle and the camera calibration data to the terminal, so that the terminal determines the parking space information and constructs an electronic map based on the images and the camera calibration data;

An information receiving module, configured to receive parking space information and electronic map information returned by the terminal;

A control module is configured to determine a parking trajectory based on the parking space information, the electronic map information and radar data, and control the vehicle to park into a target parking space based on the parking trajectory.

In a fourth aspect, embodiments of the present application provide a vehicle control device, including:

A receiving module configured to receive images and camera calibration data sent by the automatic parking assist APA controller, where the images are collected by multiple cameras installed on the vehicle;

A construction module configured to determine parking space information and construct an electronic map based on the image and the camera calibration data;

An information return module, configured to return parking space information and electronic map information to the APA controller, so that the APA controller determines the parking trajectory according to the parking space information, the electronic map information and radar data and determines the parking trajectory according to the parking space information, the electronic map information and the radar data. The vehicle trajectory controls the vehicle to park into the target parking space.

In the fifth aspect, embodiments of the present application provide an automatic parking assistance controller, including:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform vehicle control as described in the first aspect method.

In a sixth aspect, embodiments of the present application provide a terminal, including:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform vehicle control as described in the second aspect method.

In a seventh aspect, embodiments of the present application provide a vehicle control system, including the APA controller as described in the fifth aspect, and the terminal as described in the sixth aspect.

It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the application, nor is it intended to limit the scope of the application. Other features of the present application will become readily understood from the following description.

Description of drawings

Figure 1 is a flow chart of a vehicle control method provided according to Embodiment 1 of the present application;

Figure 2 is a schematic diagram of controlling a vehicle to park into a target parking space according to a parking trajectory according to Embodiment 1 of the present application;

Figure 3 is a schematic diagram of another method of controlling a vehicle to park into a target parking space based on a parking trajectory according to Embodiment 1 of the present application;

Figure 4 is a flow chart of a vehicle control method provided according to Embodiment 2 of the present application;

Figure 5 is a schematic diagram of determining parking space information based on images and camera calibration data provided according to Embodiment 2 of the present application;

Figure 6 is a schematic diagram of the connection between the APA controller and the mobile phone provided according to Embodiment 2 of the present application;

Figure 7 is a flow chart of another vehicle control method provided according to Embodiment 2 of the present application;

Figure 8 is a schematic structural diagram of a vehicle control device provided according to Embodiment 3 of the present application;

Figure 9 is a schematic structural diagram of a vehicle control device provided according to Embodiment 4 of the present application;

Figure 10 is a schematic structural diagram of an automatic parking assistance controller that implements the vehicle control method according to the embodiment of the present application;

Figure 11 is a schematic structural diagram of a terminal that implements the vehicle control method according to the embodiment of the present application;

Figure 12 is a schematic structural diagram of a vehicle control system that implements the vehicle control method according to the embodiment of the present application.

Detailed ways

Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of protection of this application.

It should be noted that the terms "first", "second", etc. in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof form, is intended to cover non-exclusive inclusion, for example, a process, method, system, product or apparatus that incorporates a series of steps or units need not be limited to those steps or units expressly listed, but may include steps or units not expressly listed. or other steps or elements inherent to such processes, methods, products or devices.

It can be understood that before using the technical solutions disclosed in the embodiments of this application, the user should be informed of the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations and obtain the user's authorization. .

Embodiment 1

Figure 1 is a flow chart of a vehicle control method provided according to Embodiment 1 of the present application. This embodiment can be applied to the situation where the automatic parking assist APA controller controls the vehicle to perform automatic parking. This method can be performed by the vehicle control device. To execute, the vehicle control device can be implemented in the form of software and/or hardware and integrated into the automatic parking assist controller. Among them, the automatic parking assist controller can sense the parking environment through cameras and radars, and automatically or manually set the target parking space according to the user's selection so that the vehicle can automatically park along the parking trajectory until it reaches the final target parking space.

As shown in Figure 1, the method includes:

S110. Send the images and camera calibration data collected by multiple cameras installed on the vehicle to the terminal, so that the terminal determines the parking space information and constructs an electronic map based on the images and camera calibration data.

Among them, the camera may refer to a video input device. Image can refer to the general name of various graphics and images collected by the camera. The image can be a photo or a video.

The number of cameras installed on the vehicle is not limited, as long as images can be collected through the cameras. For example, the number of cameras on the vehicle can be determined based on actual application requirements.

There is no limit to the type of camera, as long as it can capture images, for example, the camera is a wide-angle camera, a fish-eye camera, a monocular camera, a binocular camera or a normal-view camera.

The positions of multiple cameras installed on the vehicle are not limited, as long as images can be collected through the cameras. Specifically, the positions of the cameras on the vehicle can be determined based on the number of cameras or actual application requirements. Cameras in different positions can have different functions. For example, a camera installed on the front windshield of a vehicle can be used to automatically detect pedestrians, vehicles, route signs or traffic lights, etc.; another example, a camera installed on the side of the vehicle can be used for blind spots. Monitoring, due to the limited range of vehicle rearview mirrors, visual blind spots are easily formed where the rearview mirrors cannot be observed. By installing cameras on the sides of the vehicle, the blind spots can be basically covered. When a vehicle enters the blind spot, the user can The camera observes the vehicle driving into the blind spot, which can reduce the occurrence of traffic accidents to a certain extent.

In one embodiment, the type of camera can be determined according to the location of the camera on the vehicle. For example, the camera installed on the front windshield of the vehicle can be a monocular camera or a binocular camera. A monocular camera or a binocular camera can obtain distance information from the collected images, and can obtain the distance between the vehicle and pedestrians, other vehicles or obstacles; for another example, the camera installed on the side of the vehicle can be a wide-angle camera. Installing multiple wide-angle cameras on the side of the vehicle allows the user to observe the situation on the side of the vehicle through the camera; for another example, the camera installed on the rear of the vehicle can be a wide-angle camera or a fisheye camera, which can be used for parking assistance, allowing the user to park when parking. While driving, you can observe the situation behind the vehicle through the camera behind the vehicle.

In one embodiment, there are at least four cameras installed on the vehicle. The type of camera is a wide-angle camera or a fish-eye camera. The four cameras are respectively installed in the four directions of the vehicle, the front, rear, left, and right. By collecting the images collected by the four cameras, By splicing the images together, a panoramic view around the vehicle can be obtained, and combined with the algorithm, road line perception or parking control can be achieved.

Camera calibration data may refer to data that can reflect the relative position of the camera on the vehicle. The camera calibration data can represent the positions of multiple cameras installed on the vehicle. A coordinate system can be established with the vehicle center as the origin, and the positions of the multiple cameras relative to the vehicle center can be calibrated in the coordinate system.

In one embodiment, the number of cameras and the relative positions of the cameras on the vehicle are predetermined during vehicle production, so the camera calibration data is determined and unchanged.

In one embodiment, the number of cameras and the relative position of the camera on the vehicle can be determined according to the actual situation, and then the camera calibration data can be determined. When the number of cameras or the relative position of the camera on the vehicle changes, the camera calibration data Changes ensue.

The terminal may refer to an input and output device. The type of the terminal is not limited in this application. For example, it may be a notebook, a tablet computer, a laptop computer, a personal digital assistant and other suitable computers; for example, it may also be a variety of computers. Mobile terminals in the form of personal digital assistants, smart phones, wearable devices and other similar devices.

The method of sending images and camera calibration data collected by multiple cameras installed on the vehicle to the terminal is not limited, as long as the images and camera calibration data collected by multiple cameras installed on the vehicle can be sent to the terminal. For example, a wireless connection is established with the terminal by verifying the authority of the terminal, and then the images collected by multiple cameras installed on the vehicle and the camera calibration data are sent to the terminal.

Among them, the method of verifying the authority of the terminal is not limited, for example, it can be through password verification. The method of establishing a wireless connection with the terminal is not limited. For example, the wireless connection may be established with the terminal through wireless communication technologies such as Bluetooth, wireless network communication technology WIFI, or the 4th Generation Mobile Communication Technology (4G).

This step sends the images collected by multiple cameras installed on the vehicle and the camera calibration data to the terminal, so that the terminal determines the parking space information and builds an electronic map based on the images and camera calibration data, saving the vehicle's hardware resources and improving the efficiency of the vehicle. user experience.

S120: Receive parking space information and electronic map information returned by the terminal.

The parking space information may refer to location information where the vehicle can park. The parking space information may include but is not limited to the four corner point coordinates of the parking space or the parking space line. The four corner point coordinates of the parking space may be coordinates determined by establishing a coordinate system with the center position of the vehicle as the origin. The parking space line may refer to the The line that distinguishes parking spaces from other areas other than parking spaces. One or more parking spaces that can be used for vehicle parking can be obtained through the parking space information. When the parking space information indicates that there are multiple parking spaces that can be used for parking, the user can use the parking space information to select a target parking space among the multiple parking spaces to park the vehicle. To perform parking, the APA controller of the vehicle may randomly select a target parking space among multiple parking spaces for parking through the parking space information, where the target parking space may refer to the parking space where the vehicle is finally parked.

Electronic maps can refer to maps that are stored and consulted digitally using computer technology, also known as digital maps. The electronic map in this application may include, but is not limited to, obstacle information around the vehicle, other vehicle information, and parking space information. The obstacle information around the vehicle may include, but is not limited to, the types of obstacles around the vehicle and the locations where the obstacles are located. Position, other vehicle information may include but is not limited to the position of other vehicles relative to the center of the vehicle. The electronic map information may refer to information about the electronic map, and the electronic map information may include the electronic map.

The method of receiving the parking space information and electronic map information returned by the terminal is not limited, as long as it can receive the parking space information and electronic map information returned by the terminal. For example, the APA controller establishes a wireless connection with the terminal to receive the parking space information and electronic map information returned by the terminal.

By receiving the parking space information and electronic map information returned by the terminal, the APA controller can determine the parking trajectory based on the parking space information, electronic map information and radar data.

S130: Determine the parking trajectory based on the parking space information, electronic map information and radar data, and control the vehicle to park into the target parking space based on the parking trajectory.

The radar data may refer to data collected by a radar installed on the vehicle. Radar can refer to an electronic device that uses electromagnetic waves to detect targets. Radar emits electromagnetic waves to illuminate the target and receives its echo, thereby obtaining the distance from the target to the electromagnetic wave emission point, distance change rate (radial velocity), orientation or altitude, etc. information. By setting up radar on the vehicle, the relative distance, relative speed, angle or direction of movement between the vehicle and other objects can be detected. Radar data may include, but is not limited to, the relative distance, relative speed, angle or direction of movement between the vehicle and other objects.

There is no restriction on the type of radar installed on the vehicle, as long as radar data can be obtained through the radar That’s it. For example, it can be millimeter wave radar, infrared radar, ultrasonic radar or laser radar, etc. Different types of radars can have different effects when collecting radar data. For example, millimeter wave radar has a longer detection range, but its ability to identify obstacles is weaker than lidar; another example is that lidar has a longer detection range and can also Accurately locate obstacles.

There is no limit on the number of radars installed on the vehicle, and there is no limit on the location of the radar on the vehicle, as long as radar data can be obtained through the radar. The number of radars and the location of the radar can be determined according to actual application needs.

In one embodiment, 12 radars are installed on the vehicle, and the 12 radars are evenly distributed around the vehicle. Radar data can be obtained through the 12 radars.

The parking trajectory may refer to the trajectory of controlling the vehicle to park into the target parking space. The parking trajectory can be used to determine the steering wheel information, vehicle speed information, gear information, braking information or headlight information of the vehicle during the process of parking into the target parking space, so that the vehicle can park into the target parking space according to the parking trajectory.

Among them, the steering wheel information can represent the state of the steering wheel when the vehicle is parking, such as the steering wheel's rotation angle or the steering wheel's rotation direction. The vehicle speed information can represent the speed that the vehicle should have when parking. For example, a preset speed can be set so that the vehicle maintains the preset speed for parking. The preset speed can be set according to actual needs; another example can be based on the actual situation. Control the vehicle speed to park the vehicle, reduce the vehicle speed when the vehicle needs to turn, or maintain the preset speed when the vehicle goes straight. The gear information can represent the gear that the vehicle should be in when parking. For example, the vehicle can be parked according to the set gear. The set gear can be a gear set according to actual needs; another example can be based on the actual situation. Change the gear to park the vehicle, shift to reverse when the vehicle needs to move backwards, or select a smaller gear when the vehicle is traveling at a slower speed. Braking information can represent the braking status of the vehicle when parking, such as the frequency of braking or the strength of braking. The car light information can represent the status of the car lights when the vehicle is parked. For example, when the vehicle is parked, double flashing is turned on to remind surrounding vehicles to pay attention to avoidance.

The method of determining the parking trajectory based on the parking space information, electronic map information and radar data is not limited, as long as the parking trajectory can be determined based on the parking space information, electronic map information and radar data. For example, the target parking space can be determined based on parking space information; the position of the target parking space relative to the vehicle can be determined through electronic map information, as well as the obstacles or other vehicles around the path from the vehicle to the target parking space; the distance between the vehicle and other objects can be determined based on radar data The relative distance, relative speed, angle or direction of movement, etc.; combined with parking space information, electronic map information and radar data, the path of the vehicle into the target parking space can be planned, and the parking trajectory can be determined.

The method of controlling the vehicle to park into the target parking space based on the parking trajectory is not limited, as long as the vehicle can be controlled to park into the target parking space based on the parking trajectory. For example, determining the steering wheel information based on the parking trajectory, that is, the vehicle The state that the steering wheel should be in when parking; determine the vehicle speed information based on the parking trajectory, that is, the speed the vehicle should be when parking; determine the gear information based on the parking trajectory, that is, the gear the vehicle should be in when parking; based on the parking trajectory The trajectory determines the braking information, that is, the state of the brakes when the vehicle is parked; or the parking trajectory determines the vehicle light information, that is, the state of the vehicle lights when the vehicle is parked; and then determines the associated hardware required for the vehicle to complete parking based on the parking trajectory, and The hardware of the vehicle associated with the parking trajectory is connected, and the hardware of the relevant vehicle is controlled to perform corresponding operations according to the parking trajectory, so that the vehicle can park into the target parking space according to the parking trajectory.

Figure 2 is a schematic diagram of controlling a vehicle to park into a target parking space according to a parking trajectory provided according to Embodiment 1 of the present application. As shown in Figure 2, the vehicle parks vertically into the target parking space. From top to bottom in Figure 2 are: The first step is to plan in front of the target parking space so that the vehicle is in front of the target parking space; the second step is to plan to the area inside the target parking space and when an obstacle is detected, re-plan to the front of the target parking space; the third step is to reach the second The planned parking spot is re-planned or obstacles are encountered, and the posture of the vehicle is adjusted forward so that the vehicle can park into the target parking space at a small angle.

Figure 3 is a schematic diagram of another method of controlling a vehicle to park into a target parking space based on a parking trajectory according to Embodiment 1 of the present application. As shown in Figure 3, the vehicle parked horizontally into the target parking space. From left to right in Figure 3 are: The first step is to plan the vehicle to the left front of the target parking space so that the vehicle is in the left front of the target parking space; the second step is to plan the vehicle to the right rear and determine the angle of turning into the target parking space based on the rear position of the vehicle in the first step; the third step , the vehicle parks into the target parking space; the fourth step is to adjust the posture of the vehicle.

The technical solution of the embodiment of this application sends the image and camera calibration data to the terminal through the APA controller. The terminal determines the parking space information and constructs an electronic map. The terminal then returns the parking space information and electronic map information to the APA controller, allowing the APA to control The device determines the parking trajectory through parking space information, electronic map information and radar data, and controls the vehicle to park into the target parking space based on the parking trajectory, saving vehicle hardware resources and improving user experience.

Further, before controlling the vehicle to park into the target parking space according to the parking trajectory, the vehicle control method also includes:

Handshakes with the vehicle's associated components, which include at least one of the following: body control module, electronic parking brake system, electronic power steering system, electronic stability control unit, and automatic transmission control unit.

Among them, the associated component may refer to the vehicle's hardware related to controlling vehicle parking. The associated component shakes hands with the vehicle, which can be understood as establishing a connection between the APA controller and the vehicle's associated component, and the APA controller can send the parking trajectory. Give the corresponding associated parts and control the associated parts to perform corresponding operations so that the vehicle can park into the target parking space according to the parking trajectory.

Related components include at least one of the following: body control module, electronic parking brake system, electronic power assist Steering system, electronic stability control unit and automatic transmission control unit.

Among them, the body control module (Body Control Module, BCM) can refer to the module used to control common body functions, such as car light control, window control or central door lock control, etc. By shaking hands with the APA controller and the body control module, you can turn on the double flashes through the body control module during parking to remind surrounding vehicles to pay attention and avoid them.

Electronic Parking Brake (EPB) can refer to a system that controls vehicle parking brakes through electronic circuits. By shaking hands with the APA controller and the electronic parking brake system, parking can be achieved during the parking process. When braking is required, the electronic parking brake system controls the vehicle to perform parking brakes.

Electronic Power Steering (EPS) can refer to a power steering system that directly relies on the motor to provide auxiliary torque. EPS can determine the rotation direction of the motor and the size of the assist current through the signals of the vehicle speed sensor and torque sensor, thereby Control the steering wheel. EPS can provide different power-assisting effects of the motor at different vehicle speeds, ensuring that the vehicle is light and flexible when turning at low speeds and stable and reliable when turning at high speeds. By shaking hands with the APA controller and the electronic power steering system, the electronic power steering system can control the steering wheel rotation during parking.

Electronic Stabilization Control (ESC) can refer to a unit that can control the stability of a vehicle. ESC can control the longitudinal and lateral stability of the vehicle through a combination of multiple sensors, electronic control units and actuators. By shaking hands with the APA controller and the electronic stability control unit, the electronic stability control unit can control the vehicle's acceleration during parking to keep the vehicle stable.

The automatic transmission control unit (Transmission Control Unit, TCU) can refer to a unit that can realize automatic transmission of the vehicle through computer and power electronic drive technology. The APA controller and the automatic transmission control unit shake hands and can be passed during the parking process. The automatic transmission control unit automatically changes the vehicle's gear state.

By handshaking the associated components with the vehicle before controlling the vehicle to park into the target parking space according to the parking trajectory, the APA controller can control the corresponding associated components to perform corresponding operations based on the parking trajectory, thereby controlling the vehicle to park into the target parking space.

Further, before determining the parking trajectory based on parking space information, electronic map information and radar data, the vehicle control method also includes:

The target parking space is determined based on the user's selection operation on the parking space information.

Among them, the parking space information may include one or more parking spaces, and the user can select the parking space information to determine the final target parking space for parking the vehicle.

The user's selection operation of the parking space information is not limited, as long as the user can select the parking space information and then determine the target parking space. For example, the user selects parking space information through the central control screen on the vehicle's APA controller. The central control screen can refer to a display screen that displays vehicle audio, navigation or vehicle information. The central control screen can realize the interaction between the user and the vehicle. Interactively, different parking spaces can be displayed on the central control screen. The user selects any one of the parking spaces as the target parking space through touch operation. The APA controller determines the target parking space based on the user's selection operation of the parking space information.

By determining the target parking space based on the user's selection operation of the parking space information before determining the parking trajectory based on the parking space information, electronic map information and radar data, when the parking space information indicates that there are multiple parking spaces, the target parking space can be determined through the user's selection operation. Make the selection of target parking spaces more in line with user needs.

Further, the vehicle control method also includes:

The images collected by each camera are spliced based on the camera calibration data, and the spliced images are displayed on the vehicle's central control screen.

Multiple cameras installed at different positions on the vehicle can collect different images. For example, a camera installed in front of the vehicle can collect images in front of the vehicle, and a camera installed on the side of the vehicle can collect images on the side of the vehicle. The camera behind the vehicle can collect images from behind the vehicle. By stitching together different images collected by each camera, a complete view around the vehicle can be obtained.

The images collected by each camera are spliced according to the camera calibration data, and the spliced images are displayed on the central control screen of the vehicle. The images collected by different cameras can be spliced into a complete 360-degree surround view, and displayed on the central control screen. , allowing users to observe the situation around the vehicle through the central control screen, making the parking process safer.

Embodiment 2

Figure 4 is a flow chart of a vehicle control method provided according to Embodiment 2 of the present application. This embodiment can be applied to the situation where the terminal determines the parking space information and constructs an electronic map by receiving the information sent by the APA controller. This method can be used by the vehicle The vehicle control device can be implemented in the form of software and/or hardware and integrated in the terminal. The type of terminal is not limited. For example, it can be a notebook, a tablet computer, a laptop computer, a personal digital assistant and other suitable computers; for example, it can also be various forms of mobile terminals, such as a personal digital processor, a smart phone , wearables and other similar devices.

As shown in Figure 4, the method includes:

S210. Receive images and camera calibration data sent by the automatic parking assist APA controller. Figure Images are collected by multiple cameras installed on the vehicle.

The images sent by the APA controller can include various graphics and images collected by multiple cameras installed on the vehicle. The images can be photos or videos.

The camera calibration data sent by the APA controller can represent the positions of multiple cameras installed on the vehicle. The coordinate system can be established with the vehicle center as the origin, and the positions of the multiple cameras relative to the vehicle center can be calibrated in the coordinate system.

The method of receiving the image and camera calibration data sent by the automatic parking assist APA controller is not limited, as long as it can receive the image and camera calibration data sent by the automatic parking assist APA controller. For example, the terminal establishes a wireless connection with the APA controller to receive images and camera calibration data sent by the APA controller.

By receiving the images and camera calibration data sent by the APA controller, the terminal can obtain the positions of multiple cameras installed on the vehicle relative to the center of the vehicle, as well as the images collected by each camera, so that the terminal can obtain the images and camera calibration data based on the images. Determine parking space information and construct an electronic map.

S220: Determine parking space information and construct an electronic map based on the image and camera calibration data.

The method of determining the parking space information based on the image and camera calibration data is not limited, as long as the parking space information can be determined based on the image and camera calibration data. For example, images collected by multiple cameras can be spliced. During the image splicing process, the positions of the multiple cameras on the vehicle relative to the center of the vehicle and the images collected by each camera can be determined based on the camera calibration data; and then the splicing The final image is converted into a bird's-eye view to form a bird's-eye view; the bird's-eye view is binarized, that is, the gray value of the pixels on the bird's-eye view is set to 0 or 255, which is the process of making the entire bird's-eye view show an obvious black and white effect. , greatly reducing the amount of data in the bird's-eye view, thereby highlighting the outline of the parking space line, and then determining the parking space information based on the image and camera calibration data. Among them, the bird's-eye view can refer to a three-dimensional view drawn based on the principle of perspective, using a high-view point perspective method to look down at the ground undulations from a certain point high up. The bird's-eye view can retain the geometric characteristics of the ground lines, which is conducive to the detection of parking space lines.

In one embodiment, Figure 5 is a schematic diagram of determining parking space information based on images and camera calibration data provided according to Embodiment 2 of the present application. As shown in Figure 5, the camera is a camera installed in front of the vehicle. By collecting the The perspective of the image is converted to form a bird's-eye view, and then the bird's-eye view is binarized to extract features consistent with the parking space line (as shown in the black border in Figure 5). By processing the bird's-eye view, the two corner point coordinates or parking space lines of the parking space in front of the vehicle can be obtained. If combined with images collected by cameras installed in other locations on the vehicle, complete parking space information can be obtained.

The method of constructing the electronic map based on the image and camera calibration data is not limited, as long as the electronic map can be constructed based on the image and camera calibration data. For example, obtain through camera calibration data The position of multiple cameras on the vehicle relative to the center of the vehicle and the images collected by each camera, and the images collected by the multiple cameras on the vehicle are identified through image processing software, and the obstacles around the vehicle in the images can be obtained information and other vehicle information, and then combined with the parking space information to construct an electronic map. The electronic map can display obstacle information around the vehicle, other vehicle information and parking space information. The obstacle information may include but is not limited to the types of obstacles around the vehicle and the locations of the obstacles. Other vehicle information may include, but is not limited to, the location of other vehicles relative to the center of the vehicle.

The display method of the obstacle information and other vehicle information around the vehicle on the electronic map is not limited. For example, the obstacle information and other vehicle information around the vehicle can be displayed in the form of text on the electronic map; for another example, on the electronic map The obstacle information and other vehicle information around the vehicle are displayed in the form of icons. Different icons can be used to distinguish the obstacle information and other vehicle information around the vehicle; another example is to extract the information about the obstacles and other vehicles around the vehicle through image processing. Contours, and express these contours in the form of points, establish a coordinate system with the vehicle center as the origin, and use the coordinates to display obstacle information and other vehicle information around the vehicle in the form of points on the electronic map.

The display method of the parking space information on the electronic map is not limited. For example, the parking space information can be displayed on the electronic map in the form of a box; another example is to establish a coordinate system with the vehicle center as the origin and display the parking space information in the form of coordinates.

This step determines the parking space information and constructs an electronic map based on the image and camera calibration data, which is beneficial to determining the parking trajectory through the parking space information and electronic map.

S230. Return the parking space information and electronic map information to the APA controller, so that the APA controller determines the parking trajectory based on the parking space information, electronic map information, and radar data and controls the vehicle to park into the target parking space based on the parking trajectory.

The electronic map information may include the electronic map constructed based on the image and camera calibration data in step S220. The parking space information may be the parking space information determined based on the image and camera calibration data in step S220.

The parking space information returned to the APA controller may include but is not limited to the four corner point coordinates of the parking space or the parking space lines. By returning the parking space information to the APA controller, the APA controller can obtain one or more parking spaces that can be used for vehicle parking. . When the parking space information indicates that there are multiple parking spaces available for parking, the user can use the parking space information in the APA controller to select a target parking space among the multiple parking spaces to park the vehicle, or the vehicle's APA controller can use the parking space information to select a target parking space for parking. Parking space information randomly selects a target parking space for parking among multiple parking spaces.

The electronic map information returned to the APA controller may include but is not limited to obstacle information around the vehicle, other vehicle information and parking space information. By returning the electronic map information to the APA controller, the APA The controller obtains the position of the target parking space relative to the vehicle and the conditions of obstacles or other vehicles around the vehicle's path to the target parking space.

The technical solution of the embodiment of this application is to receive the image and camera calibration data sent by the APA controller through the terminal, determine the parking space information at the terminal and construct an electronic map and return it to the APA controller, so that the APA controller can determine the parking space information, electronic map information and The radar data determines the parking trajectory and controls the vehicle to park into the target parking space based on the parking trajectory, saving the vehicle's hardware resources and improving the user experience.

An exemplary description of this application is as follows:

Taking the terminal as a mobile phone as an example, a vision algorithm (that is, determining parking space information and constructing an electronic map based on images and camera calibration data) is proposed in the mobile phone software (Application, APP), and a regulation algorithm (that is, based on parking space information, electronic map Map information and radar data determine the parking trajectory and control the vehicle to park into the target parking space according to the parking trajectory) are set in the APA controller to implement the parking method.

Figure 6 is a schematic diagram of the connection between the APA controller and the mobile phone provided according to Embodiment 2 of the present application; as shown in Figure 6, the camera and radar installed on the vehicle can be connected to the APA controller to collect the images and radar data collected by the camera Sent to the APA controller, the cameras can be four cameras set up on the front, rear, left, and right of the vehicle, and the radar can be twelve. The APA controller can have a control algorithm, a state machine for interacting with the user, and an associated control module, such as a mobile phone There can be visual algorithms in the APP, and the APA controller and the mobile APP are connected through WIFI or Bluetooth. The APA controller can communicate with the body CAN through the Controller Area Network (CAN). The vehicle's central control display (i.e. central control screen) can display the video content incoming from the 360 surround view controller to the user.

Among them, four cameras on the front, rear, left and right can obtain image information in real time. Twelve radars can detect distance in real time. The APA controller can communicate with the mobile APP, obtain the images and camera calibration data collected by the camera through the APA controller, and send the images and camera calibration data to the visual algorithm of the mobile APP through WIFI or Bluetooth communication, so that the vision The algorithm identifies parking spaces and constructs electronic map information through image processing. The mobile APP then gives the visual algorithm results (i.e., parking space information and electronic map information) to the planning and control algorithm on the APA controller, and the planning and control algorithm determines the parking trajectory, thereby completing the process. Parking.

Figure 7 is a flow chart of another vehicle control method provided according to Embodiment 2 of the present application. As shown in Figure 7, the APA controller is connected to the mobile phone APP, and the APA controller provides the image and camera calibration data to the mobile phone APP in real time. The mobile APP's visual algorithm performs image processing. If a parking space is recognized, the parking space information and electronic map information are sent back to the APA controller. The APA controller displays the parking space map. The user can select the target parking space based on the parking space information through the parking space map on the APA controller. , if the user confirms parking, the APA controller performs a handshake with the associated components, and the control algorithm performs trajectory calculation (i.e., determines the parking trajectory) to control the vehicle. Parking.

By running the vision algorithm on the mobile phone APP, the embedded device hardware resources on the vehicle are saved and the cost is reduced; at the same time, the mobile phone update cycle is short, and after the emergence of more powerful algorithm chips, users have a greater chance of obtaining it; mobile phone resources are richer, The operating performance is better, and better visual algorithms can be run, and different levels of visual algorithms can be run according to the mobile phone hardware resources; the mobile APP upgrade is stable, convenient and fast, reducing the risk of controller failure after OTA upgrade of the embedded device visual algorithm; The input parameters of the vision algorithm can be flexibly adjusted to adapt to various vehicles.

Embodiment 3

Figure 8 is a schematic structural diagram of a vehicle control device provided according to Embodiment 3 of the present application. This embodiment can be applied to situations where an automatic parking assist APA controller controls a vehicle to perform automatic parking. As shown in Figure 8, the specific structure of the device includes:

The sending module 31 is used to send the images collected by multiple cameras installed on the vehicle and the camera calibration data to the terminal, so that the terminal determines the parking space information and constructs an electronic map based on the images and camera calibration data;

The information receiving module 32 is used to receive parking space information and electronic map information returned by the terminal;

The control module 33 is used to determine the parking trajectory according to the parking space information, electronic map information and radar data, and control the vehicle to park into the target parking space according to the parking trajectory.

The vehicle control device provided in this embodiment first sends the images collected by multiple cameras installed on the vehicle and the camera calibration data to the terminal through the sending module 31, so that the terminal determines the parking space information and constructs an electronic map based on the images and camera calibration data. ; Then the information receiving module 32 receives the parking space information and electronic map information returned by the terminal; finally, the control module 33 determines the parking trajectory according to the parking space information, electronic map information and radar data, and controls the vehicle to park into the target parking space according to the parking trajectory.

Further, before controlling the vehicle to park into the target parking space according to the parking trajectory, the device also includes:

The associated component handshake module is used to perform associated component handshaking with the vehicle. The associated component includes at least one of the following: body control module, electronic parking brake system, electronic power steering system, electronic stability control unit, and automatic transmission control unit.

Further, before determining the parking trajectory based on parking space information, electronic map information and radar data, the device also includes:

The target parking space determination module is used to determine the target parking space based on the user's selection operation of the parking space information.

Furthermore, the device also includes:

The spliced image display module is used to display the images collected by each camera based on the camera calibration data. Perform splicing and display the spliced image on the vehicle's central control screen.

The vehicle control device provided by the embodiment of the present application can execute the vehicle control method provided by the first embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

Embodiment 4

Figure 9 is a schematic structural diagram of a vehicle control device provided according to Embodiment 4 of the present application. This embodiment is applicable to the situation where the terminal determines parking space information and constructs an electronic map by receiving information sent by the APA controller. As shown in Figure 9, the specific structure of the device includes:

The receiving module 41 is used to receive images and camera calibration data sent by the automatic parking assist APA controller. The images are collected by multiple cameras installed on the vehicle;

Building module 42, used to determine parking space information and build an electronic map based on images and camera calibration data;

The information return module 43 is used to return parking space information and electronic map information to the APA controller, so that the APA controller determines the parking trajectory based on the parking space information, electronic map information and radar data and controls the vehicle to park into the target parking space based on the parking trajectory.

The vehicle control device provided in this embodiment first receives the image and camera calibration data sent by the automatic parking assist APA controller through the receiving module 41. The images are collected by multiple cameras installed on the vehicle; then through the building module 42, the image and the camera calibration data are collected. The calibration data determines the parking space information and constructs an electronic map; finally, the information return module 43 returns the parking space information and electronic map information to the APA controller, so that the APA controller determines the parking trajectory based on the parking space information, electronic map information and radar data and determines the parking trajectory according to the parking space information. The vehicle trajectory controls the vehicle to park into the target parking space.

The vehicle control device provided in the embodiment of the present application can execute the vehicle control method provided in the second embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

Embodiment 5

Figure 10 is a schematic structural diagram of an automatic parking assist controller that implements the vehicle control method according to the embodiment of the present application. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit the implementation of the present application as described and/or claimed herein.

As shown in Figure 10, the automatic parking assist controller 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (Read-Only Memory, ROM) 12, a random access memory ( Random Access Memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor. The processor 11 can be loaded into the computer program according to the computer program stored in the read-only memory (ROM) 12 or from the storage unit 18. Computer programs in random access memory (RAM) 13 to perform various appropriate actions and processes. In RAM 13, Various programs and data required for the operation of the automatic parking assist controller 10 may also be stored. The processor 11 , the ROM 12 and the RAM 13 are connected to each other via the bus 14 . An input/output (I/O) interface 15 is also connected to the bus 14 .

Multiple components in the automatic parking assist controller 10 are connected to the I/O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, For example, magnetic disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the automatic parking assist controller 10 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunications networks.

Processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), various dedicated artificial intelligence (Artificial Intelligence, AI) computing chips, various running Machine learning model algorithm processor, digital signal processor (Digital Signal Processor, DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as vehicle control methods.

In some embodiments, the vehicle control method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18 . In some embodiments, part or all of the computer program may be loaded and/or installed onto the automatic parking assist controller 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the vehicle control method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the vehicle control method in any other suitable manner (eg, by means of firmware).

Figure 11 is a schematic structural diagram of a terminal that implements the vehicle control method according to the embodiment of the present application. Terminal is intended to mean notebooks, tablets, laptops, personal digital assistants and other suitable computers, or various forms of mobile terminals such as personal digital assistants, smartphones, wearable devices and other similar devices.

As shown in Figure 11, the terminal 20 includes at least one processor 21, and a memory communicatively connected to the at least one processor 21, such as a read-only memory (ROM) 22, a random access memory (RAM) 23, etc., wherein the memory stores A computer program executable by at least one processor. The processor 21 may execute according to a computer program stored in a read-only memory (ROM) 22 or loaded from a storage unit 28 into a random access memory (RAM) 23 Various appropriate actions and treatments. In the RAM 23, various programs and data required for the operation of the terminal 20 can also be stored. The processor 21, the ROM 22 and the RAM 23 are connected to each other via the bus 24. An input/output (I/O) interface 25 is also connected to the bus twenty four.

Multiple components in the terminal 20 are connected to the I/O interface 25, including: an input unit 26, such as a keyboard, a mouse, etc.; an output unit 27, such as various types of displays, speakers, etc.; a storage unit 28, such as a magnetic disk, an optical disk, etc. ; And communication unit 29, such as network card, modem, wireless communication transceiver, etc. The communication unit 29 allows the terminal 20 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunications networks.

Processor 21 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 21 performs various methods and processes described above, such as vehicle control methods.

In some embodiments, the vehicle control method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 28 . In some embodiments, part or all of the computer program may be loaded and/or installed onto the terminal 20 via the ROM 22 and/or the communication unit 29. When the computer program is loaded into RAM 23 and executed by processor 21, one or more steps of the vehicle control method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the vehicle control method in any other suitable manner (eg, by means of firmware).

Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Product (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), computer hardware, firmware, software, and/or they implemented in a combination. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.

Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that the computer program, when executed by the processor, causes the functions/operations specified in the flowcharts and/or block diagrams to be implemented. A computer program can be executed entirely on a machine, partially on a Executes on the machine, as a stand-alone software package that executes partially on the machine and partially on the remote machine or entirely on the remote machine or server.

In the context of this application, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (Erasable Programmable Read-Only Memory, EPROM) or flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any of the above Suitable combination.

To provide interaction with a user, the systems and techniques described herein may be implemented on an automated parking assist controller or terminal having a display device for displaying information to a user (e.g., A cathode ray tube (CRT) or liquid crystal display (LCD) monitor); and a keyboard and pointing device (such as a mouse or trackball) through which the user can transmit input Provided to automatic parking assist controller or terminal. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN), blockchain network, and the Internet.

Computing systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other. The server can be a cloud server, Also known as cloud computing server or cloud host, it is a host product in the cloud computing service system to solve the difficult management and business expansion problems in traditional physical host and virtual private server (VPS) services. The defect of sexual weakness.

Figure 12 is a schematic structural diagram of a vehicle control system that implements the vehicle control method according to the embodiment of the present application. As shown in FIG. 12 , the vehicle control system 50 includes an APA controller 51 and a terminal 52 . The APA controller 51 and the terminal 52 can communicate by establishing a wireless connection. The method of establishing the wireless connection is not limited. For example, the APA controller 51 and the terminal 52 can establish a wireless connection through wireless communication technologies such as Bluetooth, wireless network communication technology WIFI, or fourth-generation mobile communication technology. connect.

A wireless connection is established between the APA controller 51 and the terminal 52. The APA controller 51 can send images and camera calibration data collected by multiple cameras installed on the vehicle to the terminal 52. The terminal 52 receives the images sent by the APA controller 51. and camera calibration data to determine the parking space information and construct an electronic map, and then return the parking space information and electronic map information to the APA controller 51, so that the APA controller 51 determines the parking trajectory based on the parking space information, electronic map information and radar data, and determines the parking trajectory according to the parking space information, electronic map information and radar data. The vehicle trajectory controls the vehicle to park into the target parking space.

This application provides a vehicle control method, device, automatic parking assist controller, terminal and system. By setting the visual algorithm in the terminal, it saves the hardware resources of the vehicle and solves the problem of upgrading the visual algorithm in the vehicle's APA controller. More complicated problems are solved, which improves the user experience.

It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in this application can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solution of this application can be achieved, there is no limitation here.

Claims (10)

1. A vehicle control method, applied to an automatic parking assist APA controller, the method includes:

   Send the images collected by multiple cameras installed on the vehicle and the camera calibration data to the terminal, so that the terminal determines the parking space information and constructs an electronic map based on the images and the camera calibration data;

   Receive parking space information and electronic map information returned by the terminal;

   A parking trajectory is determined based on the parking space information, the electronic map information and radar data, and the vehicle is controlled to park into a target parking space based on the parking trajectory.
2. The method according to claim 1, before controlling the vehicle to park into a target parking space according to the parking trajectory, further comprising:

   Perform handshakes with associated components of the vehicle, where the associated components include at least one of the following: a body control module, an electronic parking brake system, an electronic power steering system, an electronic stability control unit, and an automatic transmission control unit.
3. The method according to claim 1, before determining the parking trajectory based on the parking space information, the electronic map information and radar data, further comprising:

   The target parking space is determined according to the user's selection operation on the parking space information.
4. The method of claim 1, further comprising:

   The images collected by each of the cameras are spliced according to the camera calibration data, and the spliced images are displayed on the central control screen of the vehicle.
5. A vehicle control method, applied to a terminal, the method includes:

   Receive images and camera calibration data sent by the automatic parking assist APA controller, where the images are collected by multiple cameras installed on the vehicle;

   Determine parking space information and construct an electronic map based on the image and the camera calibration data;

   Return parking space information and electronic map information to the APA controller, so that the APA controller determines the parking trajectory according to the parking space information, the electronic map information and radar data and controls the vehicle according to the parking trajectory Park into the target parking space.
6. A vehicle control device including:

   A sending module configured to send the images collected by multiple cameras installed on the vehicle and the camera calibration data to the terminal, so that the terminal determines the parking space information and constructs an electronic map based on the images and the camera calibration data;

   An information receiving module, configured to receive parking space information and electronic map information returned by the terminal;

   A control module is configured to determine a parking trajectory based on the parking space information, the electronic map information and radar data, and control the vehicle to park into a target parking space based on the parking trajectory.
7. A vehicle control device including:

   A receiving module configured to receive images and camera calibration data sent by the automatic parking assist APA controller, where the images are collected by multiple cameras installed on the vehicle;

   A construction module configured to determine parking space information and construct an electronic map based on the image and the camera calibration data;

   An information return module, configured to return parking space information and electronic map information to the APA controller, so that the APA controller determines the parking trajectory according to the parking space information, the electronic map information and radar data and determines the parking trajectory according to the parking space information, the electronic map information and the radar data. The vehicle trajectory controls the vehicle to park into the target parking space.
8. An automatic parking assist controller, including:

   at least one processor; and

   a memory communicatively connected to the at least one processor; wherein,

   The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can perform any one of claims 1-4 The vehicle control method.
9. A terminal that includes:

   at least one processor; and

   a memory communicatively connected to the at least one processor; wherein,

   The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform vehicle control as claimed in claim 5 method.
10. A vehicle control system includes the APA controller as claimed in claim 8 and the terminal as claimed in claim 9.