[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Next Article in Journal
16 Ch × 200 GHz DWDM-Passive Optical Fiber Sensor Network Based on a Power Measurement Method for Water-Level Monitoring of the Spent Fuel Pool in a Nuclear Power Plant
Next Article in Special Issue
Topological Navigation for Autonomous Underwater Vehicles in Confined Semi-Structured Environments
Previous Article in Journal
Keeping Track of Phaeodactylum tricornutum (Bacillariophyta) Culture Contamination by Potentiometric E-Tongue
Previous Article in Special Issue
Functional Self-Awareness and Metacontrol for Underwater Robot Autonomy
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 21
Issue 12
10.3390/s21124053
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Towards the Design and Implementation of an Image-Based Navigation System of an Autonomous Underwater Vehicle Combining a Color Recognition Technique and a Fuzzy Logic Controller

by
Yu-Hsien Lin
*,
Chao-Ming Yu
and
Chia-Yu Wu
Department of Systems & Naval Mechatronic Engineering, National Cheng-Kung University, Tainan City 70101, Taiwan
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(12), 4053; https://doi.org/10.3390/s21124053
Submission received: 27 April 2021 / Revised: 10 June 2021 / Accepted: 10 June 2021 / Published: 12 June 2021
(This article belongs to the Special Issue Intelligence and Autonomy for Underwater Robotic Vehicles)
Browse Figures
Figure 1
<p>Interior structure of the AUV.</p> "> Figure 2
<p>The system architecture of the AUV.</p> "> Figure 3
<p>Flow chart of the image-processing module.</p> "> Figure 4
<p>The procedure of image processing: (<b>a</b>) capturing the video from the AUV front camera; (<b>b</b>) conversion from the RGB color space to the HSV color space; (<b>c</b>) histogram equalization, binarization, and filter; (<b>d</b>) filter and morphology of dilation and erosion; and (<b>e</b>) obtaining the color features of the object.</p> "> Figure 5
<p>The ROI selections for (<b>a</b>) case A, (<b>b</b>) case B, (<b>c</b>) case C, (<b>d</b>) case D, (<b>e</b>) case E, (<b>f</b>) case F, (<b>g</b>) case G, and (<b>h</b>) case H, respectively.</p> "> Figure 6
<p>Definitions of the tracking control parameters values: (<b>a</b>) the benchmark, (<b>b</b>) the orientations, and (<b>c</b>) the distance.</p> "> Figure 7
<p>Relationship between the earth-fixed, body-fixed, and vehicle image coordinate systems.</p> "> Figure 8
<p>Relationship between the target’s image horizontal coordinate axis and the FLC yaw angle.</p> "> Figure 9
<p>Relationship between the target’s image vertical coordinate axis and the FLC pitch angle.</p> "> Figure 10
<p>Relationship between the target’s size within the image coordinates and the FLC propeller revolution speed.</p> "> Figure 11
<p>The recognition results of the image with the EKF.</p> "> Figure 12
<p>The exterior and the dimension of (<b>a</b>) the stability water tank, and (<b>b</b>) the towing tank.</p> "> Figure 13
<p>Schematic of the AUV and target object in the dynamic experiment.</p> "> Figure 14
<p>Environment of the dynamic experiment.</p> "> Figure 15
<p>The flow chart of the image-based navigation system in the AUV.</p> "> Figure 16
<p>Dynamic guidance experiment with a towing speed of 0.2 m/s. (<b>a</b>) Displacement trajectory on the image coordinates, (<b>b</b>) time series of the image coordinate Y−axis and AUV yaw angle, (<b>c</b>) time series of image coordinate Y−axis and AUV vertical rudder, (<b>d</b>) time series of the image coordinate Z−axis and the AUV vertical rudder, and (<b>e</b>) time series of the image coordinate surface area and the AUV propeller revolution speed.</p> "> Figure 17
<p>Dynamic guidance experiment with a towing speed of 0.4 m/s. (<b>a</b>) Displacement trajectory on the image coordinates, (<b>b</b>) time series of the image coordinate Y−axis and AUV yaw angle, (<b>c</b>) time series of image coordinate Y−axis and AUV vertical rudder, (<b>d</b>) time series of the image coordinate Z−axis and the AUV vertical rudder, and (<b>e</b>) time series of the image coordinate surface area and the AUV propeller revolution speed.</p> "> Figure 18
<p>Dynamic guidance experiment with a towing speed of 0.6 m/s. (<b>a</b>) Displacement trajectory on the image coordinates, (<b>b</b>) time series of the image coordinate Y−axis and AUV yaw angle, (<b>c</b>) time series of image coordinate Y−axis and AUV vertical rudder, (<b>d</b>) time series of the image coordinate Z−axis and the AUV vertical rudder, and (<b>e</b>) time series of the image coordinate surface area and the AUV propeller revolution speed.</p> "> Figure 19
<p>Dynamic guidance experiment with a towing speed of 0.8 m/s. (<b>a</b>) Displacement trajectory on the image coordinates, (<b>b</b>) time series of the image coordinate Y−axis and AUV yaw angle, (<b>c</b>) time series of image coordinate Y−axis and AUV vertical rudder, (<b>d</b>) time series of the image coordinate Z−axis and the AUV vertical rudder, and (<b>e</b>) time series of the image coordinate surface area and the AUV propeller revolution speed.</p> "> Figure 20
<p>Image coordinate system logs of the AUV in dynamic guidance experiments with towing speeds of 0.2, 0.4, 0.6, and 0.8 m/s.</p> "> Figure 21
<p>Statistical results of the dynamic guidance experiments in different towing speeds: the mean values and standard deviations of (<b>a</b>) <math display="inline"><semantics> <mrow> <mfenced close="|" open="|"> <mrow> <msub> <mi>V</mi> <mi>y</mi> </msub> <mo>−</mo> <msub> <mi>V</mi> <mrow> <mi>y</mi> <mo>,</mo> <mo> </mo> <mi>c</mi> </mrow> </msub> </mrow> </mfenced> </mrow> </semantics></math> and (<b>b</b>) <math display="inline"><semantics> <mrow> <mfenced close="|" open="|"> <mrow> <msub> <mi>V</mi> <mi>z</mi> </msub> <mo>−</mo> <msub> <mi>V</mi> <mrow> <mi>z</mi> <mo>,</mo> <mo> </mo> <mi>c</mi> </mrow> </msub> </mrow> </mfenced> </mrow> </semantics></math>.</p> ">
Versions Notes

Abstract

:
This study proposes the development of an underwater object-tracking control system through an image-processing technique. It is used for the close-range recognition and dynamic tracking of autonomous underwater vehicles (AUVs) with an auxiliary light source for image processing. The image-processing technique includes color space conversion, target and background separation with binarization, noise removal with image filters, and image morphology. The image-recognition results become more complete through the aforementioned process. After the image information is obtained for the underwater object, the image area and coordinates are further adopted as the input values of the fuzzy logic controller (FLC) to calculate the rudder angle of the servomotor, and the propeller revolution speed is defined using the image information. The aforementioned experiments were all conducted in a stability water tank. Subsequently, the FLC was combined with an extended Kalman filter (EKF) for further dynamic experiments in a towing tank. Specifically, the EKF predicts new coordinates according to the original coordinates of an object to resolve data insufficiency. Consequently, several tests with moving speeds from 0.2 m/s to 0.8 m/s were analyzed to observe the changes in the rudder angles and the sensitivity of the propeller revolution speed.

1. Introduction

With the mature development of marine engineering, underwater technology has gradually become significant. The University of Washington constructed the first self-propelled underwater research vehicle (SPURV) for the research of underwater diffusion effects and acoustic transmission [1]. Underwater vehicles have gradually become autonomous, and more precise missions are possible with computers and electronic technologies. Allen, et al. [2] developed a new generation AUV through the reduction of cost and volume, and these have been successfully applied in scientific research and military contexts. Due to the influence of the unknown and uncertain factors of pressure, light, and ocean currents in the deep-sea environment, underwater technicians encounter safety threats and other difficulties in conducting missions.
In deep water where human missions are not possible, AUVs equipped with machine vision processing technology can be applied. Machine vision processing technology mainly employs cameras and computers to analyze images and provide control information of the AUV drive system. There have been numerous research studies on the application of an AUV equipped with machine vision processing technology, such as the detection of underwater man-made structures and pipeline detection [3,4,5,6,7,8,9,10], auxiliary sonar image navigation [11,12], simultaneous localization and mapping (SLAM) [13,14,15,16,17], obstacle avoidance [18,19], identifying and tracking the habitats of sea animals [20], underwater docking systems [21,22,23,24,25,26,27], and object tracking [28,29,30,31,32,33,34,35,36].
Thereupon, AUV navigation has become a major challenge in the research field. Particularly, path planning and object tracking are critical for the successful operation of AUVs. There have been various research studies discussing AUV navigation technologies [37,38,39,40,41,42]. Specifically, optical guidance is conducted through the visual system consisting of charge-coupled device (CCD) cameras and frame grabbers [21]. In addition, the processed images contain much information on the environment [43]. Compared with other underwater sensors, optical devices are generally considered to be less expensive but provide abundant information. Bazeille, et al. [44] identified underwater objects through the detection of all image colors known to be compatible. Yu, et al. [45] demonstrated that color extraction could be used for the detection of artificial underwater boundary markers and studied multiple colors that can be observed in underwater environments.
The purpose of this study is to develop an AUV underwater image-based navigation system with a combination of color recognition and fuzzy logic controllers (FLCs). In the experiment, a red ball is considered as an underwater object, and a dual-lens camera is used to capture images for processing. Since environmental factors (such as turbidity and brightness) can cause errors in image recognition, binarization can be used to separate objects from backgrounds. The contours can then be completed through noise removal and morphology. The image information obtained provides the input values of the FLC, which are used to control servo motors and propellers in this study. Eventually, an extended Kalman filter (EKF) would be integrated into the control system and be used to realize the data estimation of the dynamic guidance experiment.
For the remainder of this paper, Section 2 illustrates the AUV exterior design. Section 3 introduces the recognition of the image-processing method used in this study. In Section 4, the image-based navigation system is discussed, and the design and parameter adjustment methods for the FLC are explained. Section 5 verifies the AUV image-based navigation system through experimental results and analysis. Section 6 provides conclusions and discussion regarding the aforementioned results.

2. AUV Design and Structure

The exterior design of the AUV used in this study is a torpedo type with a length of 1.8 m and a diameter of 0.17 m. The system architecture of the AUV is described in Section 2.1, Section 2.2 introduces the image-processing module as the main sensing input source, and Section 2.3 describes the function of the motion control module.

2.1. System Structure of the AUV

The interior structure of the AUV, such as the drive mechanism and various control components, comprises three parts: the bow, the parallel middle body, and the stern. The bow is installed with the image system and LED apertures. The parallel middle body contains the attitude and heading reference system (AHRS), electric power equipment, communication equipment, and controllers. The stern section has a propeller and four sets of independent rudders. The AHRS module used in this study includes a multi-axis magnetometer, gyroscope, and accelerometer. It can perceive the AUV’s pitch, roll, and yaw motion variables in space; an ultra-slim and fanless PC is used as the main controller for the integration of all systems, as shown in Figure 1. On the other hand, the AUV architecture is presented in Figure 2. The aim of this study is to integrate these image-processing methods into the AUV for realizing dynamic and real-time visual tracking. Since the identification and control system considered in the AUV is subject to the power consumption and the performance of the processor, it would be more efficient to use the present methods than complex deep-learning methods.

2.2. Image-Processing Module

In this study, the dual-lens camera installed at the front of the bow is adopted to frontal images acquisition. For close-range detection, the image information is captured by this device effectively. Related image information can be transmitted back to the main control system for processing, and object tracking can be conducted through the operation and power system. A dual-lens camera is used in the image-processing module (resolution is 1280 × 780). Particularly, the camera is suitable for particular conditions (when additional light sources are difficult to use or light sources are insufficient), featuring a low-illuminance lens. Thus, it makes the AUV have visual functions and can carry out visualization tasks. Insufficient light in underwater environments influences the precision of the visual recognition. For this purpose, an auxiliary LED light source is installed outside the bow in this study. The LED light has directionality and can increase centrality. In addition, brightness can be adjusted for comparing the recognition efficiency in various lighting conditions.

2.3. Motion Control Module

Due to the influences of the AUV’s weight and resistance, the motor torque and propeller revolution speed required consideration. However, a motor with a high propeller revolution speed and low torque may fail to propel the vehicle due to excessive resistance. Therefore, a motor with a high torque and low propeller revolution speed is selected to ensure the vehicle’s ability to move underwater. In the AUV, a brushless DC (BLDC) motor is selected with an encoder, and four sets of independent servomotors are selected for the driving force of the AUV motion. The encoder plays a crucial role in the motion control module. It can track the revolution motion in real time and obtain accurate feedback data. The output signals are directly converged into digital signals to angular position and sent back to the controller via an RS232 cable.

3. Recognition of Image Features

Images captured with cameras are 3D images in RGB color space; R stands for red, G for green, and B for blue. Each color has a value range of positive integers from 0 to 255, forming the color of each pixel in the image. The RGB color space is an additive color system [46], where the RGB correlation in the pixel can be too high and the difference value of the color changes caused by brightness is more challenging to express. Typically, images captured by photosensitive components, such as cameras, are easily affected by ambient brightness changes, resulting in calculation errors. Notably, light in underwater environments is subject to strong changes and effects. The HSV color space is selected for this study to reduce the effects caused by changes in ambient light sources and to increase the resolution. The HSV color space can be converted from the RGB color space as shown in Equations (1)–(3). As explained by Smith in 1978 [47], H in the HSV color space stands for hue, with the 360 degrees of the circle in the cylindrical coordinates as the value range, and all colors are arranged on this circle—for example, yellow is approximately 60°; S denotes saturation and has a range of 0–100%—smaller values indicate lower saturation and that the displayed color is close to white; and V refers to value and has a range of 0–100%, indicating the brightness displayed by the color—the smaller the value is, the darker the color is. This color space is unique in that the three visual vectors—hue, saturation, and value—are highly independent. Moreover, the differences caused by changes in the ambient light source can be defined and distinguished. Therefore, this color space is especially suitable for situations that require the independent identification of hue, saturation, and value.
H = 0 ° ,       i f   R , G , B m a x = R , G , B m i n 60 ° · G − B R , G , B m a x − R , G , B m i n + 0 ° ,       i f   R , G , B m a x = R , G ≥ B 60 ° · G − B R , G , B m a x − R , G , B m i n + 360 ° , i f   R , G , B m a x = R , G < B 60 ° · B − R R , G , B m a x − R , G , B m i n + 120 ° ,       i f   R , G , B m a x = G 60 ° · R − G R , G , B m a x − R , G , B m i n + 240 ° ,       i f   R , G , B m a x = B
S = 0 , i f   R , G , B m a x = 0 R , G , B m a x − R , G , B min R , G , B m a x , o t h e r w i s e
V = R , G , B m a x
where R , G , B m a x is the largest value of R, G, and B in each pixel; and   R , G , B m i n is the smallest value of R, G, and B in each pixel.
In order to achieve the primary purpose of object recognition and AUV tracking, the target characteristics captured by the cameras are implemented in the navigation after analysis through the use of AUV and recognition technology during underwater object recognition. The major challenge in using image-processing technology for detection is that objects appear differently underwater compared to in air. Consequently, in this study, some characteristics in the images were enhanced for the computer to process the information. A red ball was designated as the target because red is extremely observable in underwater environments. Since the HSV color space can be used to reduce the effects caused by changes in ambient light sources and to increase the resolution, the image-processing technology is still able to recognize the object correctly in a similarly colored background. In addition, the respective color attenuation is small with increasing distance. Therefore, some processing methods are adopted to eliminate the influence of noise (attenuation and scattering of light by suspended particles) and improve detection. The image-processing methods of this study include image capture, color space conversion, image binarization, filter, and morphology operation [48]. The advantage of these image-processing methods is to obtain both the color and morphological information of the object in an efficient way for further processing. The calculation procedure and the procedure for the proposed methods are shown in Figure 3.
The first step in the image recognition procedure is obtaining a continuous image from the image module in the fore part, as shown in Figure 4a. Subsequently, the image color space is converted from an RGB color space to an HSV color space, and the brightness dimension of the image is calculated as shown in Figure 4b. Next, the histogram equalization, image binarization, and image filter process are performed [49], as shown in Figure 4c. After the conversion from the RGB color space to the HSV color space is completed, the values of HSV can be designed as the threshold in the binarization according to the environmental change. When it is implemented in the real environment, the threshold values of HSV can be fine-tuned and designed to improve the recognition ability. Subsequently, the erosion and dilation basic methods are applied morphologically [50], as shown in Figure 4d.
As shown in Figure 4e, the features of the object could be identified using color recognition and selection of the region of interest (ROI) area. Specifically, the features are firstly detected by using the Canny edge detector [51], and then they are determined by adopting the Teh-Chin Algorithm [52] for the ROI processing of the target. By calculating the ROI area center of the object on the AUV image coordinate and the area of the selection, the rate between sizes of the object can be estimated [53], and then the AUV is able to complete the calculation process for real-time dynamic recognition of the image features. Eventually, the image coordinate and the ROI area can be defined as the object-tracking control parameters, i.e., the orientations and the distance [51].
In order to discuss the robustness of the present image-processing method, an experiment with similar objects and similarly colored backgrounds was implemented, as shown in Table 1 and Figure 5a–h. In the experiment, the thresholds of the HSV values when identifying the objects in similarly colored backgrounds were analyzed. It is apparent that the image-processing methods in this study are able to identify the object by adjusting the thresholds of the HSV values. However, if there are two different objects of the same color, e.g., a sphere and an ellipsoid, the image-processing methods would select both of them as the ROI simultaneously. In a future study, a more robust image-processing method, such as a Region-Based Convolutional Neural Network (R-CNN) [54], would be considered to recognize the features of the different morphologies in the same color.

4. Image-Based Navigation System

Since the underwater image-based navigation system is a combination of color recognition and fuzzy controllers, the details of the technologies would be introduced in this section.

4.1. Relationships between Different Coordinate Systems

The AUV control coordinate system is defined using the body-fixed coordinate system, and the trajectory relationships of the AUV in space are defined using the earth-fixed coordinate system. Figure 6 shows the vehicle image coordinate system with the red ball as an object. The coordinate value and area value O V y , V z , A r e a of the object within the image coordinates were calculated to define the relative horizontal and vertical relationship between the AUV and the object and to define the relative distance between the AUV and the object.
Figure 6a shows the benchmark visual image when the relative distance of the target matches the requirements, at which the forward speed and the heading angle are set at zero. Compared to the visual image in Figure 6a, the visual image shown in Figure 6b can be defined as the operation strategy of turning left and down when the object is in the lower-left corner of the AUV. On the other hand, the visual image shown in Figure 6c can be identified as the operation strategy with backward speed when the area of the object is larger than that in Figure 6a.
Furthermore, the relative relationship and the distance between the target object and the AUV in space can be estimated, as shown in Figure 7. The earth-fixed coordinates of target object T are set as T x , y , z ; the image target O, which is projected to and captured by the camera inside the AUV, is defined as O V y , V z in the image coordinate system; and the relationship is illustrated in Equation (4), with O a r e a and T s i z e (defined in Equation (5)) representing the area size of the target object in the earth-fixed coordinate system and vehicle image coordinate system, respectively, where F d is the focal length of the vehicle camera and F s is a fixed scale factor.
T x , y , z   ⇒ O V y , V z = i j = F d / x y z
T s i z e = F s · O a r e a

4.2. Fuzzy Logic Control (FLC)

The navigation drive and propulsion control use fuzzy logic control (FLC) [55], processing nonstable signals with a simple mathematical model based on nonlinear control logic. When unstable signals cannot provide accurate values for performing the control calculations, a defined value range model with high tolerance is used as the constraint condition to form fuzzy controls. In the application of digital computer controls, the advantages of FLC advantages are its similarity to human thinking models and the ability to implement gentle processing protocols for unstable signal sources [56].
Fuzzy sets are the theoretical basis of FLC, and they are defined by the characteristic function as below:
S A x = 1       ,     i f   x ∈   A 0       ,     i f   x ∉   A
where x is expressed as an element, and S A x is the characteristic function of A.
Zadeh [55] quantified and calculated fuzzy sets from the precise value of the traditional set through a predefined membership function. When membership function of fuzzy set A defined into U A x is discrete, then fuzzy set A can be expressed as Equation (7). If membership function U A x is continuous, then A is expressed as Equation (8).
A = U A x 1 x 1 + U A x 2 x 2 + … + U A x n x n
A = ∫ 0 ∞ U A x x
where U is an all fuzzy set.
In this study, fuzzy logic is applied on three groups of single-in, single-out controllers. Fuzzy inference uses generalized modus ponens [57] and establishes a fuzzy rule base by using the Tsukamoto fuzzy model through the IF-THEN inference rule, as shown in Equation (9) [58]. In the system, when the input value is imported into the fuzzy controller, the monotonic membership function corresponding to the fuzzy rule library is subjected to the defuzzifier and is mapped onto the value range to obtain the control coefficients, where C 1 , C 2 , and C 3 represent the monotonic membership functions.
I f   a   i s   s m a l l ,   t h e n   b   i s   C 1 I f   a   i s   m e d i u m ,   t h e n   b   i s   C 2 I f   a   i s   l a r g e ,   t h e n   b   i s   C 3
The maneuverability of the AUV is a coupled, complex integrated system that is nonlinearly coupled, and numerous undefined coefficients are present in the differential equation of its six degrees of motion. The advantages of FLC in AUV applications, as highlighted by Smith, et al. [59], are integrated into the motion drive and propulsion control system to form the fuzzy control framework of the yaw, pitch angle, and the propeller revolution speed.

4.3. Servomotor Controller Design

To enable stable horizontal control of the AUV at depths, its ballast arrangement is in a state of neutral buoyancy deviated toward positive buoyancy (0.8 kgf). Therefore, only the control of the vertical rudder plane angle ( δ V ) is discussed. To maintain the target at the image centre, the target detected by the camera is calculated by the controller. The centre is defined at 0° of the servomotor, and every graduation is 5°. The maximum angle change is ±30°. When the centroid of the object is on the right side of the image, the heading control is at a negative value.
The control principle of this system dominated the servomotor. When the target is on the right side of the image, the centre distance is subtracted from the coordinate as the error value of the FLC. The area where the target object is displayed on the vehicle camera image coordinate   O V y is the FLC input variable for defining the following six fuzzy values: strongleft, left, middle, right, strongright, and Vright.
The FLC calculation and control target is the AUV yaw angle ( B N ) , ranging from −30° to 30°; the six defined output rudder controls are strongleft, left, middle, right, strongright, and Vright. The FLC framework of the yaw angle is presented in Figure 8, Equation (10), and Equation (11), respectively.
I f   O V y < ( 0.1 V y , m a x )   i s   s t r o n g l e f t F L C i n       t h e n   s t r o n g l e f t F L C o u t   i s − Y a r I f   ( 0.1 V y , m a x ) ≤ O V y < ( 0.3 V y , m a x )   i s   l e f t F L C i n       t h e n   l e f t F L C o u t   i s − Y a r + 2 Y a r / 5 I f   ( 0.3 V y , m a x ) ≤ O V y < ( 0.5 V y , m a x )   i s   m i d d l e F L C i n         t h e n   k e e p F L C o u t   i s − Y a r + 4 Y a r / 5 I f   ( 0.5 V y , m a x ) ≤ O V y < ( 0.7 V y , m a x )   i s   r i g h t F L C i n       t h e n   r i g h t F L C o u t   i s − Y a r + 6 Y a r / 5 I f   ( 0.7 V y , m a x ) ≤ O V y < ( 0.9 V y , m a x )   i s   s t r o n g r i g h t F L C i n       t h e n   s t r o n g r i g h t F L C o u t   i s − Y a r + 8 Y a r / 5 I f   ( 0.9 V y , m a x ) ≤ O V y   i s   V r i g h t F L C i n       t h e n   V r i g h t F L C o u t   i s   Y a r
B N = − Y a r   ,   i f   O V y < ( 0.1 V y , m a x )   − Y a r + 2 Y a r / 5   ,   i f   ( 0.1 V y , m a x ) ≤ O V y < ( 0.3 V y , m a x )   − Y a r + 4 Y a r / 5   ,   i f   ( 0.3 V y , m a x ) ≤ O V y < ( 0.5 V y , m a x )   − Y a r + 6 Y a r / 5   ,   i f   ( 0.5 V y , m a x ) ≤ O V y < ( 0.7 V y , m a x )   − Y a r + 8 Y a r / 5   ,   i f   ( 0.7 V y , m a x ) ≤ O V y < ( 0.9 V y , m a x )   Y a r   ,   i f   ( 0.9 V y , m a x ) ≤ O V y  
where ±   Y a r is the maximum range value that the AUV vertical rudders are able to control.
Regarding the vehicle’s pitch angle, on the basis of the image feedback framework and by using the fuzzy logical controller FLC, the input variable calculates and controls the AUV pitch angle B M   according to the image coordinates of the target object on the image vertical coordinate T V z . Six input fuzzy values are defined: strongup, up, middle, down, strongdown, and Vdown. The vehicle’s pitch angle is the calculation and control target, ranging from −30° to positive 30°; the six defined fuzzy values are strongup, up, middle, down, strongdown, and Vdown. The pitch angle FLC framework is illustrated in Figure 9, and expressed in Equation (12).
B M = − P a r   ,   i f   O V z < ( 0.1 V z , m a x )   − P a r + 2 P a r / 5   ,   i f   ( 0.1 V z , m a x ) ≤ O V z < ( 0.3 V z , m a x )       − P a r + 4 P a r / 5   ,   i f   ( 0.3 V z , m a x ) ≤ O V z < ( 0.5 V z , m a x )   − P a r + 6 P a r / 5   ,   i f   ( 0.5 V z , m a x ) ≤ O V z < ( 0.7 V z , m a x )     − P a r + 8 P a r / 5   ,   i f   ( 0.7 V z , m a x ) ≤ O V z < ( 0.9 V z , m a x )   P a r   ,   i f   ( 0.9 V z , m a x ) ≤ O V z  
where ±   P a r is the maximum range value that the AUV horizontal rudders are able to control.

4.4. Propeller Controller Design

The control method of the AUV’s propeller revolution speed depends on the area of the target. After the target is captured in the image, the area is calculated to define the revolution speed configuration. The judgment method is that the larger the area is, the lower the revolution speed is. By contrast, the smaller the area is, the higher the revolution speed is.
The FLC input variable calculates and controls the AUV propeller revolution speed ( P s ) according to the target object coordinate T V z and size T s i z e displayed on the image coordinate. Six input fuzzy values are defined: small, Lsmall, middle, Lbig, big, and Vbig. The propeller revolution speed ( P s ) is the output variable, with a range between P s − i n v e r s e   and P s , m a x ; the six defined fuzzy values are fast, Lfast, keep, Lslow, slow, and safe. The FLC framework of the propeller revolution speed is presented in Figure 10 and can be expressed in Equation (13).
P s = P s , m a x   , i f   O a r e a < 0.1 O a r e a , m a x   o r   O V z < ( 0.1 V z , m a x )   P s − i n v e r s e   , i f   O s i z e ≥ 0.9 O a r e a , m a x     o r   ( 0.1 V z , m a x ) ≤ O V z < ( 0.3 V z , m a x )   P s , m a x − P s , m a x / 5   ,   i f 0.1 O a r e a , m a x ≤ O a r e a < 0.3 O a r e a , m a x   o r   ( 0.3 V z , m a x ) ≤ O V z < ( 0.5 V z , m a x   )   P s , m a x − 2 P s , m a x / 5   ,   i f   0.3 O a r e a , m a x ≤ O a r e a < 0.5 O a r e a , m a x   o r   ( 0.5 V z , m a x ) ≤ O V z < ( 0.7 V z , m a x )   P s , m a x − 3 P s , m a x / 5   ,   i f   0.5 O a r e a , m a x ≤ O a r e a < 0.7 O a r e a , m a x   o r   ( 0.7 V z , m a x ) ≤ O V z < ( 0.9 V z , m a x )   P s , m a x − 4 P s , m a x / 5   ,   i f   0.7 O a r e a , m a x ≤ O a r e a < 0.9 O a r e a , m a x     o r   ( 0.9 V z , m a x ) ≤ O V z  
where P s , m a x is the maximum propeller revolution speed of the AUV in the towing tank experiment, and P s − i n v e r s e   is the reverse propeller revolution speed when the safety distance control mechanism is activated.

4.5. Extended Kalman Filter (EKF)

KFs [60] are based on linear assumptions and are estimated based on the state estimation value of the previous moment and the state observation value of the current moment. Because noise or unstable measurement values need not be eliminated and processed in advance, estimations can be performed with recursive data processing algorithms in the time domain. Furthermore, observation data and historical data need not be logged. However, through each regression, the predicted value and the observed value are weighted to obtain an estimate close to the dynamic target.
Because the KF is an estimate based on linear assumptions, divergence exists when it is used on nonlinear systems. Therefore, many studies have expanded the application of KFs to nonlinear systems, and the core processing protocol involves the linearization of nonlinear systems. Sorenson and Stubberud [61] proposed the EKF, and Alspach and Sorenson [62] proposed Bayesian estimation, which linearizes nonlinear systems by using a Gaussian distribution approximation. Regarding other methods for linearizing nonlinear systems, Chen [63] argued that an expanded Taylor series can be used to approximate nonlinear systems, and the nonlinear system can be transformed into a linearized system by simplifying or ignoring the higher order terms of the nonlinear system. Although the EKF uses approximation methods to solve nonlinear system problems, it conversely lowers the precision of the control model and drastically increases the amount of data processing. Furthermore, when highly variable coefficients are present in the nonlinear system, the estimation results diverge and result in estimation control failure.
Since the integration of the camera measurement and the controller operation inevitably causes signal delays, the system performance would be improved by using the EKF. In this study, the purpose of the dynamic guidance experiment is to understand both the qualitative and quantitative results of the system performance. Therefore, the red ball is selected as the target in the experiment due to its size and geometry. The free movements of the red ball are tracked with the control of the propeller to ensure that the target remained in the frame. The coordinates of the target are read to estimate its next coordinates. Figure 11 shows the underwater image-recognition results. The coordinates are documented at a speed of 15 frames per second, and the largest area in the image is identified and framed with a rectangle. The centre of the rectangle represents the position of the object. The blue circle is the coordinate of the actual object, and the green circle is the coordinate predicted by the EKF. Differences are observed between the actual and predicted positions.

5. Experimental Results and Discussion

The experiments were conducted in the stability water tank and towing tank at Department of Systems and Naval Mechatronic Engineering of National Cheng Kung University (NCKU). The underwater image recognition tests are conducted primarily in the stability water tank. The experiments of the FLC system were carried out in the towing tank, and the comparison results for the EKF were added for verification. In this study, underwater image processing was conducted in the stability water tank by using a dual-lens camera with varying brightness values. The area and coordinate data were calculated by processing the image information in various conditions. The recognition results were matched with an FLC and EKF to be included in the AUV control system. The underwater object tracking of the AUV was adjusted dynamically with the image-recognition results.

5.1. Experimental Environment and Equipment

The stability water tank of NCKU is 6 m in length, 2 m in width, and 1.3 m in depth. The towing tank of NCKU is 165 m in length and 8 m in width, and the working water depth is 4 m. The exterior and the dimensions of the stability water tank and the towing tank are indicated in Figure 12a,b, respectively.

5.2. Experimental Procedure

This study was conducted in both a stability water tank and a towing tank. The stability water tank was used to test the vehicle’s attitude adjustment, buoyancy balance, water-tightness, payload control, and examined for image-recognition processing. Finally, the test results were introduced into the AUV for the dynamic experiment in the towing tank. The target used for the underwater recognition was a red ball of 24 cm in diameter, fastened to the support frame, and installed securely onto the towing carriage. The object was submerged in the water; the top of the object was 0.14 m away from the water surface, and the minimal safety distance between the object and the AUV was 0.18 m; the experimental environment is presented in Figure 13 and Figure 14. During the experiment, the movement of the towing carriage drove the object, and the image-based navigation system was tested under various speed conditions of the object.
During the tests in the towing tank, to eliminate the noise generated in the measurement process and achieve accurate estimation results at the current position, the influence of the EKF on the predicted coordinate results is further discussed in this study. The flow chart of the image-based navigation system is introduced in Figure 15.
In the flowchart, the object mapped onto the image coordinate system O V y , V z can be calculated when the ROI is selected. Subsequently, the information obtained from the ROI would enter the procedure for the propeller and rudder controls by using the FLC. In order to improve performances of the propeller and rudder controls, the output control values, i.e., yaw angle ( B N ) , pitch angle B M , and propeller revolution speed ( P s ) of the FLC, in the preceding 15 s would be adopted as the input data of the EKF, and the AUV motion control can be performed by using the predicted data of the EKF.

5.3. Experimental Results and Data Analysis

Figure 16, Figure 17, Figure 18 and Figure 19 present the AHRS measurements when the object moved at speeds of 0.2, 0.4, 0.6, and 0.8 m/s during the dynamic experiments. Figure 16a–e presents the results of the dynamic experiment with the object moving at 0.2 m/s; in order, the figure presents the relationship of the object’s displacement trajectory in the vehicle’s image coordinate system with that in the earth-fixed coordinates system (Figure 16a), the time series of the target object on the vehicle’s image coordinate system Y-axis and the AUV’s yaw angle on the body-fixed coordinates system (Figure 16b), the time series of the target object on the vehicle image coordinate system Y-axis and the AUV vertical rudder (Figure 16c), the time series of the target object on the vehicle image coordinate system Z-axis and the AUV’s pitch angle on the body-fixed coordinates system (Figure 16d), and the time series of the target’s area size on the vehicle image coordinate system and the AUV propeller revolution speed (Figure 16e).
The results in Figure 16a indicate that, except for the displacement trajectory exhibiting greater fluctuations as a result of the towing carriage decelerating in the final stage, the target object is controlled in the area range defined by the image coordinate O V y , V z . Figure 16b demonstrates consistent trends between the object on the vehicle image coordinate system Y-axis and the AUV yaw angle on the body-fixed coordinate system; this indicates that the image-based navigation system can track the target object’s horizontal movement changes in real time. Figure 16c reveals that the target on the vehicle image coordinate system Y-axis and the AUV vertical rudder are almost synchronized, indicating that the motion controller can respond to changes in the vehicle image coordinate system Y-axis in real time. Figure 16d demonstrates consistent trends between the object on the vehicle image coordinate system Z-axis and the AUV pitch angle on the body-fixed coordinate system; this indicates that the image-based navigation system can track the object’s vertical movement changes in real time. Figure 16e demonstrates exceedingly similar trends between the AUV propeller revolution speed and changes in the area size of the target object, indicating that the AUV propeller revolution speed can reflect surface area changes in the vehicle image coordinate system in real time.
Because of the acceleration of the object’s moving speed, as illustrated in Figure 17a, Figure 18a, and Figure 19a, the object exhibited greater fluctuations in displacement, but these are still controlled within the area range defined by the image coordinate O V y , V z . Figure 17b, Figure 18b, and Figure 19b indicate gradual increasing changes to the object on the vehicle image coordinate system Y-axis and the AUV yaw angle on the body-fixed coordinate system, compared to Figure 16b. Similarly, the time series of vertical rudders in Figure 17c, Figure 18c, and Figure 19c exhibit consistency with the changes along the vehicle image coordinate system Y-axis. Furthermore, with the acceleration of the towing carriage, the change fluctuations in the target object along the vehicle image coordinate system Z-axis and the AUV’s pitch angles in the body-fixed coordinate system also increased (Figure 17d, Figure 18d, and Figure 19d). A comparison of Figure 16e, Figure 17e, Figure 18e, and Figure 19e demonstrates little difference in propeller revolution speed; this is because the moving speed of the target objects is relatively stable, and as a result, surface area changes in the vehicle image coordinate system are also more stable.
Figure 20 presents the image coordinate system logs of the AUV in the dynamic guidance experiments with towing speeds of 0.2, 0.4, 0.6, and 0.8 m/s. The experimental results indicated that, under these four towing speeds, the AUV could track the object in real-time and maintain the target object in the vehicle image coordinate system. With the acceleration of the towing speed, the experimental data gradually became more centralized, especially in the image coordinate Y-axis ( V y ). This finding highlights that the moving speed of the target object is a factor affecting the AUV tracking efficiency.
Figure 21a,b present the statistical results of V y − V y ,   c and V z − V z ,   c in the dynamic guidance experiments with towing speeds of 0.2, 0.4, 0.6, and 0.8 m/s. It is found in Figure 21a that the standard deviations of V y − V y ,   c are 45.35, 26.83, 20.67, and 28.18 pixels. Furthermore, the standard deviations of V z − V z ,   c in Figure 21b are 25.09, 32.54, 39.89, and 28.11 pixels. Meanwhile, the results indicate that the mean values of V y − V y ,   c are 74.33, 51.97, 37.96, and 34.84 pixels, whereas the ones of V z − V z ,   c are 43.97, 45.99, 44.21, and 45.33 pixels. As the towing speed accelerates, the experimental data of V y − V y ,   c gradually become more centralized. However, the increase in the towing speed has less influence on the results of V z − V z ,   c . This is because the AUV, when it tracks the target object, can more easily maintain the target object at a safe distance at low speeds than at high speeds. However, this also slows down the real-time calibration of the yaw angle. The experimental results also reveal that the moving speed of the target object has little effect on the mean value of V z − V z ,   c and standard deviation; instead, the moving speed has a closer relationship with the changes in the target object’s size, and as a result, effects on the real-time calibration of the pitch angle are minuscule when the moving speed is constant.

6. Conclusions

This study presents a technique for image recognition and a vehicle motion control method based on an image-based navigation system and implements it in an AUV of independent development. With the combination of the image-processing methods and the effective motion control method, a dynamic guidance task of low power consumption and high efficiency for the AUV can be achieved. The data of the body-fixed coordinates are mapped using the image coordinates to define the correspondence of the distance and the orientation between the target and the AUV. The features of the object could be identified using color recognition and selection of the ROI area. By calculating the ROI area center of the object on the AUV image coordinates and the area of the selection, the relative relationship and distance between the object and the AUV can be estimated. Furthermore, the attitude and the propeller revolution speed of the AUV can be adjusted automatically through establishing the FLC rule base. It is verified that the FLC control of the AUV can be performed well by using the predicted data of the EKF. It is realized in the experiment that real-time recognition is controlled to achieve operational performance at a towing speed up to 0.8 m/s. Finally, autonomous dynamic control on the rudders and the propeller has been successfully verified by testing the maneuverability of the AUV in the towing tank. Compared with our previous study [48], the advantage of this study is that the features of the object could be identified using color recognition and selection of the region of interest (ROI) area. Meanwhile, the FLC control of the AUV can be performed well by using the predicted data of the EKF.

Author Contributions

Y.-H.L. is the principal investigator and C.-M.Y. and C.-Y.W. are the research assistants in this project. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Ministry of Science and Technology for a grant under Contract No. MOST 109-2221-E-006-100-MY2.

Data Availability Statement

Data sharing not applicable.

Acknowledgments

The authors would like to express their thanks to the Ministry of Science and Technology for a grant under Contract No. MOST 109-2221-E-006-100-MY2.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Widditsch, H. SPURV-The First Decade; Applied Physics Laboratory, University of Washington: Seattle, WA, USA, 1973. [Google Scholar]
  2. Allen, B.; Stokey, R.; Austin, T.; Forrester, N.; Goldsborough, R.; Purcell, M.; von Alt, C. REMUS: A small, low cost AUV; system description, field trials and performance results. In Proceedings of the Oceans’ 97 MTS/IEEE Conference, Halifax, NS, Canada, 6–9 October 1997; pp. 994–1000. [Google Scholar]
  3. Foresti, G.L. Visual inspection of sea bottom structures by an autonomous underwater vehicle. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2001, 31, 691–705. [Google Scholar] [CrossRef]
  4. Zhou, Z.; Shen, H.; Huang, H.; Zhou, H.; Wan, Z.; Wang, Z.; Xu, Y. Autonomous Underwater Pipeline Tracking Control Based on Visual Images. In Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia, 12–15 December 2018. [Google Scholar]
  5. Hallset, J.O. Simple vision tracking of pipelines for an autonomous underwater vehicle. In Proceedings of the 1991 IEEE International Conference on Robotics and Automation, Sacramento, CA, USA, 9–11 April 1991; pp. 2767–2772. [Google Scholar]
  6. Balasuriya, A.; Ura, T. Vision-based underwater cable detection and following using AUVs. In Proceedings of the OCEANS’02 MTS/IEEE, Biloxi, MI, USA, 29–31 October 2002; pp. 1582–1587. [Google Scholar]
  7. Inzartsev, A.V.; Pavin, A.M. AUV cable tracking system based on electromagnetic and video data. In Proceedings of the OCEANS 2008-MTS/IEEE Kobe Techno-Ocean, Kobe, Japan, 8–11 April 2008; pp. 1–6. [Google Scholar]
  8. Li, D.-j.; Chen, Y.-h.; Shi, J.-g.; Yang, C.-j. Autonomous underwater vehicle docking system for cabled ocean observatory network. Ocean Eng. 2015, 109, 127–134. [Google Scholar] [CrossRef]
  9. Khan, A.; Ali, S.S.A.; Meriaudeau, F.; Malik, A.S.; Soon, L.S.; Seng, T.N. Visual feedback–based heading control of autonomous underwater vehicle for pipeline corrosion inspection. Int. J. Adv. Robot. Syst. 2017, 14, 1729881416658171. [Google Scholar] [CrossRef] [Green Version]
  10. Allibert, G.; Hua, M.-D.; Krupínski, S.; Hamel, T. Pipeline following by visual servoing for Autonomous Underwater Vehicles. Control Eng. Pract. 2019, 82, 151–160. [Google Scholar] [CrossRef] [Green Version]
  11. Raaj, Y.; John, A.; Jin, T. 3D Object Localization using Forward Looking Sonar (FLS) and Optical Camera via particle filter based calibration and fusion. In Proceedings of the OCEANS 2016 MTS/IEEE Monterey, Monterey, CA, USA, 19–23 September 2016; pp. 1–10. [Google Scholar]
  12. Nikolovska, A. AUV based flushed and buried object detection. In Proceedings of the OCEANS 2015-Genova, Genova, Italy, 18–21 May 2015; pp. 1–5. [Google Scholar]
  13. Sáez, J.M.; Escolano, F. Entropy minimization SLAM using stereo vision. In Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain, 18–22 April 2005; pp. 36–43. [Google Scholar]
  14. Paull, L.; Saeedi, S.; Seto, M.; Li, H. AUV navigation and localization: A review. IEEE J. Ocean. Eng. 2013, 39, 131–149. [Google Scholar] [CrossRef]
  15. Jung, J.; Lee, Y.; Kim, D.; Lee, D.; Myung, H.; Choi, H.-T. AUV SLAM using forward/downward looking cameras and artificial landmarks. In Proceedings of the 2017 IEEE Underwater Technology (UT), Busan, Korea, 21–24 February 2017; pp. 1–3. [Google Scholar]
  16. Palomeras, N.; Carreras, M.; Andrade-Cetto, J. Active SLAM for autonomous underwater exploration. Remote Sens. 2019, 11, 2827. [Google Scholar] [CrossRef] [Green Version]
  17. Rahman, S.; Li, A.Q.; Rekleitis, I. Svin2: An underwater slam system using sonar, visual, inertial, and depth sensor. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 3–8 November 2019; pp. 1861–1868. [Google Scholar]
  18. Gaya, J.O.; Gonçalves, L.T.; Duarte, A.C.; Zanchetta, B.; Drews, P.; Botelho, S.S. Vision-based obstacle avoidance using deep learning. In Proceedings of the 2016 XIII Latin American Robotics Symposium and IV Brazilian Robotics Symposium (LARS/SBR), Recife, Brazil, 8–12 October 2016; pp. 7–12. [Google Scholar]
  19. Yan, Z.; Li, J.; Zhang, G.; Wu, Y. A real-time reaction obstacle avoidance algorithm for autonomous underwater vehicles in unknown environments. Sensors 2018, 18, 438. [Google Scholar] [CrossRef] [Green Version]
  20. Packard, G.E.; Kukulya, A.; Austin, T.; Dennett, M.; Littlefield, R.; Packard, G.; Purcell, M.; Stokey, R.; Skomal, G. Continuous autonomous tracking and imaging of white sharks and basking sharks using a REMUS-100 AUV. In Proceedings of the 2013 OCEANS-San Diego, San Diego, CA, USA, 23–27 September 2013; pp. 1–5. [Google Scholar]
  21. Park, J.-Y.; Jun, B.-h.; Lee, P.-m.; Oh, J. Experiments on vision guided docking of an autonomous underwater vehicle using one camera. Ocean Eng. 2009, 36, 48–61. [Google Scholar] [CrossRef]
  22. Teo, K.; An, E.; Beaujean, P.-P.J. A robust fuzzy autonomous underwater vehicle (AUV) docking approach for unknown current disturbances. IEEE J. Ocean. Eng. 2012, 37, 143–155. [Google Scholar] [CrossRef]
  23. Stokey, R.; Purcell, M.; Forrester, N.; Austin, T.; Goldsborough, R.; Allen, B.; von Alt, C. A docking system for REMUS, an autonomous underwater vehicle. In Proceedings of the Oceans’97. MTS/IEEE Conference, Halifax, NS, Canada, 6–9 October 1997; pp. 1132–1136. [Google Scholar]
  24. Jung, J.; Choi, J.; Lee, Y.; Choi, H.-T. AUV localization using depth perception of underwater structures from a monocular camera. In Proceedings of the OCEANS 2016 MTS/IEEE Monterey, Monterey, CA, USA, 19–23 September 2016; pp. 1–4. [Google Scholar]
  25. Lin, R.; Zhang, F.; Li, D.; Lin, M.; Zhou, G.; Yang, C. An Improved Localization Method for the Transition between Autonomous Underwater Vehicle Homing and Docking. Sensors 2021, 21, 2468. [Google Scholar] [CrossRef]
  26. Yan, Z.; Gong, P.; Zhang, W.; Li, Z.; Teng, Y. Autonomous underwater vehicle vision guided docking experiments based on L-shaped light array. IEEE Access 2019, 7, 72567–72576. [Google Scholar] [CrossRef]
  27. Myint, M.; Yonemori, K.; Lwin, K.N.; Yanou, A.; Minami, M. Dual-eyes vision-based docking system for autonomous underwater vehicle: An approach and experiments. J. Intell. Robot. Syst. 2018, 92, 159–186. [Google Scholar] [CrossRef]
  28. Balasuriya, A.; Ura, T. Autonomous target tracking by underwater robots based on vision. In Proceedings of the 1998 International Symposium on Underwater Technology, Tokyo, Japan, 17–17 April 1998; pp. 191–197. [Google Scholar]
  29. Ruiz, I.T.; Petillot, Y.; Lane, D.; Bell, J. Tracking Objects in Underwater Multibeam Sonar Images. Available online: http://home.eps.hw.ac.uk/~ceeyrp/WWW/Research/Petillot99.pdf (accessed on 27 May 2021).
  30. Fan, Y.; Balasuriya, A. Optical flow based speed estimation in AUV target tracking. In Proceedings of the MTS/IEEE Oceans 2001 An Ocean Odyssey (IEEE Cat. No. 01CH37295), Honolulu, HI, USA, 5–8 November 2001; pp. 2377–2382. [Google Scholar]
  31. Hildebrandt, M.; Kirchner, F. Imu-aided stereo visual odometry for ground-tracking auv applications. In Proceedings of the OCEANS’10 IEEE SYDNEY, Sydney, NSW, Australia, 24–27 May 2010; pp. 1–8. [Google Scholar]
  32. Kim, D.; Lee, D.; Myung, H.; Choi, H.-T. Object detection and tracking for autonomous underwater robots using weighted template matching. In Proceedings of the 2012 Oceans-Yeosu, Yeosu, Korea, 21–24 May 2012; pp. 1–5. [Google Scholar]
  33. Myint, M.; Yonemori, K.; Yanou, A.; Lwin, K.N.; Minami, M.; Ishiyama, S. Visual servoing for underwater vehicle using dual-eyes evolutionary real-time pose tracking. J. Robot. Mechatron. 2016, 28, 543–558. [Google Scholar] [CrossRef]
  34. Kumar, G.S.; Painumgal, U.V.; Kumar, M.C.; Rajesh, K. Autonomous underwater vehicle for vision based tracking. Procedia Comput. Sci. 2018, 133, 169–180. [Google Scholar] [CrossRef]
  35. Zhao, Q.; Tao, H. Object tracking using color correlogram. In Proceedings of the 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, Beijing, China, 15–16 October 2005; pp. 263–270. [Google Scholar]
  36. Asif, M.; Arshad, M.R. An Active Contour and Kalman Filter for Underwater Target Tracking and Navigation; INTECH Open Access Publisher Online: London, UK, 2006. [Google Scholar]
  37. Hagen, P.E.; Størkersen, N.; Marthinsen, B.-E.; Sten, G.; Vestgård, K. Rapid environmental assessment with autonomous underwater vehicles—Examples from HUGIN operations. J. Marine Syst. 2008, 69, 137–145. [Google Scholar] [CrossRef]
  38. Miller, P.A.; Farrell, J.A.; Zhao, Y.; Djapic, V. Autonomous underwater vehicle navigation. IEEE J. Ocean. Eng. 2010, 35, 663–678. [Google Scholar] [CrossRef]
  39. Leonard, J.J.; Bahr, A. Autonomous underwater vehicle navigation. In Springer Handbook of Ocean Engineering; Springer: Cham, Switzerland, 2016; pp. 341–358. [Google Scholar]
  40. Sahu, B.K.; Subudhi, B. Adaptive tracking control of an autonomous underwater vehicle. Int. J. Autom. Comput. 2014, 11, 299–307. [Google Scholar] [CrossRef] [Green Version]
  41. Pfingsthorn, M.; Rathnam, R.; Luczynski, T.; Birk, A. Full 3D navigation correction using low frequency visual tracking with a stereo camera. In Proceedings of the OCEANS 2016-Shanghai, Shanghai, China, 10–13 April 2016; pp. 1–6. [Google Scholar]
  42. Pérez-Alcocer, R.; Torres-Méndez, L.A.; Olguín-Díaz, E.; Maldonado-Ramírez, A.A. Vision-based autonomous underwater vehicle navigation in poor visibility conditions using a model-free robust control. J. Sensors 2016, 2016. [Google Scholar] [CrossRef] [Green Version]
  43. Papanikolopoulos, N.P.; Khosla, P.K. Adaptive robotic visual tracking: Theory and experiments. IEEE Transactions on Automatic Control 1993, 38, 429–445. [Google Scholar] [CrossRef]
  44. Bazeille, S.; Quidu, I.; Jaulin, L. Color-based underwater object recognition using water light attenuation. Intell. Serv. Robot. 2012, 5, 109–118. [Google Scholar] [CrossRef]
  45. Yu, S.-C.; Ura, T.; Fujii, T.; Kondo, H. Navigation of autonomous underwater vehicles based on artificial underwater landmarks. In Proceedings of the MTS/IEEE Oceans 2001. An Ocean Odyssey. (IEEE Cat. No. 01CH37295), Honolulu, HI, USA, 5–8 November 2001; pp. 409–416. [Google Scholar]
  46. Evans, R.M. Maxwell’s color photograph. Sci. Am. 1961, 205, 118–131. [Google Scholar] [CrossRef]
  47. Smith, A.R. Color gamut transform pairs. ACM Siggraph Comput. Graph. 1978, 12, 12–19. [Google Scholar] [CrossRef]
  48. Yu, C.-M.; Lin, Y.-H. Experimental Analysis of a Visual-Recognition Control for an Autonomous Underwater Vehicle in a Towing Tank. Appl. Sci. 2020, 10, 2480. [Google Scholar] [CrossRef] [Green Version]
  49. Woods, R.; Gonzalez, R. Real-time digital image enhancement. Proceedings of the IEEE 1981, 69, 643–654. [Google Scholar] [CrossRef]
  50. Gonzalez, R.C.; Woods, R.E. Digital Image Processing; Pearson: Upper Saddle River, NJ, USA, 2007; Volume 2, p. 1. [Google Scholar]
  51. Canny, J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 1986, 679–698. [Google Scholar] [CrossRef]
  52. Teh, C.-H.; Chin, R.T. On the detection of dominant points on digital curves. IEEE Trans. Pattern Anal. Mach. Intell. 1989, 11, 859–872. [Google Scholar] [CrossRef] [Green Version]
  53. Ahn, S.J.; Rauh, W.; Warnecke, H.-J. Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola. Pattern Recognit. 2001, 34, 2283–2303. [Google Scholar] [CrossRef]
  54. Gandhi, R. R-cnn, fast r-cnn, faster r-cnn, yolo—object detection algorithms. Available online: https://towardsdatascience.com/r-cnn-fast-r-cnn-faster-r-cnn-yolo-object-detection-algorithms-36d53571365e (accessed on 27 March 2021).
  55. Zadeh, L.A. Fuzzy sets. Inf. Control 1965, 8, 338–353. [Google Scholar] [CrossRef] [Green Version]
  56. Zadeh, L.A. Fuzzy logic= computing with words. In Computing with Words in Information/Intelligent Systems 1; Springer: Cham, Switzerland, 1999; pp. 3–23. [Google Scholar]
  57. Dubois, D.; Prade, H. Fuzzy logics and the generalized modus ponens revisited. Cybern. Syst. 1984, 15, 293–331. [Google Scholar] [CrossRef]
  58. Kiszka, J.B.; Kochańska, M.E.; Sliwińska, D.S. The influence of some fuzzy implication operators on the accuracy of a fuzzy model-Part I. Fuzzy Sets Syst. 1985, 15, 111–128. [Google Scholar] [CrossRef]
  59. Smith, S.; Rae, G.; Anderson, D. Applications of fuzzy logic to the control of an autonomous underwater vehicle. In Proceedings of the 1993 Second IEEE International Conference on Fuzzy Systems, San Francisco, CA, USA, 28 March–1 April 1993; pp. 1099–1106. [Google Scholar]
  60. Kalman, R.E. A New approach to Linear Filtering and Prediction Problems. J. Basic Eng. Mar. 1960, 82, 35–45. [Google Scholar] [CrossRef] [Green Version]
  61. Sorenson, H.W.; Stubberud, A.R. Non-linear filtering by approximation of the a posteriori density. Int. J. Control 1968, 8, 33–51. [Google Scholar] [CrossRef]
  62. Alspach, D.; Sorenson, H. Nonlinear Bayesian estimation using Gaussian sum approximations. IEEE Trans. Autom. Control 1972, 17, 439–448. [Google Scholar] [CrossRef]
  63. Chen, S. Kalman filter for robot vision: A survey. IEEE Trans. Ind. Electron. 2011, 59, 4409–4420. [Google Scholar] [CrossRef]
Sensors 21 04053 g001 550
Figure 1. Interior structure of the AUV.
Figure 1. Interior structure of the AUV.
Sensors 21 04053 g001
Sensors 21 04053 g002 550
Figure 2. The system architecture of the AUV.
Figure 2. The system architecture of the AUV.
Sensors 21 04053 g002
Sensors 21 04053 g003 550
Figure 3. Flow chart of the image-processing module.
Figure 3. Flow chart of the image-processing module.
Sensors 21 04053 g003
Sensors 21 04053 g004 550
Figure 4. The procedure of image processing: (a) capturing the video from the AUV front camera; (b) conversion from the RGB color space to the HSV color space; (c) histogram equalization, binarization, and filter; (d) filter and morphology of dilation and erosion; and (e) obtaining the color features of the object.
Figure 4. The procedure of image processing: (a) capturing the video from the AUV front camera; (b) conversion from the RGB color space to the HSV color space; (c) histogram equalization, binarization, and filter; (d) filter and morphology of dilation and erosion; and (e) obtaining the color features of the object.
Sensors 21 04053 g004
Sensors 21 04053 g005 550
Figure 5. The ROI selections for (a) case A, (b) case B, (c) case C, (d) case D, (e) case E, (f) case F, (g) case G, and (h) case H, respectively.
Figure 5. The ROI selections for (a) case A, (b) case B, (c) case C, (d) case D, (e) case E, (f) case F, (g) case G, and (h) case H, respectively.
Sensors 21 04053 g005
Sensors 21 04053 g006 550
Figure 6. Definitions of the tracking control parameters values: (a) the benchmark, (b) the orientations, and (c) the distance.
Figure 6. Definitions of the tracking control parameters values: (a) the benchmark, (b) the orientations, and (c) the distance.
Sensors 21 04053 g006
Sensors 21 04053 g007 550
Figure 7. Relationship between the earth-fixed, body-fixed, and vehicle image coordinate systems.
Figure 7. Relationship between the earth-fixed, body-fixed, and vehicle image coordinate systems.
Sensors 21 04053 g007
Sensors 21 04053 g008 550
Figure 8. Relationship between the target’s image horizontal coordinate axis and the FLC yaw angle.
Figure 8. Relationship between the target’s image horizontal coordinate axis and the FLC yaw angle.
Sensors 21 04053 g008
Sensors 21 04053 g009 550
Figure 9. Relationship between the target’s image vertical coordinate axis and the FLC pitch angle.
Figure 9. Relationship between the target’s image vertical coordinate axis and the FLC pitch angle.
Sensors 21 04053 g009
Sensors 21 04053 g010 550
Figure 10. Relationship between the target’s size within the image coordinates and the FLC propeller revolution speed.
Figure 10. Relationship between the target’s size within the image coordinates and the FLC propeller revolution speed.
Sensors 21 04053 g010
Sensors 21 04053 g011 550
Figure 11. The recognition results of the image with the EKF.
Figure 11. The recognition results of the image with the EKF.
Sensors 21 04053 g011
Sensors 21 04053 g012 550
Figure 12. The exterior and the dimension of (a) the stability water tank, and (b) the towing tank.
Figure 12. The exterior and the dimension of (a) the stability water tank, and (b) the towing tank.
Sensors 21 04053 g012
Sensors 21 04053 g013 550
Figure 13. Schematic of the AUV and target object in the dynamic experiment.
Figure 13. Schematic of the AUV and target object in the dynamic experiment.
Sensors 21 04053 g013
Sensors 21 04053 g014 550
Figure 14. Environment of the dynamic experiment.
Figure 14. Environment of the dynamic experiment.
Sensors 21 04053 g014
Sensors 21 04053 g015 550
Figure 15. The flow chart of the image-based navigation system in the AUV.
Figure 15. The flow chart of the image-based navigation system in the AUV.
Sensors 21 04053 g015
Sensors 21 04053 g016 550
Figure 16. Dynamic guidance experiment with a towing speed of 0.2 m/s. (a) Displacement trajectory on the image coordinates, (b) time series of the image coordinate Y−axis and AUV yaw angle, (c) time series of image coordinate Y−axis and AUV vertical rudder, (d) time series of the image coordinate Z−axis and the AUV vertical rudder, and (e) time series of the image coordinate surface area and the AUV propeller revolution speed.
Figure 16. Dynamic guidance experiment with a towing speed of 0.2 m/s. (a) Displacement trajectory on the image coordinates, (b) time series of the image coordinate Y−axis and AUV yaw angle, (c) time series of image coordinate Y−axis and AUV vertical rudder, (d) time series of the image coordinate Z−axis and the AUV vertical rudder, and (e) time series of the image coordinate surface area and the AUV propeller revolution speed.
Sensors 21 04053 g016
Sensors 21 04053 g017 550
Figure 17. Dynamic guidance experiment with a towing speed of 0.4 m/s. (a) Displacement trajectory on the image coordinates, (b) time series of the image coordinate Y−axis and AUV yaw angle, (c) time series of image coordinate Y−axis and AUV vertical rudder, (d) time series of the image coordinate Z−axis and the AUV vertical rudder, and (e) time series of the image coordinate surface area and the AUV propeller revolution speed.
Figure 17. Dynamic guidance experiment with a towing speed of 0.4 m/s. (a) Displacement trajectory on the image coordinates, (b) time series of the image coordinate Y−axis and AUV yaw angle, (c) time series of image coordinate Y−axis and AUV vertical rudder, (d) time series of the image coordinate Z−axis and the AUV vertical rudder, and (e) time series of the image coordinate surface area and the AUV propeller revolution speed.
Sensors 21 04053 g017
Sensors 21 04053 g018 550
Figure 18. Dynamic guidance experiment with a towing speed of 0.6 m/s. (a) Displacement trajectory on the image coordinates, (b) time series of the image coordinate Y−axis and AUV yaw angle, (c) time series of image coordinate Y−axis and AUV vertical rudder, (d) time series of the image coordinate Z−axis and the AUV vertical rudder, and (e) time series of the image coordinate surface area and the AUV propeller revolution speed.
Figure 18. Dynamic guidance experiment with a towing speed of 0.6 m/s. (a) Displacement trajectory on the image coordinates, (b) time series of the image coordinate Y−axis and AUV yaw angle, (c) time series of image coordinate Y−axis and AUV vertical rudder, (d) time series of the image coordinate Z−axis and the AUV vertical rudder, and (e) time series of the image coordinate surface area and the AUV propeller revolution speed.
Sensors 21 04053 g018
Sensors 21 04053 g019 550
Figure 19. Dynamic guidance experiment with a towing speed of 0.8 m/s. (a) Displacement trajectory on the image coordinates, (b) time series of the image coordinate Y−axis and AUV yaw angle, (c) time series of image coordinate Y−axis and AUV vertical rudder, (d) time series of the image coordinate Z−axis and the AUV vertical rudder, and (e) time series of the image coordinate surface area and the AUV propeller revolution speed.
Figure 19. Dynamic guidance experiment with a towing speed of 0.8 m/s. (a) Displacement trajectory on the image coordinates, (b) time series of the image coordinate Y−axis and AUV yaw angle, (c) time series of image coordinate Y−axis and AUV vertical rudder, (d) time series of the image coordinate Z−axis and the AUV vertical rudder, and (e) time series of the image coordinate surface area and the AUV propeller revolution speed.
Sensors 21 04053 g019
Sensors 21 04053 g020 550
Figure 20. Image coordinate system logs of the AUV in dynamic guidance experiments with towing speeds of 0.2, 0.4, 0.6, and 0.8 m/s.
Figure 20. Image coordinate system logs of the AUV in dynamic guidance experiments with towing speeds of 0.2, 0.4, 0.6, and 0.8 m/s.
Sensors 21 04053 g020
Sensors 21 04053 g021 550
Figure 21. Statistical results of the dynamic guidance experiments in different towing speeds: the mean values and standard deviations of (a) V y − V y ,   c and (b) V z − V z ,   c .
Figure 21. Statistical results of the dynamic guidance experiments in different towing speeds: the mean values and standard deviations of (a) V y − V y ,   c and (b) V z − V z ,   c .
Sensors 21 04053 g021
Table
Table 1. The RGB values of the background colors and the HSV threshold values of the objects.
Table 1. The RGB values of the background colors and the HSV threshold values of the objects.
TestObject R , G , B b a c k g r o u n d H , S , V l o w e r H , S , V u p p e r
ASphere(150, 20, 20)(1, 100, 100)(10, 240, 219)
B(200, 20, 20)(1, 100, 100)(10, 240, 219)
C(220, 20, 20)(1, 100, 100)(10, 240, 219)
D(255, 20, 20)(1, 100, 100)(10, 240, 219)
ESphere and Ellipsoid(150, 20, 20)(1, 100, 100)(10, 240, 219)
F(200, 20, 20)(1, 100, 100)(10, 240, 219)
G(220, 20, 20)(1, 100, 100)(10, 240, 219)
H(255, 20, 20)(1, 100, 100)(10, 240, 219)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Lin, Y.-H.; Yu, C.-M.; Wu, C.-Y. Towards the Design and Implementation of an Image-Based Navigation System of an Autonomous Underwater Vehicle Combining a Color Recognition Technique and a Fuzzy Logic Controller. Sensors 2021, 21, 4053. https://doi.org/10.3390/s21124053

AMA Style

Lin Y-H, Yu C-M, Wu C-Y. Towards the Design and Implementation of an Image-Based Navigation System of an Autonomous Underwater Vehicle Combining a Color Recognition Technique and a Fuzzy Logic Controller. Sensors. 2021; 21(12):4053. https://doi.org/10.3390/s21124053

Chicago/Turabian Style

Lin, Yu-Hsien, Chao-Ming Yu, and Chia-Yu Wu. 2021. "Towards the Design and Implementation of an Image-Based Navigation System of an Autonomous Underwater Vehicle Combining a Color Recognition Technique and a Fuzzy Logic Controller" Sensors 21, no. 12: 4053. https://doi.org/10.3390/s21124053

APA Style

Lin, Y. -H., Yu, C. -M., & Wu, C. -Y. (2021). Towards the Design and Implementation of an Image-Based Navigation System of an Autonomous Underwater Vehicle Combining a Color Recognition Technique and a Fuzzy Logic Controller. Sensors, 21(12), 4053. https://doi.org/10.3390/s21124053

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop