Search Results (22)

As the demand for 3D information continues to grow in various fields, technologies are rapidly being used to acquire such information. Laser-based estimation and multi-view images are popular methods for sensing 3D information, while deep learning techniques are also being developed. However, the former method requires precise sensing equipment or large observation systems, while the latter relies on substantial prior information in the form of extensive learning datasets. Given these limitations, our research aims to develop a method that is independent of learning and makes it possible to capture a wide range of 3D information using a compact device. This paper introduces a novel approach for estimating the 3D information of an observed scene utilizing a monocular image based on a catadioptric imaging system employing an omnidirectional camera and a spherical mirror. By employing a curved mirror, it is possible to capture a large area in a single observation. At the same time, using an omnidirectional camera enables the creation of a simplified imaging system. The proposed method focuses on a spherical or spherical cap-shaped mirror in the scene. It estimates the mirror’s position from the captured images, allowing for the estimation of the scene with great flexibility. Simulation evaluations are conducted to validate the characteristics and effectiveness of our proposed method. Full article

This paper considers the task of appearance-based localization: visual place recognition from omnidirectional images obtained from catadioptric cameras. The focus is on designing an efficient neural network architecture that accurately and reliably recognizes indoor scenes on distorted images from a catadioptric camera, even in self-similar environments with few discernible features. As the target application is the global localization of a low-cost service mobile robot, the proposed solutions are optimized toward being small-footprint models that provide real-time inference on edge devices, such as Nvidia Jetson. We compare several design choices for the neural network-based architecture of the localization system and then demonstrate that the best results are achieved with embeddings (global descriptors) yielded by exploiting transfer learning and fine tuning on a limited number of catadioptric images. We test our solutions on two small-scale datasets collected using different catadioptric cameras in the same office building. Next, we compare the performance of our system to state-of-the-art visual place recognition systems on the publicly available COLD Freiburg and Saarbrücken datasets that contain images collected under different lighting conditions. Our system compares favourably to the competitors both in terms of the accuracy of place recognition and the inference time, providing a cost- and energy-efficient means of appearance-based localization for an indoor service robot. Full article

This paper proposes a near-central camera model and its solution approach. ’Near-central’ refers to cases in which the rays do not converge to a point and do not have severely arbitrary directions (non-central cases). Conventional calibration methods are difficult to apply in such cases. Although the generalized camera model can be applied, dense observation points are required for accurate calibration. Moreover, this approach is computationally expensive in the iterative projection framework. We developed a noniterative ray correction method based on sparse observation points to address this problem. First, we established a smoothed three-dimensional (3D) residual framework using a backbone to avoid using the iterative framework. Second, we interpolated the residual by applying local inverse distance weighting on the nearest neighbor of a given point. Specifically, we prevented excessive computation and the deterioration in accuracy that may occur in inverse projection through the 3D smoothed residual vectors. Moreover, the 3D vectors can represent the ray directions more accurately than the 2D entities. Synthetic experiments show that the proposed method can achieve prompt and accurate calibration. The depth error is reduced by approximately 63% in the bumpy shield dataset, and the proposed approach is noted to be two digits faster than the iterative methods. Full article

Since infrared reflectography was first applied in the 1960s to visualize the underdrawings of ancient paintings, several devices and scanning techniques were successfully proposed both as prototypes and commercial instruments. In fact, because of the sensors’ small dimension, typically ranging from 0.1 to 0.3 megapixels, scanning is always required. Point, line, and image scanners are all viable options to obtain an infrared image of the painting with adequate spatial resolution. This paper presents a newly developed, tailormade scanning system based on an InGaAs camera equipped with a catadioptric long-focus lens in a fixed position, enabling all movements to occur by means of a rotating mirror and precision step motors. Given the specific design of this system, as the mirror rotates, refocus of the lens is necessary and it is made possible by an autofocus system involving a laser distance meter and a motorized lens. The system proved to be lightweight, low cost, easily portable, and suitable for the examination of large-scale painting surfaces by providing high-resolution reflectograms. Furthermore, high-resolution images at different wavelengths can be obtained using band-pass filters. The in-situ analysis of a 16th-century panel painting is also discussed as a representative case study to demonstrate the effectiveness and reliability of the system described herein. Full article

The remote sensing imaging requirements of aerial cameras require their optical system to have wide temperature adaptability. Based on the optical passive athermal technology, the expression of thermal power offset of a single lens in the catadioptric optical system is first derived, and then a mathematical model for efficient optimization of materials is established; finally, the mechanical material combination (mirror and housing material) is optimized according to the comprehensive weight of offset with temperature change and the position change of the equivalent single lens, and achieve optimization of the lens material on an athermal map. In order to verify the effectiveness of the method, an example of a catadioptric aerial optical system with a focal length of 350 mm is designed. The results show that in the temperature range of −40 °C to 60 °C, the diffraction-limited MTF of the designed optical system is 0.59 (at 68 lp/mm), the MTF of each field of view is greater than 0.39, and the thermal defocus is less than 0.004 mm, which is within one time of the focal depth, indicating that the imaging quality of the optical system basically does not change with temperature, meeting the stringent application requirements of the aerial camera. Full article

Omnidirectional optoelectronic systems (OOES) find applications in many areas where a wide viewing angle is crucial. The disadvantage of these systems is the large distortion of the images, which makes it difficult to make wide use of them. The purpose of this study is the development an algorithm for the precision calibration of an omnidirectional camera using a statistical approach. The calibration approach comprises three basic stages. The first stage is the formation of a cloud of points characterizing the view field of the virtual perspective camera. In the second stage, a calibration procedure that provides the projection function for the camera calibration is performed. The projection functions of traditional perspective lenses and omnidirectional wide-angle fisheye lenses with a viewing angle of no less than 180° are compared. The construction of the corrected image is performed in the third stage. The developed algorithm makes it possible to obtain an image for part of the field of view of an OOES by correcting the distortion from the original omnidirectional image.Using the developed algorithm, a non-mechanical pivoting camera based on an omnidirectional camera is implemented. The achieved mean squared error of the reproducing points from the original omnidirectional image onto the image with corrected distortion is less than the size of a very few pixels. Full article

In response to the problem of the small field of vision in 3D reconstruction, a 3D reconstruction system based on a catadioptric camera and projector was built by introducing a traditional camera to calibrate the catadioptric camera and projector system. Firstly, the intrinsic parameters of the camera and the traditional camera are calibrated separately. Then, the calibration of the projection system is accomplished by the traditional camera. Secondly, the coordinate system is introduced to calculate, respectively, the position of the catadioptric camera and projector in the coordinate system, and the position relationship between the coordinate systems of the catadioptric camera and the projector is obtained. Finally, the projector is used to project the structured light fringe to realize the reconstruction using a catadioptric camera. The experimental results show that the reconstruction error is 0.75 mm and the relative error is 0.0068 for a target of about 1 m. The calibration method and reconstruction method proposed in this paper can guarantee the ideal geometric reconstruction accuracy. Full article

The use of airborne-mapping technology plays a key role in the acquisition of large-scale basic geographic data information, especially in various important civil/military-mapping missions. However, most airborne-mapping cameras are limited by parameters, such as the flight altitude, working-environment temperature, and so on. To solve this problem, in this paper, we designed a panchromatic wide-spectrum optical system with a focusing function. Based on the catadioptric optical structure, the optical system approached a telecentric optical structure. Sharp images at different object distances could be acquired by micro-moving the focusing lens. At the same time, an optical passive compensation method was adopted to realize an athermalization design in the range of −40–60 °C. According to the design parameters of the optical system, we analyzed the influence of system focusing on mapping accuracy during the focusing process of the airborne-mapping camera. In the laboratory, the camera calibration and imaging experiments were performed at different focusing positions. The results show that the experimental data are consistent with the analysis results. Due to the limited experiment conditions, only a single flight experiment was performed. The results show that the airborne-mapping camera can achieve 1:5000 scale-imaging accuracy. Flight experiments for different flight altitudes are being planned, and the relevant experimental data will be released in the future. In conclusion, the airborne-mapping camera is expected to be applied in various high-precision scale-mapping fields. Full article

The optical system that combines imaging and image motion compensation is conducive to the miniaturization of aerial mapping cameras, but the movement of optical element for image motion compensation will cause a decrease in image quality. To solve this problem, reducing the sensitivity of moving optical element is one of the effective ways to ensure the imaging quality of aerial mapping cameras. Therefore, this paper proposes an optimization method for the low tilt sensitivity of the secondary mirror based on the Nodal aberration theory. In this method, the analytical expressions of the tilt sensitivity of the secondary mirror in different tilt directions are given in the form of zernike polynomial coefficients, and the influence of the field of view on the sensitivity is expressed in the mathematical model. The desensitization optimization function and desensitization optimization method are proposed. The catadioptric optical system with a focal length of 350 mm is used for desensitization optimization. The results show that the desensitization function proposed in this paper is linearly related to the decrease of sensitivity within a certain range, and the standard deviation of the system after desensitization is 0.020, which is 59% of the system without desensitization. Compared with the traditional method, the method in this paper widens the range of angle reduction sensitivity and has a better desensitization effect. The research results show that the optimization method for low tilt sensitivity of the secondary mirror based on the Nodal aberration theory proposed in this paper reduces the tilt sensitivity of the secondary mirror, revealing that the reduction of the sensitivity depends on the reduction of the aberration coefficient related to the misalignment in the field of view, which is critical for the development of an optical system for aerial mapping cameras that combines imaging and image motion compensation. Full article

When a traditional visual SLAM system works in a dynamic environment, it will be disturbed by dynamic objects and perform poorly. In order to overcome the interference of dynamic objects, we propose a semantic SLAM system for catadioptric panoramic cameras in dynamic environments. A real-time instance segmentation network is used to detect potential moving targets in the panoramic image. In order to find the real dynamic targets, potential moving targets are verified according to the sphere’s epipolar constraints. Then, when extracting feature points, the dynamic objects in the panoramic image are masked. Only static feature points are used to estimate the pose of the panoramic camera, so as to improve the accuracy of pose estimation. In order to verify the performance of our system, experiments were conducted on public data sets. The experiments showed that in a highly dynamic environment, the accuracy of our system is significantly better than traditional algorithms. By calculating the RMSE of the absolute trajectory error, we found that our system performed up to 96.3% better than traditional SLAM. Our catadioptric panoramic camera semantic SLAM system has higher accuracy and robustness in complex dynamic environments. Full article

This paper presents a novel self-localization technique for mobile robots using a central catadioptric camera. A unified sphere model for the image projection is derived by the catadioptric camera calibration. The geometric property of the camera projection model is utilized to obtain the intersections of the vertical lines and ground plane in the scene. Different from the conventional stereo vision techniques, the feature points are projected onto a known planar surface, and the plane equation is used for depth computation. The 3D coordinates of the base points on the ground are calculated using the consecutive image frames. The derivation of motion trajectory is then carried out based on the computation of rotation and translation between the robot positions. We develop an algorithm for feature correspondence matching based on the invariability of the structure in the 3D space. The experimental results obtained using the real scene images have demonstrated the feasibility of the proposed method for mobile robot localization applications. Full article

This paper presents a novel self-localization technique for mobile robots based on image feature matching from omnidirectional vision. The proposed method first constructs a virtual space with synthetic omnidirectional imaging to simulate a mobile robot equipped with an omnidirectional vision system in the real world. In the virtual space, a number of vertical and horizontal lines are generated according to the structure of the environment. They are imaged by the virtual omnidirectional camera using the catadioptric projection model. The omnidirectional images derived from the virtual and real environments are then used to match the synthetic lines and real scene edges. Finally, the pose and trajectory of the mobile robot in the real world are estimated by the efficient perspective-n-point (EPnP) algorithm based on the line feature matching. In our experiments, the effectiveness of the proposed self-localization technique was validated by the navigation of a mobile robot in a real world environment. Full article

In this work, we present a panoramic 3D stereo reconstruction system composed of two catadioptric cameras. Each one consists of a CCD camera and a parabolic convex mirror that allows the acquisition of catadioptric images. We describe the calibration approach and propose the improvement of existing deep feature matching methods with epipolar constraints. We show that the improved matching algorithm covers more of the scene than classic feature detectors, yielding broader and denser reconstructions for outdoor environments. Our system can also generate accurate measurements in the wild without large amounts of data used in deep learning-based systems. We demonstrate the system’s feasibility and effectiveness as a practical stereo sensor with real experiments in indoor and outdoor environments. Full article

The purpose of this paper is to investigate the problem of 3D human tracking in complex environments using a particle filter with images captured by a catadioptric vision system. This issue has been widely studied in the literature on RGB images acquired from conventional perspective cameras, while omnidirectional images have seldom been used and published research works in this field remains limited. In this study, the Riemannian varieties was considered in order to compute the gradient on spherical images and generate a robust descriptor used along with an SVM classifier for human detection. Original likelihood functions associated with the particle filter are proposed, using both geodesic distances and overlapping regions between the silhouette detected in the images and the projected 3D human model. Our approach was experimentally evaluated on real data and showed favorable results compared to machine learning based techniques about the 3D pose accuracy. Thus, the Root Mean Square Error (RMSE) was measured by comparing estimated 3D poses and truth data, resulting in a mean error of 0.065 m when walking action was applied. Full article

Omnidirectional and 360° images are becoming widespread in industry and in consumer society, causing omnidirectional computer vision to gain attention. Their wide field of view allows the gathering of a great amount of information about the environment from only an image. However, the distortion of these images requires the development of specific algorithms for their treatment and interpretation. Moreover, a high number of images is essential for the correct training of computer vision algorithms based on learning. In this paper, we present a tool for generating datasets of omnidirectional images with semantic and depth information. These images are synthesized from a set of captures that are acquired in a realistic virtual environment for Unreal Engine 4 through an interface plugin. We gather a variety of well-known projection models such as equirectangular and cylindrical panoramas, different fish-eye lenses, catadioptric systems, and empiric models. Furthermore, we include in our tool photorealistic non-central-projection systems as non-central panoramas and non-central catadioptric systems. As far as we know, this is the first reported tool for generating photorealistic non-central images in the literature. Moreover, since the omnidirectional images are made virtually, we provide pixel-wise information about semantics and depth as well as perfect knowledge of the calibration parameters of the cameras. This allows the creation of ground-truth information with pixel precision for training learning algorithms and testing 3D vision approaches. To validate the proposed tool, different computer vision algorithms are tested as line extractions from dioptric and catadioptric central images, 3D Layout recovery and SLAM using equirectangular panoramas, and 3D reconstruction from non-central panoramas. Full article