[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
7.3
3.4
Journals
Sensors
Special Issues
Deep Learning-Based Human Intention and Trajectory Prediction Systems Using Sensors
sensors-logo
Submit to Sensors Review for Sensors Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Deep Learning-Based Human Intention and Trajectory Prediction Systems Using Sensors

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Intelligent Sensors".

Deadline for manuscript submissions: closed (15 April 2023) | Viewed by 14239

Special Issue Editors

Dr. Alwin Poulose

E-Mail Website
Guest Editor
School of Data Science, Indian Institute of Science Education and Research (IISER), Thiruvananthapuram 695551, Kerala, India
Interests: indoor localization; human activity recognition; facial emotion recognition; behavior prediction; localization and mapping
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Antonio Fernández-Caballero

E-Mail Website
Guest Editor
Instituto de Investigación en Informática de Albacete, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
Interests: pattern recognition; human–computer interaction; affective computing; computer vision; multi-sensor fusion
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Human intention prediction (HIP) is an emerging research area and has a significant role in daily life. In the HIP, the system gathers data from wearable sensors and predicts human behavior. The most common sensors used in the HIP system are smartphone IMU sensors, camera sensors, smartwatches, and Wi-Fi access points. The HIP system effectively utilizes these sensors and predicts human intention. However, the existing HIP systems are not free from sensor errors, and combining different sensor data for intention prediction is also a very challenging task for HIP researchers. The HIP system consists of indoor localization, human activity recognition (HAR), and facial emotion recognition (FER). We estimate the user's trajectory from the smartphone IMU sensor and Wi-Fi access points in indoor localization. The localization system continuously tracks the user movements and calculates the user’s position. The smartphone IMU sensor data are also used for HAR and identify human activities such as walking, standing, sitting, running, jumping, sit-ups, dancing, lying, push-ups, etc. In FER, we use smartphone cameras to estimate users' emotions, including happiness, sadness, anger, fear, surprise, neutral, and disgust. This Special Issue focuses on papers that provide up-to-date information on human intention prediction, including indoor localization, human activity recognition, and facial emotion recognition. Authors are invited to submit original contributions or survey papers for publication in the open-access Sensors journal.

Topics of interest include (but are not limited to) the following:

  1. Indoor Localization :
    1. Image-based localization technologies;
    2. Dead Reckoning;
    3. Wi-Fi RSSI/Cellular/Bluetooth-based indoor positioning;
    4. RFID/UWB/Infrared/Ultrasonic/Zigbee/VLC/Acoustic Signal based indoor positioning;
    5. Hybrid positioning;
    6. Localization techniques: Triangulation, Lateration, CSI,RSS,TOA,TDOA,RTOF, Localization algorithms, Angulation, AOS,AOD,POA, NFER;
  2. Human Activity Recognition (HAR):
    1. Wearable sensor-based HAR;
    2. Radiofrequency-based HAR;
    3. Vision-based HAR;
    4. Sensors fusion for HAR;
    5. Deep and machine learning techniques for HAR;
    6. HAR applications and architectures;
    7. The human body and pose estimation;
    8. Novel datasets for activity recognition;
    9. Sensing technologies for activity recognition;
    10. Gesture recognition;
    11. Behavior recognition;
  3. Facial Emotion Recognition (FER):
    1. Sensor-based emotion recognition;
    2. Image or signal enhancement for emotion recognition;
    3. Computer vision for FER;
    4. Speech emotion recognition (SER);
    5. Security and privacy concerns in emotional detection;
    6. FER datasets;
    7. FER for autonomous driving systems;
    8. Driver monitoring system (DMS);
    9. Gaze recognition;
    10. Facial landmark detection.

Dr. Alwin Poulose
Prof. Dr. Antonio Fernández-Caballero
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Indoor Localization
  • Dead Reckoning Wi-Fi
  • Bluetooth
  • RFID
  • Localization algorithms
  • SLAM
  • Trilateration
  • Fingerprinting
  • Human Activity Recognition (HAR)
  • Deep Learning
  • Facial Emotion Recognition (FER)
  • Driver Monitoring Systems (DMS)

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (4 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

16 pages, 3862 KiB  
Article
Pashto Handwritten Invariant Character Trajectory Prediction Using a Customized Deep Learning Technique
by Fazli Khaliq, Muhammad Shabir, Inayat Khan, Shafiq Ahmad, Muhammad Usman, Muhammad Zubair and Shamsul Huda
Sensors 2023, 23(13), 6060; https://doi.org/10.3390/s23136060 - 30 Jun 2023
Cited by 2 | Viewed by 2314
Abstract
Before the 19th century, all communication and official records relied on handwritten documents, cherished as valuable artefacts by different ethnic groups. While significant efforts have been made to automate the transcription of major languages like English, French, Arabic, and Chinese, there has been [...] Read more.
Before the 19th century, all communication and official records relied on handwritten documents, cherished as valuable artefacts by different ethnic groups. While significant efforts have been made to automate the transcription of major languages like English, French, Arabic, and Chinese, there has been less research on regional and minor languages, despite their importance from geographical and historical perspectives. This research focuses on detecting and recognizing Pashto handwritten characters and ligatures, which is essential for preserving this regional cursive language in Pakistan and its status as the national language of Afghanistan. Deep learning techniques were employed to detect and recognize Pashto characters and ligatures, utilizing a newly developed dataset specific to Pashto. A further enhancement was done on the dataset by implementing data augmentation, i.e., scaling and rotation on Pashto handwritten characters and ligatures, which gave us many variations of a single trajectory. Different morphological operations for minimizing gaps in the trajectories were also performed. The median filter was used for the removal of different noises. This dataset will be combined with the existing PHWD-V2 dataset. Various deep-learning techniques were evaluated, including VGG19, MobileNetV2, MobileNetV3, and a customized CNN. The customized CNN demonstrated the highest accuracy and minimal loss, achieving a training accuracy of 93.98%, validation accuracy of 92.08% and testing accuracy of 92.99%. Full article
(This article belongs to the Special Issue Deep Learning-Based Human Intention and Trajectory Prediction Systems Using Sensors)
►▼ Show Figures

Figure 1

Figure 1
<p>Partial view of Pashto handwritten ligatures.</p> Full article ">Figure 2
<p>Pashto handwritten word ligatures.</p> Full article ">Figure 3
<p>Pashto handwritten character dysconnectivity.</p> Full article ">Figure 4
<p>Valid and invalid hook detection and recognition.</p> Full article ">Figure 5
<p>Customized 5-layer CNN model.</p> Full article ">Figure 6
<p>General overview of the proposed framework.</p> Full article ">Figure 7
<p>Dataset collection phase.</p> Full article ">Figure 8
<p>Partial view of dataset without gridlines.</p> Full article ">Figure 9
<p>Partial view of dataset cropping phase.</p> Full article ">Figure 10
<p>Partial view of noise-free image dataset.</p> Full article ">Figure 11
<p>VGG19 training and validation (<b>a</b>) accuracy and (<b>b</b>) loss.</p> Full article ">Figure 12
<p>MobileNetV2 training and validation (<b>a</b>) accuracy and (<b>b</b>) loss.</p> Full article ">Figure 13
<p>MobileNetV3Large training and validation (<b>a</b>) accuracy and (<b>b</b>) loss.</p> Full article ">Figure 14
<p>Customized CNN training and validation (<b>a</b>) accuracy and (<b>b</b>) loss.</p> Full article ">
17 pages, 2712 KiB  
Article
Vehicle Trajectory Prediction via Urban Network Modeling
by Xinyan Qin, Zhiheng Li, Kai Zhang, Feng Mao and Xin Jin
Sensors 2023, 23(10), 4893; https://doi.org/10.3390/s23104893 - 19 May 2023
Cited by 5 | Viewed by 1969
Abstract
Taxis are an important component of the transportation system, and empty taxis represent a significant waste of transportation resources. To alleviate the imbalance between supply and demand and relieve traffic congestion, real-time prediction of taxi trajectories is necessary. Most existing trajectory prediction studies [...] Read more.
Taxis are an important component of the transportation system, and empty taxis represent a significant waste of transportation resources. To alleviate the imbalance between supply and demand and relieve traffic congestion, real-time prediction of taxi trajectories is necessary. Most existing trajectory prediction studies focus on extracting time-series information but do not capture spatial information sufficiently. In this paper, we focus on the construction of an urban network and propose an urban topology-encoding spatiotemporal attention network (UTA) to address destination prediction problems. Firstly, this model discretizes the production and attraction units of transportation, combining them with key nodes in the road network to form an urban topological network. Secondly, GPS records are matched to the urban topological map to construct a topological trajectory, which significantly improves trajectory consistency and endpoint certainty, helping to model destination prediction problems. Thirdly, semantic information concerning surrounding space is attached to effectively mine the spatial dependencies of trajectories. Finally, after the topological encoding of city space and trajectories, this algorithm proposes a topological graph neural network to model the attention calculation with the trajectory context, comprehensively considering the spatiotemporal characteristics of the trajectories and improving prediction accuracy. We solve the prediction problems with the UTA model and also compare it with some other classical models, such as the HMM, RNN, LSTM, and transformer. The results suggest that all the models work well in combination with the proposed urban model (with a rough increase of 2%), while the UTA model is less affected by data sparsity. Full article
(This article belongs to the Special Issue Deep Learning-Based Human Intention and Trajectory Prediction Systems Using Sensors)
►▼ Show Figures

Figure 1

Figure 1
<p>The study area. (<b>a</b>) The position of Hangzhou in China. (<b>b</b>) The core business zone of modern Hangzhou.</p> Full article ">Figure 2
<p>The processes of developing a topological map. (<b>a</b>) The original map of an intersection. (<b>b</b>) The final topological map.</p> Full article ">Figure 3
<p>Vehicle trajectory prediction process.</p> Full article ">Figure 4
<p>Architecture of the proposed UTA.</p> Full article ">Figure 5
<p>Comparisons of AUC performance under different trajectory lengths.</p> Full article ">
26 pages, 32032 KiB  
Article
Online Boosting-Based Target Identification among Similar Appearance for Person-Following Robots
by Redhwan Algabri and Mun-Taek Choi
Sensors 2022, 22(21), 8422; https://doi.org/10.3390/s22218422 - 2 Nov 2022
Cited by 4 | Viewed by 6181
Abstract
It is challenging for a mobile robot to follow a specific target person in a dynamic environment, comprising people wearing similar-colored clothes and having the same or similar height. This study describes a novel framework for a person identification model that identifies a [...] Read more.
It is challenging for a mobile robot to follow a specific target person in a dynamic environment, comprising people wearing similar-colored clothes and having the same or similar height. This study describes a novel framework for a person identification model that identifies a target person by merging multiple features into a single joint feature online. The proposed framework exploits the deep learning output to extract four features for tracking the target person without prior knowledge making it generalizable and more robust. A modified intersection over union between the current frame and the last frame is proposed as a feature to distinguish people, in addition to color, height, and location. To improve the performance of target identification in a dynamic environment, an online boosting method was adapted by continuously updating the features in every frame. Through extensive real-life experiments, the effectiveness of the proposed method was demonstrated by showing experimental results that it outperformed the previous methods. Full article
(This article belongs to the Special Issue Deep Learning-Based Human Intention and Trajectory Prediction Systems Using Sensors)
►▼ Show Figures

Figure 1

Figure 1
<p>Flowchart of the proposed system.</p> Full article ">Figure 2
<p>Human detection and color extraction within the region of interest.</p> Full article ">Figure 3
<p>Mobile robot mounted with an RGB-D camera and necessary sensors.</p> Full article ">Figure 4
<p>Comparison of weak learner accuracy.</p> Full article ">Figure 5
<p>Comparison of computational time for the weak learner.</p> Full article ">Figure 6
<p>Comparison of computational time for online boosting algorithms.</p> Full article ">Figure 7
<p>Comparison of online boosting accuracy with the number of weak learners.</p> Full article ">Figure 8
<p>Comparison of online boosting accuracy with an increase in the number of the training samples.</p> Full article ">Figure 9
<p>Realistic scenario of the robot and people in the environment.</p> Full article ">Figure 10
<p>Height feature. (<b>a</b>) Height of the target person with respect to the ground plane (<b>top</b> plot). (<b>b</b>) Height difference of the target person between the current frame and last tracked frame (<b>middle</b> plot). (<b>c</b>) Normalization of the height difference (<b>bottom</b> plot).</p> Full article ">Figure 11
<p>IoU feature.</p> Full article ">Figure 12
<p>Localization feature. (<b>a</b>) Horizontal angle of the target person with respect to the center of the image (<b>top</b> plot). (<b>b</b>) Angle difference of the target person between the current frame and last tracked frame (<b>middle</b> plot). (<b>c</b>) Normalization of the angle difference (<b>bottom</b> plot).</p> Full article ">Figure 13
<p>Color feature.</p> Full article ">Figure 14
<p>Normalization features for the target person and other people: (<b>a</b>) Localization feature (<b>top</b> plot). (<b>b</b>) IoU feature (<b>top middle</b> plot). (<b>c</b>) Height feature (<b>bottom middle</b> plot). (<b>d</b>) Color feature (<b>bottom</b> plot).</p> Full article ">Figure 15
<p>Snapshots of the experiments for three target persons with three different colors: robot’s view.</p> Full article ">Figure 16
<p>Snapshots of the experiments for target identification under a different lighting environment.</p> Full article ">Figure 17
<p>Snapshots of the following failure using only two features of the color and height: robot’s view.</p> Full article ">
15 pages, 4430 KiB  
Article
Human Arm Motion Prediction for Collision Avoidance in a Shared Workspace
by Pu Zheng, Pierre-Brice Wieber, Junaid Baber and Olivier Aycard
Sensors 2022, 22(18), 6951; https://doi.org/10.3390/s22186951 - 14 Sep 2022
Cited by 8 | Viewed by 2596
Abstract
Industry 4.0 transforms classical industrial systems into more human-centric and digitized systems. Close human–robot collaboration is becoming more frequent, which means security and efficiency issues need to be carefully considered. In this paper, we propose to equip robots with exteroceptive sensors and online [...] Read more.
Industry 4.0 transforms classical industrial systems into more human-centric and digitized systems. Close human–robot collaboration is becoming more frequent, which means security and efficiency issues need to be carefully considered. In this paper, we propose to equip robots with exteroceptive sensors and online motion generation so that the robot is able to perceive and predict human trajectories and react to the motion of the human in order to reduce the occurrence of the collisions. The dataset for training is generated in a real environment in which a human and a robot are sharing their workspace. An Encoder–Decoder based network is proposed to predict the human hand trajectories. A Model Predictive Control (MPC) framework is also proposed, which is able to plan a collision-free trajectory in the shared workspace based on this human motion prediction. The proposed framework is validated in a real environment that ensures collision free collaboration between humans and robots in a shared workspace. Full article
(This article belongs to the Special Issue Deep Learning-Based Human Intention and Trajectory Prediction Systems Using Sensors)
►▼ Show Figures

Figure 1

Figure 1
<p>Abstract flow diagram of control architecture.</p> Full article ">Figure 2
<p>An illustration of separating plane between two objects.</p> Full article ">Figure 3
<p>Example demonstration of RGB-D image for mapping human hand in 3D space, (<b>a</b>) shows RGB image on which hand joins are detected, (<b>b</b>) shows the bounding box around the points in the depth image, and (<b>c</b>) shows the mapping of points in 3D space.</p> Full article ">Figure 4
<p>ROS based controller architecture to enable collaboration between humans and robots in shared environments.</p> Full article ">Figure 5
<p>Demonstration of the environment for dataset generation, (<b>a</b>) shows the person working in shared environment with a cobot, (<b>b</b>) shows the possible goals on which the hand should be moving, (<b>c</b>) shows one sample hand motion trajectory generated over one minute, (<b>d</b>) shows the sub-trajectory over 12 observations, and (<b>e</b>) shows the trajectory divided into two sequences (<span class="html-italic">x</span> for training and <span class="html-italic">y</span> for prediction).</p> Full article ">Figure 6
<p>Proposed model for human hand motion prediction, (<b>a</b>) shows the architecture of the encoder–decoder LSTM neural network, and (<b>b</b>) shows the overview of LSTM cell architecture.</p> Full article ">Figure 7
<p>Training and validation loss of proposed model, (<b>a</b>) shows the training and validation MAE for the model with Gaussian noise, and (<b>b</b>) shows the training and validation MAE with random noise.</p> Full article ">Figure 8
<p>The MAE of proposed model, (<b>a</b>) shows the training and validation MAE for the model with Gaussian noise, (<b>b</b>,<b>c</b>) show the comparative validation and training MAE with a different configuration of noise.</p> Full article ">Figure 9
<p>The qualitative visualization of collaborative environment between cobot and human, (<b>a</b>) shows the distance between cobot and hand, which is much larger than safety distance (20 cm), (<b>b</b>) shows the behavior of the cobot when the human hand is intentionally placed for possible collision, the cobot deviates from its initial trajectory to avoid the collision, (<b>c</b>,<b>d</b>) show that the cobot achieves its goal without stopping or hurting the human while keeping a safe distance. The sub-figures from (<b>a’</b>–<b>d’</b>) show the corresponding visualization of the same data in RVIZ.</p> Full article ">Figure 10
<p>A case study on which collision can not be avoided, (<b>a</b>) human blocks cobot motion intentionally, (<b>b</b>) the trajectory generator ensures that the cobot is at rest to avoid collision.</p> Full article ">
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-4
Back to TopTop