[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Sign in to use this feature.

Years

Between: -

Subjects

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Journals

Article Types

Countries / Regions

remove_circle_outline
remove_circle_outline
remove_circle_outline

Search Results (153)

Search Parameters:
Keywords = anomaly detection (A.D.)

Order results
Result details
Results per page
Select all
Export citation of selected articles as:
26 pages, 5616 KiB  
Article
Enhancing Intelligent Transport Systems Through Decentralized Security Frameworks in Vehicle-to-Everything Networks
by Usman Tariq and Tariq Ahamed Ahanger
World Electr. Veh. J. 2025, 16(1), 24; https://doi.org/10.3390/wevj16010024 - 3 Jan 2025
Viewed by 762
Abstract
Vehicle Ad hoc Networks (VANETs) play an essential role in intelligent transportation systems (ITSs) by improving road safety and traffic management through robust decentralized communication between vehicles and infrastructure. Yet, decentralization introduces security vulnerabilities, including spoofing, tampering, and denial-of-service attacks, which can compromise [...] Read more.
Vehicle Ad hoc Networks (VANETs) play an essential role in intelligent transportation systems (ITSs) by improving road safety and traffic management through robust decentralized communication between vehicles and infrastructure. Yet, decentralization introduces security vulnerabilities, including spoofing, tampering, and denial-of-service attacks, which can compromise the reliability and safety of vehicular communications. Traditional centralized security mechanisms are often inadequate in providing the real-time response and scalability required by such dispersed networks. This research promotes a shift toward distributed and real-time technologies, including blockchain and secure multi-party computation, to enhance communication integrity and privacy, ultimately strengthening system resilience by eliminating single points of failure. A core aspect of this study is the novel D-CASBR framework, which integrates three essential components. First, it employs hybrid machine learning methods, such as ElasticNet and Gradient Boosting, to facilitate real-time anomaly detection, identifying unusual activities as they occur. Second, it utilizes a consortium blockchain to provide secure and transparent information exchange among authorized participants. Third, it implements a fog-enabled reputation system that uses distributed fog computing to effectively manage trust within the network. This comprehensive approach addresses latency issues found in conventional systems while significantly improving the reliability and efficacy of threat detection, achieving 95 percent anomaly detection accuracy with minimal false positives. The result is a substantial advancement in securing vehicular networks. Full article
Show Figures

Figure 1

Figure 1
<p>Taxonomy of vulnerabilities in VANETs. (Summarized according to reference [<a href="#B2-wevj-16-00024" class="html-bibr">2</a>,<a href="#B3-wevj-16-00024" class="html-bibr">3</a>,<a href="#B4-wevj-16-00024" class="html-bibr">4</a>,<a href="#B5-wevj-16-00024" class="html-bibr">5</a>,<a href="#B6-wevj-16-00024" class="html-bibr">6</a>,<a href="#B7-wevj-16-00024" class="html-bibr">7</a>,<a href="#B8-wevj-16-00024" class="html-bibr">8</a>]).</p>
Full article ">Figure 2
<p>Framework architecture of novel decentralized intrusion detection system with Collaborative Anomaly Scoring and Blockchain-based Reputation (D-CASBR).</p>
Full article ">Figure 3
<p>Simulation environment and factors.</p>
Full article ">Figure 4
<p>Impact of message loss rate on D-CASBR.</p>
Full article ">Figure 5
<p>Anomaly detection ratio with number of nodes. [<a href="#B11-wevj-16-00024" class="html-bibr">11</a>,<a href="#B12-wevj-16-00024" class="html-bibr">12</a>,<a href="#B13-wevj-16-00024" class="html-bibr">13</a>,<a href="#B14-wevj-16-00024" class="html-bibr">14</a>,<a href="#B15-wevj-16-00024" class="html-bibr">15</a>].</p>
Full article ">Figure A1
<p>These taxonomies (<b>a</b>–<b>f</b>) illustrate the categorically sorted lists of VANET-focused vulnerability-driven listings.</p>
Full article ">Figure A1 Cont.
<p>These taxonomies (<b>a</b>–<b>f</b>) illustrate the categorically sorted lists of VANET-focused vulnerability-driven listings.</p>
Full article ">Figure A1 Cont.
<p>These taxonomies (<b>a</b>–<b>f</b>) illustrate the categorically sorted lists of VANET-focused vulnerability-driven listings.</p>
Full article ">Figure A1 Cont.
<p>These taxonomies (<b>a</b>–<b>f</b>) illustrate the categorically sorted lists of VANET-focused vulnerability-driven listings.</p>
Full article ">
16 pages, 2278 KiB  
Article
Enhancing VANET Security: An Unsupervised Learning Approach for Mitigating False Information Attacks in VANETs
by Abinash Borah and Anirudh Paranjothi
Electronics 2025, 14(1), 58; https://doi.org/10.3390/electronics14010058 - 26 Dec 2024
Viewed by 415
Abstract
Vehicular ad hoc networks (VANETs) enable communication among vehicles and between vehicles and infrastructure to provide safety and comfort to the users. Malicious nodes in VANETs may broadcast false information to create the impression of a fake event or road congestion. In addition, [...] Read more.
Vehicular ad hoc networks (VANETs) enable communication among vehicles and between vehicles and infrastructure to provide safety and comfort to the users. Malicious nodes in VANETs may broadcast false information to create the impression of a fake event or road congestion. In addition, several malicious nodes may collude to collectively launch a false information attack to increase the credibility of the attack. Detection of these attacks is critical to mitigate the potential risks they bring to the safety of users. Existing techniques for detecting false information attacks in VANETs use different approaches such as machine learning, blockchain, trust scores, statistical methods, etc. These techniques rely on historical information about vehicles, artificial data used to train the technique, or coordination among vehicles. To address these limitations, we propose a false information attack detection technique for VANETs using an unsupervised anomaly detection approach. The objective of the proposed technique is to detect false information attacks based on only real-time characteristics of the network, achieving high accuracy and low processing delay. The performance evaluation results show that our proposed technique offers 30% lower data processing delay and a 17% lower false positive rate compared to existing approaches in scenarios with high proportions of malicious nodes. Full article
(This article belongs to the Special Issue Machine Learning and Cybersecurity—Trends and Future Challenges)
Show Figures

Figure 1

Figure 1
<p>An example of a distance-based anomaly.</p>
Full article ">Figure 2
<p>The overall approach of the proposed false information detection technique.</p>
Full article ">Figure 3
<p>An example of approximating neighbor count for bin <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>b</mi> </mrow> <mrow> <mi>i</mi> </mrow> </msub> </mrow> </semantics></math> with three bins.</p>
Full article ">Figure 4
<p>Results for the urban scenario: (<b>a</b>) Data processing time vs. percentage of malicious nodes; (<b>b</b>) accuracy vs. percentage of malicious nodes; (<b>c</b>) precision vs. percentage of malicious nodes; (<b>d</b>) recall vs. percentage of malicious nodes; (<b>e</b>) F1 score vs. percentage of malicious nodes; (<b>f</b>) FPR vs. percentage of malicious nodes.</p>
Full article ">Figure 4 Cont.
<p>Results for the urban scenario: (<b>a</b>) Data processing time vs. percentage of malicious nodes; (<b>b</b>) accuracy vs. percentage of malicious nodes; (<b>c</b>) precision vs. percentage of malicious nodes; (<b>d</b>) recall vs. percentage of malicious nodes; (<b>e</b>) F1 score vs. percentage of malicious nodes; (<b>f</b>) FPR vs. percentage of malicious nodes.</p>
Full article ">Figure 5
<p>Results for the highway scenario: (<b>a</b>) Data processing time vs. percentage of malicious nodes; (<b>b</b>) accuracy vs. percentage of malicious nodes; (<b>c</b>) precision vs. percentage of malicious nodes; (<b>d</b>) recall vs. percentage of malicious nodes; (<b>e</b>) F1 score vs. percentage of malicious nodes; (<b>f</b>) FPR vs. percentage of malicious nodes.</p>
Full article ">Figure 5 Cont.
<p>Results for the highway scenario: (<b>a</b>) Data processing time vs. percentage of malicious nodes; (<b>b</b>) accuracy vs. percentage of malicious nodes; (<b>c</b>) precision vs. percentage of malicious nodes; (<b>d</b>) recall vs. percentage of malicious nodes; (<b>e</b>) F1 score vs. percentage of malicious nodes; (<b>f</b>) FPR vs. percentage of malicious nodes.</p>
Full article ">
16 pages, 754 KiB  
Article
Improved Deep Support Vector Data Description Model Using Feature Patching for Industrial Anomaly Detection
by Wei Huang, Yongjie Li, Zhaonan Xu, Xinwei Yao and Rongchun Wan
Sensors 2025, 25(1), 67; https://doi.org/10.3390/s25010067 - 26 Dec 2024
Viewed by 306
Abstract
In industrial contexts, anomaly detection is crucial for ensuring quality control and maintaining operational efficiency in manufacturing processes. Leveraging high-level features extracted from ImageNet-trained networks and the robust capabilities of the Deep Support Vector Data Description (SVDD) model for anomaly detection, this paper [...] Read more.
In industrial contexts, anomaly detection is crucial for ensuring quality control and maintaining operational efficiency in manufacturing processes. Leveraging high-level features extracted from ImageNet-trained networks and the robust capabilities of the Deep Support Vector Data Description (SVDD) model for anomaly detection, this paper proposes an improved Deep SVDD model, termed Feature-Patching SVDD (FPSVDD), designed for unsupervised anomaly detection in industrial applications. This model integrates a feature-patching technique with the Deep SVDD framework. Features are extracted from a pre-trained backbone network on ImageNet, and each extracted feature is split into multiple small patches of appropriate size. This approach effectively captures both macro-structural information and fine-grained local information from the extracted features, enhancing the model’s sensitivity to anomalies. The feature patches are then aggregated and concatenated for further training with the Deep SVDD model. Experimental results on both the MvTec AD and CIFAR-10 datasets demonstrate that our model outperforms current mainstream approaches and provides significant improvements in anomaly detection performance, which is vital for industrial quality assurance and defect detection in real-time manufacturing scenarios. Full article
(This article belongs to the Section Industrial Sensors)
Show Figures

Figure 1

Figure 1
<p>The whole structure of the FPSVDD model.</p>
Full article ">Figure 2
<p>The workflow of the FPSVDD model.</p>
Full article ">Figure 3
<p>Illustration of feature patching, aggregation, and concatenation.</p>
Full article ">Figure 4
<p>AUROC values versus training epochs on different datasets. The dataset and the normal class within each dataset are indicated above each subplot. For example, “CIFAR10-truck” denotes that the results in this subplot are based on the truck class as the normal class in the CIFAR10 dataset.</p>
Full article ">
30 pages, 5783 KiB  
Article
Enhancing Time Series Anomaly Detection: A Knowledge Distillation Approach with Image Transformation
by Haiwoong Park and Hyeryung Jang
Sensors 2024, 24(24), 8169; https://doi.org/10.3390/s24248169 - 21 Dec 2024
Viewed by 547
Abstract
Anomaly detection is critical in safety-sensitive fields, but faces challenges from scarce abnormal data and costly expert labeling. Time series anomaly detection is relatively challenging due to its reliance on sequential data, which imposes high computational and memory costs. In particular, it is [...] Read more.
Anomaly detection is critical in safety-sensitive fields, but faces challenges from scarce abnormal data and costly expert labeling. Time series anomaly detection is relatively challenging due to its reliance on sequential data, which imposes high computational and memory costs. In particular, it is often composed of real-time collected data that tends to be noisy, making preprocessing an essential step. In contrast, image anomaly detection has leveraged advancements in technologies for analyzing spatial patterns and visual features, achieving high accuracy and promoting research aimed at improving efficiency. We propose a novel framework that bridges image anomaly detection with time series data. Using Gramian Angular Field (GAF) transformations, we convert time series into images and apply state-of-the-art techniques, Reverse Distillation (RD) and EfficientAD (EAD), for efficient and accurate anomaly detection. Tailored preprocessing and transformations further enhance performance and interoperability. When evaluated on the multivariate time series anomaly detection dataset Secure Water Treatment (SWaT) and the univariate datasets University of California, Riverside (UCR) and Numenta Anomaly Benchmark (NAB), our approach demonstrated high recall overall and achieved approximately 99% F1 scores on some univariate datasets, proving its effectiveness as a novel solution for time series anomaly detection. Full article
(This article belongs to the Section Fault Diagnosis & Sensors)
Show Figures

Figure 1

Figure 1
<p>Overview of reverse distillation.</p>
Full article ">Figure 2
<p>Overview of EfficientAD.</p>
Full article ">Figure 3
<p>The overall structure of the proposed framework.</p>
Full article ">Figure 4
<p>Anomaly types of time series data.</p>
Full article ">Figure 5
<p>Time series decomposition results.</p>
Full article ">Figure 6
<p>Comparison of time series data and GAF representations.</p>
Full article ">Figure 7
<p>Visualization of SWaT raw data, LGAF transformed images, and anomaly maps for normal and abnormal data segments. The top row corresponds to the normal data segment, and the bottom row represents the abnormal data segment.</p>
Full article ">Figure 8
<p>Comparison of data segmentation methods.</p>
Full article ">Figure 9
<p>Comparison of LGAF and GGAF on NAB train and test data, including anomaly maps. The top row shows results from LGAF, and the bottom row shows results from GGAF.</p>
Full article ">Figure 10
<p>Visualization of ground truth and detection results on SWaT dataset.</p>
Full article ">Figure 10 Cont.
<p>Visualization of ground truth and detection results on SWaT dataset.</p>
Full article ">
40 pages, 49128 KiB  
Article
Self-Supervised Autoencoders for Visual Anomaly Detection
by Alexander Bauer, Shinichi Nakajima and Klaus-Robert Müller
Mathematics 2024, 12(24), 3988; https://doi.org/10.3390/math12243988 - 18 Dec 2024
Viewed by 503
Abstract
We focus on detecting anomalies in images where the data distribution is supported by a lower-dimensional embedded manifold. Approaches based on autoencoders have aimed to control their capacity either by reducing the size of the bottleneck layer or by imposing sparsity constraints on [...] Read more.
We focus on detecting anomalies in images where the data distribution is supported by a lower-dimensional embedded manifold. Approaches based on autoencoders have aimed to control their capacity either by reducing the size of the bottleneck layer or by imposing sparsity constraints on their activations. However, none of these techniques explicitly penalize the reconstruction of anomalous regions, often resulting in poor detection. We tackle this problem by adapting a self-supervised learning regime that essentially implements a denoising autoencoder with structured non-i.i.d. noise. Informally, our objective is to regularize the model to produce locally consistent reconstructions while replacing irregularities by acting as a filter that removes anomalous patterns. Formally, we show that the resulting model resembles a nonlinear orthogonal projection of partially corrupted images onto the submanifold of uncorrupted examples. Furthermore, we identify the orthogonal projection as an optimal solution for a specific regularized autoencoder related to contractive and denoising variants. In addition, orthogonal projection provides a conservation effect by largely preserving the original content of its arguments. Together, these properties facilitate an accurate detection and localization of anomalous regions by means of the reconstruction error. We support our theoretical analysis by achieving state-of-the-art results (image/pixel-level AUROC of 99.8/99.2%) on the MVTec AD dataset—a challenging benchmark for anomaly detection in the manufacturing domain. Full article
Show Figures

Figure 1

Figure 1
<p>Anomaly detection results of our approach on a few images from the MVTec AD dataset. The first row shows the input images and the second row an overlay with the predicted anomaly heatmap.</p>
Full article ">Figure 2
<p>Illustration of the reconstruction effect of our model trained either on the wood, carpet, or grid images (without defects) from the MVTec AD dataset.</p>
Full article ">Figure 3
<p>Illustration of our anomaly detection process after the training. Given input <math display="inline"><semantics> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> </semantics></math>, we first, see (<b>1</b>) compute an output <math display="inline"><semantics> <mrow> <msub> <mi>f</mi> <mi mathvariant="bold-italic">θ</mi> </msub> <mrow> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>)</mo> </mrow> </mrow> </semantics></math> by replicating normal regions and replacing irregularities with locally consistent patterns. Then, see (<b>2</b>), we compute a pixel-wise squared difference <math display="inline"><semantics> <msup> <mrow> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>−</mo> <msub> <mi>f</mi> <mi mathvariant="bold-italic">θ</mi> </msub> <mrow> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mn>2</mn> </msup> </semantics></math>, which is subsequently averaged over the color channels to produce the difference map <math display="inline"><semantics> <mrow> <mi>Diff</mi> <mrow> <mo>[</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>,</mo> <msub> <mi>f</mi> <mi mathvariant="bold-italic">θ</mi> </msub> <mrow> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>)</mo> </mrow> <mo>]</mo> </mrow> <mo>∈</mo> <msup> <mi mathvariant="double-struck">R</mi> <mrow> <mi>h</mi> <mo>×</mo> <mi>w</mi> </mrow> </msup> </mrow> </semantics></math>. In the last step, see (<b>3</b>), we apply a series of averaging convolutions <math display="inline"><semantics> <msub> <mi>G</mi> <mi>k</mi> </msub> </semantics></math> to the difference map to produce our final anomaly heatmap <math display="inline"><semantics> <mrow> <msubsup> <mi>anomap</mi> <mrow> <msub> <mi>f</mi> <mi mathvariant="bold-italic">θ</mi> </msub> </mrow> <mrow> <mi>n</mi> <mo>,</mo> <mi>k</mi> </mrow> </msubsup> <mrow> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>)</mo> </mrow> </mrow> </semantics></math>.</p>
Full article ">Figure 4
<p>Illustration of data generation for training. After randomly choosing the number and locations of the patches to be modified, we create new content by gluing the extracted patches with the corresponding replacements. Given a real-valued mask <math display="inline"><semantics> <mrow> <mi mathvariant="bold-italic">M</mi> <mo>∈</mo> <msup> <mrow> <mo>[</mo> <mn>0</mn> <mo>,</mo> <mn>1</mn> <mo>]</mo> </mrow> <mrow> <mover accent="true"> <mi>h</mi> <mo>˜</mo> </mover> <mo>×</mo> <mover accent="true"> <mi>w</mi> <mo>˜</mo> </mover> <mo>×</mo> <mn>3</mn> </mrow> </msup> </mrow> </semantics></math> marking corrupted regions within a patch, an original image patch <math display="inline"><semantics> <mi mathvariant="bold-italic">x</mi> </semantics></math>, and a corresponding replacement <math display="inline"><semantics> <mi mathvariant="bold-italic">y</mi> </semantics></math>, we create the next corrupted patch by merging the two patches together according to the formula <math display="inline"><semantics> <mrow> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>:</mo> <mo>=</mo> <mi mathvariant="bold-italic">M</mi> <mo>⊙</mo> <mi mathvariant="bold-italic">y</mi> <mo>+</mo> <mover accent="true"> <mi mathvariant="bold-italic">M</mi> <mo stretchy="false">¯</mo> </mover> <mo>⊙</mo> <mi mathvariant="bold-italic">x</mi> </mrow> </semantics></math>. All mask shapes <math display="inline"><semantics> <mi mathvariant="bold-italic">M</mi> </semantics></math> are created by applying Gaussian distortion to the same (static) mask, representing a filled disk at the center of the patch with a smoothly fading boundary toward the exterior of the disk.</p>
Full article ">Figure 5
<p>Illustration of our network architecture SDC-CCM including the convex combination module (CCM) marked in brown and the skip-connections represented by the horizontal arrows. Without these additional elements, we obtain our baseline architecture SDC-AE.</p>
Full article ">Figure 6
<p>Illustration of the CCM module. The module receives two inputs: <math display="inline"><semantics> <msub> <mi mathvariant="bold-italic">x</mi> <mi>s</mi> </msub> </semantics></math> along the skip connection and <math display="inline"><semantics> <msub> <mi mathvariant="bold-italic">x</mi> <mi>c</mi> </msub> </semantics></math> from the current layer below. In the first step (image on the left), we compute the squared difference of the two and stack it together with the original values <math display="inline"><semantics> <mrow> <mo>[</mo> <msub> <mi mathvariant="bold-italic">x</mi> <mi>s</mi> </msub> <mo>,</mo> <msub> <mi mathvariant="bold-italic">x</mi> <mi>c</mi> </msub> <mo>,</mo> <msup> <mrow> <mo>(</mo> <msub> <mi mathvariant="bold-italic">x</mi> <mi>s</mi> </msub> <mo>−</mo> <msub> <mi mathvariant="bold-italic">x</mi> <mi>c</mi> </msub> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mo>]</mo> </mrow> </semantics></math>. This combined feature map is processed by two convolutional layers. The first layer uses batch normalization with ReLU activation. The second layer uses batch normalization and a sigmoid activation function to produce a coefficient matrix <math display="inline"><semantics> <mi mathvariant="bold-italic">β</mi> </semantics></math>. In the second step (image on the right), we compute the output of the module as a (component-wise) convex combination <math display="inline"><semantics> <mrow> <msub> <mi mathvariant="bold-italic">x</mi> <mi>o</mi> </msub> <mo>=</mo> <mi mathvariant="bold-italic">β</mi> <mo>·</mo> <msub> <mi mathvariant="bold-italic">x</mi> <mi>s</mi> </msub> <mo>+</mo> <mrow> <mo>(</mo> <mn mathvariant="bold">1</mn> <mo>−</mo> <mi mathvariant="bold-italic">β</mi> <mo>)</mo> </mrow> <mo>·</mo> <msub> <mi mathvariant="bold-italic">x</mi> <mi mathvariant="bold-italic">c</mi> </msub> </mrow> </semantics></math>, where <math display="inline"><semantics> <mn mathvariant="bold">1</mn> </semantics></math> is a tensor of ones.</p>
Full article ">Figure 7
<p>Illustration of the general concept of the orthogonal projection <span class="html-italic">f</span> onto a data manifold <math display="inline"><semantics> <mi mathvariant="script">D</mi> </semantics></math>. Here, anomalous samples <math display="inline"><semantics> <mrow> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>∈</mo> <msup> <mi mathvariant="double-struck">R</mi> <mi>n</mi> </msup> </mrow> </semantics></math> (red dots) are projected to points <math display="inline"><semantics> <mrow> <mi mathvariant="bold-italic">x</mi> <mo>:</mo> <mo>=</mo> <mi>f</mi> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>)</mo> <mo>∈</mo> <mi mathvariant="script">D</mi> </mrow> </semantics></math> (blue dots) in a way that minimizes the distance <math display="inline"><semantics> <mrow> <mi>d</mi> <mrow> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>,</mo> <mi mathvariant="bold-italic">x</mi> <mo>)</mo> </mrow> <mo>=</mo> <msub> <mo movablelimits="true" form="prefix">inf</mo> <mrow> <mi mathvariant="bold-italic">y</mi> <mo>∈</mo> <mi mathvariant="script">D</mi> </mrow> </msub> <mi>d</mi> <mrow> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>,</mo> <mi mathvariant="bold-italic">y</mi> <mo>)</mo> </mrow> </mrow> </semantics></math>.</p>
Full article ">Figure 8
<p>Illustration of the connections between the different types of regularized autoencoders. For a small variance of the corruption noise, the DAE becomes similar to the CAE. This, in turn, gives rise to the RCAE, where the contraction is imposed explicitly on the whole reconstruction mapping. A special instance of PAE given by the orthogonal projection yields an optimal solution for the optimization problem of the RCAE. On the other hand, the training objective for PAE can be seen as an extension of DAE to more complex input modifications beyond additive noise. Finally, a common variant of the sparse autoencoder (SAE) applies an <math display="inline"><semantics> <msup> <mi>l</mi> <mn>1</mn> </msup> </semantics></math> penalty on the hidden units, resulting in saturation toward zero similar to the CAE.</p>
Full article ">Figure 9
<p>Illustration of the conservation effect of the orthogonal projections with respect to different <math display="inline"><semantics> <msup> <mi>l</mi> <mi>p</mi> </msup> </semantics></math>-norms. Here, the anomalous sample <math display="inline"><semantics> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> </semantics></math> is orthogonally projected onto the manifold <math display="inline"><semantics> <mi mathvariant="script">D</mi> </semantics></math> (depicted by a red ellipsoid) according to <math display="inline"><semantics> <mrow> <mrow> <mo stretchy="false">∥</mo> </mrow> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>−</mo> <msubsup> <mi mathvariant="bold-italic">y</mi> <mi>p</mi> <mo>*</mo> </msubsup> <msub> <mrow> <mo stretchy="false">∥</mo> </mrow> <mi>p</mi> </msub> <mo>=</mo> <msub> <mo movablelimits="true" form="prefix">inf</mo> <mrow> <mi mathvariant="bold-italic">y</mi> <mo>∈</mo> <mi mathvariant="script">D</mi> </mrow> </msub> <msub> <mrow> <mo stretchy="false">∥</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>−</mo> <mi mathvariant="bold-italic">y</mi> <mo stretchy="false">∥</mo> </mrow> <mi>p</mi> </msub> </mrow> </semantics></math> for <math display="inline"><semantics> <mrow> <mi>p</mi> <mo>∈</mo> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mn>2</mn> <mo>,</mo> <mo>∞</mo> <mo>}</mo> </mrow> </semantics></math>. The remaining three colors (green, blue, and yellow) represent rescaled unit circles around <math display="inline"><semantics> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> </semantics></math> with respect to <math display="inline"><semantics> <msup> <mi>l</mi> <mn>1</mn> </msup> </semantics></math>, <math display="inline"><semantics> <msup> <mi>l</mi> <mn>2</mn> </msup> </semantics></math> and <math display="inline"><semantics> <msup> <mi>l</mi> <mo>∞</mo> </msup> </semantics></math>-norms. The intersection points of each circle with <math display="inline"><semantics> <mi mathvariant="script">D</mi> </semantics></math> mark the orthogonal projection of <math display="inline"><semantics> <mi mathvariant="bold-italic">x</mi> </semantics></math> onto <math display="inline"><semantics> <mi mathvariant="script">D</mi> </semantics></math> for the corresponding norm. We can see that projections <math display="inline"><semantics> <msubsup> <mi mathvariant="bold-italic">y</mi> <mi>p</mi> <mo>*</mo> </msubsup> </semantics></math> for lower <span class="html-italic">p</span>-values better preserve the content in <math display="inline"><semantics> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> </semantics></math> according to the higher sparsity of the difference <math display="inline"><semantics> <mrow> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>−</mo> <msubsup> <mi mathvariant="bold-italic">y</mi> <mi>p</mi> <mo>*</mo> </msubsup> </mrow> </semantics></math>, which results in smaller modified regions <math display="inline"><semantics> <mrow> <mi>S</mi> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>,</mo> <msubsup> <mi mathvariant="bold-italic">y</mi> <mi>p</mi> <mo>*</mo> </msubsup> <mo>)</mo> </mrow> </semantics></math>.</p>
Full article ">Figure 10
<p>Illustration of the concept of a transition set. Consider a 2D image tensor identified with a column vector <math display="inline"><semantics> <mrow> <mi mathvariant="bold-italic">x</mi> <mo>∈</mo> <msup> <mi mathvariant="double-struck">R</mi> <mi>n</mi> </msup> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <mi>n</mi> <mo>=</mo> <msup> <mn>20</mn> <mn>2</mn> </msup> </mrow> </semantics></math>, which is partitioned according to <math display="inline"><semantics> <mrow> <mi>S</mi> <mo>⊆</mo> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mo>…</mo> <mo>,</mo> <mi>n</mi> <mo>}</mo> </mrow> </semantics></math> (gray area) and <math display="inline"><semantics> <mrow> <mover accent="true"> <mi>S</mi> <mo stretchy="false">¯</mo> </mover> <mo>:</mo> <mo>=</mo> <mrow> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mo>…</mo> <mo>,</mo> <mi>n</mi> <mo>}</mo> </mrow> <mo>∖</mo> <mi>S</mi> </mrow> </semantics></math> (union of light blue and dark blue areas). The transition set <span class="html-italic">B</span> (dark blue area) glues the two disconnected sets <span class="html-italic">S</span> and <math display="inline"><semantics> <mrow> <mover accent="true"> <mi>S</mi> <mo stretchy="false">¯</mo> </mover> <mo>∖</mo> <mi>B</mi> </mrow> </semantics></math> together such that <math display="inline"><semantics> <mrow> <mi mathvariant="bold-italic">x</mi> <mo>∈</mo> <mi mathvariant="script">D</mi> </mrow> </semantics></math> is feasible.</p>
Full article ">Figure 11
<p>Illustration of our anomaly segmentation results (with SDC-CCM) as an overlay of the original image and the anomaly heatmap. Each row shows three random examples from a category (carpet, grid, leather, transistor, and cable) in the MVTec AD dataset. In each pair, the first image represents the input to the model and the second image a corresponding anomaly heatmap.</p>
Full article ">Figure A1
<p>Illustration of a counterexample for the claim that orthogonal projections maximally preserve normal regions in the inputs. Here, <math display="inline"><semantics> <mrow> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>∈</mo> <msup> <mi mathvariant="double-struck">Z</mi> <mn>5</mn> </msup> </mrow> </semantics></math> is the modified version of the original input <math display="inline"><semantics> <mrow> <mi mathvariant="bold-italic">x</mi> <mo>∈</mo> <mi mathvariant="script">D</mi> </mrow> </semantics></math> according to the partition <math display="inline"><semantics> <mrow> <mi>S</mi> <mo>,</mo> <mover accent="true"> <mi>S</mi> <mo stretchy="false">¯</mo> </mover> </mrow> </semantics></math> and <math display="inline"><semantics> <mrow> <mi>f</mi> <mo>(</mo> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> <mo>)</mo> </mrow> </semantics></math> denotes the orthogonal projection of <math display="inline"><semantics> <mover accent="true"> <mi mathvariant="bold-italic">x</mi> <mo stretchy="false">^</mo> </mover> </semantics></math> onto <math display="inline"><semantics> <mi mathvariant="script">D</mi> </semantics></math> with respect to the <math display="inline"><semantics> <msub> <mi>l</mi> <mn>2</mn> </msub> </semantics></math>-norm. This example also shows that orthogonality property is dependent on our choice of the distance metric.</p>
Full article ">Figure A2
<p>Illustration of the concept of a transition set on two examples with different shapes. Each of the two images represents an MRF <math display="inline"><semantics> <mrow> <mi mathvariant="bold-italic">x</mi> <mo>=</mo> <mo>(</mo> <msub> <mi>x</mi> <mn>1</mn> </msub> <mo>,</mo> <mo>…</mo> <mo>,</mo> <msub> <mi>x</mi> <mi>n</mi> </msub> <mo>)</mo> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <mi>n</mi> <mo>∈</mo> <mi mathvariant="double-struck">N</mi> </mrow> </semantics></math> of the Markov order <math display="inline"><semantics> <mrow> <mi>K</mi> <mo>∈</mo> <mi mathvariant="double-struck">N</mi> </mrow> </semantics></math> with nodes corresponding to the individual pixels with values from a finite set of states <math display="inline"><semantics> <mrow> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>∈</mo> <mi>I</mi> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <mo>|</mo> <mi>I</mi> <mo>|</mo> <mo>&lt;</mo> <mo>∞</mo> </mrow> </semantics></math>. The grey area marks the corrupted region <math display="inline"><semantics> <mrow> <mi>S</mi> <mo>⊆</mo> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mo>…</mo> <mo>,</mo> <mi>n</mi> <mo>}</mo> </mrow> </semantics></math>, where the union of the dark blue and light blue areas is the complement <math display="inline"><semantics> <mrow> <mover accent="true"> <mi>S</mi> <mo stretchy="false">¯</mo> </mover> <mo>:</mo> <mo>=</mo> <mrow> <mo>{</mo> <mn>1</mn> <mo>,</mo> <mo>…</mo> <mo>,</mo> <mi>n</mi> <mo>}</mo> </mrow> <mo>∖</mo> <mi>S</mi> </mrow> </semantics></math> marking the normal region. The dark blue part of <math display="inline"><semantics> <mover accent="true"> <mi>S</mi> <mo stretchy="false">¯</mo> </mover> </semantics></math> corresponds to the transition set <math display="inline"><semantics> <mrow> <mi>B</mi> <mo>⊆</mo> <mover accent="true"> <mi>S</mi> <mo stretchy="false">¯</mo> </mover> </mrow> </semantics></math>. <math display="inline"><semantics> <mrow> <mi>W</mi> <mo>⩽</mo> <msup> <mrow> <mo>|</mo> <mi>I</mi> <mo>|</mo> </mrow> <mi>K</mi> </msup> </mrow> </semantics></math> denotes (loosely) the width at the thickest part of the tube <span class="html-italic">B</span> around <span class="html-italic">S</span>.</p>
Full article ">Figure A3
<p>Illustration of the importance of modeling long-range dependencies facilitated by dilated convolutions for achieving accurate reconstruction. We can observe how the reconstruction of the model without the SDC modules (middle image) suffers from a blind spot effect toward the center of the corrupted region. This happens due to the insufficient context provided by the normal areas, forcing the model to predict an average of all possibilities.</p>
Full article ">Figure A4
<p>Illustration of the qualitative improvement when using SDC-CCM over SDC-AE. We show six examples: three from the "cable" category and three from the "transistor" category of the MVTec AD dataset. Each row displays the original image, the reconstruction produced by SDC-CCM (reconstruction II), the reconstruction produced by SDC-AE (reconstruction I), the anomaly heatmap from SDC-CCM (anomaly heatmap II), and the anomaly heatmap from SDC-AE (anomaly heatmap I). Note the significant improvement in the quality of the heatmaps.</p>
Full article ">Figure A5
<p>Illustration of the qualitative improvement when using SDC-CCM over SDC-AE on texture categories from the MVTec AD dataset. We show five examples, one from each of the following categories: “carpet”, “grid”, “leather”, “tile”, and “wood”. Each row displays the original image, the reconstruction produced by SDC-CCM (reconstruction II), the reconstruction produced by SDC-AE (reconstruction I), the anomaly heatmap from SDC-CCM (anomaly heatmap II), and the anomaly heatmap from SDC-AE (anomaly heatmap I).</p>
Full article ">
17 pages, 1606 KiB  
Article
Patch-Wise-Based Self-Supervised Learning for Anomaly Detection on Multivariate Time Series Data
by Seungmin Oh, Le Hoang Anh, Dang Thanh Vu, Gwang Hyun Yu, Minsoo Hahn and Jinsul Kim
Mathematics 2024, 12(24), 3969; https://doi.org/10.3390/math12243969 - 17 Dec 2024
Viewed by 639
Abstract
Multivariate time series anomaly detection is a crucial technology to prevent unexpected errors from causing critical impacts. Effective anomaly detection in such data requires accurately capturing temporal patterns and ensuring the availability of adequate data. This study proposes a patch-wise framework for anomaly [...] Read more.
Multivariate time series anomaly detection is a crucial technology to prevent unexpected errors from causing critical impacts. Effective anomaly detection in such data requires accurately capturing temporal patterns and ensuring the availability of adequate data. This study proposes a patch-wise framework for anomaly detection. The proposed approach comprises four key components: (i) maintaining continuous features through patching, (ii) incorporating various temporal information by learning channel dependencies and adding relative positional bias, (iii) achieving feature representation learning through self-supervised learning, and (iv) supervised learning based on anomaly augmentation for downstream tasks. The proposed method demonstrates strong anomaly detection performance by leveraging patching to maintain temporal continuity while effectively learning data representations and handling downstream tasks. Additionally, it mitigates the issue of insufficient anomaly data by supporting the learning of diverse types of anomalies. The experimental results show that our model achieved a 23% to 205% improvement in the F1 score compared to existing methods on datasets such as MSL, which has a relatively small amount of training data. Furthermore, the model also delivered a competitive performance on the SMAP dataset. By systematically learning both local and global dependencies, the proposed method strikes an effective balance between feature representation and anomaly detection accuracy, making it a valuable tool for real-world multivariate time series applications. Full article
(This article belongs to the Special Issue Recent Advances in Time Series Analysis)
Show Figures

Figure 1

Figure 1
<p>Patch-wise learning framework: (<b>Left</b>) representation learning based on self-supervised learning using patching, (<b>Right</b>) supervised learning based on anomaly augmentation.</p>
Full article ">Figure 2
<p>Self-supervised learning-based representation learning architecture.</p>
Full article ">Figure 3
<p>Supervised learning for anomaly detection.</p>
Full article ">Figure 4
<p>Anomaly augmentation.</p>
Full article ">Figure 5
<p>Comparison of channel-dependent (CD) and channel-independent (CI) strategies in time series reconstruction.</p>
Full article ">Figure 6
<p>Visualization of anomaly datasets.</p>
Full article ">Figure 7
<p>(<b>a</b>) F1 score performance based on value embedding in the MSL dataset; (<b>b</b>) F1 score performance based on value embedding in the SMAP dataset; (<b>c</b>) F1 score performance based on value embedding in the SMD dataset.</p>
Full article ">
24 pages, 3202 KiB  
Article
Monitoring Critical Health Conditions in the Elderly: A Deep Learning-Based Abnormal Vital Sign Detection Model
by Murad A. Rassam and Amal A. Al-Shargabi
Technologies 2024, 12(12), 258; https://doi.org/10.3390/technologies12120258 - 13 Dec 2024
Viewed by 980
Abstract
Global population aging creates distinct healthcare needs, particularly for older adults and those with serious illnesses. There are several gaps in current models for monitoring elderly individuals. These include the limited application of advanced deep learning techniques in elderly health monitoring, the lack [...] Read more.
Global population aging creates distinct healthcare needs, particularly for older adults and those with serious illnesses. There are several gaps in current models for monitoring elderly individuals. These include the limited application of advanced deep learning techniques in elderly health monitoring, the lack of real-time anomaly detection for vital signs, the absence of robust evaluations using real-world data, and the failure to tailor monitoring systems specifically for the unique needs of elderly individuals. This study addresses these gaps by proposing a Hierarchical Attention-based Temporal Convolutional Network (HATCN) model, which enhances anomaly detection accuracy and validates effectiveness using real-world datasets. While the HATCN approach has been used in other fields, it has not yet been applied to elderly healthcare monitoring, making this contribution novel. Specifically, this study introduces a Hierarchical Attention-based Temporal Convolutional Network with Anomaly Detection (HATCN-AD) model, based on the real-world MIMIC-II dataset. The model was validated using two subjects from the MIMIC-II dataset: Subject 330 (Dataset 1) and Subject 441 (Dataset 2). For Dataset 1 (Subject 330), the model achieved an accuracy of 99.15% and precision of 99.47%, with stable recall (99.45%) and F1-score (99.46%). Similarly, for Dataset 2 (Subject 441), the model achieved 99.11% accuracy, 99.35% precision, and an F1-score of 99.44% at 100 epochs. The results show that the HATCN-AD model outperformed similar models, achieving high recall and precision with low false positives and negatives. This ensures accurate anomaly detection for real-time healthcare monitoring. By combining Temporal Convolutional Networks and attention mechanisms, the HATCN-AD model effectively monitors elderly patients’ vital signs. Full article
(This article belongs to the Section Information and Communication Technologies)
Show Figures

Figure 1

Figure 1
<p>IoMT architecture.</p>
Full article ">Figure 2
<p>Proposed HATCN-AD model architecture.</p>
Full article ">Figure 3
<p>HATCN architecture.</p>
Full article ">Figure 4
<p>Sensor readings for three selected vital signs.</p>
Full article ">Figure 5
<p>Results on Dataset 1 with batch size = 32: (<b>a</b>) Model loss for 50 epochs. (<b>b</b>) Model accuracy for 50 epochs. (<b>c</b>) Model loss for 100 epochs. (<b>d</b>) Model accuracy for 100 epochs.</p>
Full article ">Figure 6
<p>Results on Dataset 1 with batch size = 64: (<b>a</b>) Model loss for 50 epochs. (<b>b</b>) Model accuracy for 50 epochs. (<b>c</b>) Model loss for 100 epochs. (<b>d</b>) Model accuracy for 100 epochs.</p>
Full article ">Figure 7
<p>Results on Dataset 2 with batch size = 32: (<b>a</b>) Model loss for 50 epochs. (<b>b</b>) Model accuracy for 50 epochs. (<b>c</b>) Model loss for 100 epochs. (<b>d</b>) Model accuracy for 100 epochs.</p>
Full article ">Figure 8
<p>Results on Dataset 2 with batch size = 64: (<b>a</b>) Model loss for 50 epochs. (<b>b</b>) Model accuracy for 50 epochs. (<b>c</b>) Model loss for 100 epochs. (<b>d</b>) Model accuracy for 100 epochs.</p>
Full article ">
21 pages, 4145 KiB  
Article
UniFlow: Unified Normalizing Flow for Unsupervised Multi-Class Anomaly Detection
by Jianmei Zhong and Yanzhi Song
Information 2024, 15(12), 791; https://doi.org/10.3390/info15120791 - 10 Dec 2024
Viewed by 642
Abstract
Multi-class anomaly detection is more efficient and less resource-consuming in industrial anomaly detection scenes that involve multiple categories or exhibit large intra-class diversity. However, most industrial image anomaly detection methods are developed for one-class anomaly detection, which typically suffer significant performance drops in [...] Read more.
Multi-class anomaly detection is more efficient and less resource-consuming in industrial anomaly detection scenes that involve multiple categories or exhibit large intra-class diversity. However, most industrial image anomaly detection methods are developed for one-class anomaly detection, which typically suffer significant performance drops in multi-class scenarios. Research specifically targeting multi-class anomaly detection remains relatively limited. In this work, we propose a powerful unified normalizing flow for multi-class anomaly detection, which we call UniFlow. A multi-cognitive visual adapter (Mona) is employed in our method as the feature adaptation layer to adapt image features for both the multi-class anomaly detection task and the normalizing flow model, facilitating the learning of general knowledge of normal images across multiple categories. We adopt multi-cognitive convolutional networks with high capacity to construct the coupling layers within the normalizing flow model for more effective multi-class distribution modeling. In addition, we employ a multi-scale feature fusion module to aggregate features from various levels, thereby obtaining fused features with enhanced expressive capabilities. UniFlow achieves a class-average image-level AUROC of 99.1% and a class-average pixel-level AUROC of 98.0% on MVTec AD, outperforming the SOTA multi-class anomaly detection methods. Extensive experiments on three benchmark datasets, MVTec AD, VisA, and BTAD, demonstrate the efficacy and superiority of our unified normalizing flow in multi-class anomaly detection. Full article
Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>One-class anomaly detection versus multi-class anomaly detection.</p>
Full article ">Figure 2
<p><b>Overview of UniFlow.</b> Mona FA: Mona feature adaptation. Ds: Downsample. MC Affine Layer: Multi-cognitive affine coupling layer. MC Additive Layer: Multi-cognitive additive coupling layer. Two Mona feature adaptation layers are employed to adapt the features from two stages. We downsample the feature maps extracted in the second stage to half of their original size via average pooling. Notably, Gaussian noise is added to the features only during the training phase.</p>
Full article ">Figure 3
<p>(<b>a</b>) The architecture of Mona. (<b>b</b>) The architecture of multi-cognitive convolutional module. (<b>c</b>) Mona-tuning in each SwinBlock.</p>
Full article ">Figure 4
<p>The architecture of the multi-cognitive additive coupling layer. Multi-cognitive Conv: Multi-cognitive convolutional module in <a href="#information-15-00791-f003" class="html-fig">Figure 3</a>b. <math display="inline"><semantics> <mrow> <mn>1</mn> <mo>×</mo> <mn>1</mn> </mrow> </semantics></math> Conv: A convolutional layer with a kernel size of <math display="inline"><semantics> <mrow> <mn>1</mn> <mo>×</mo> <mn>1</mn> </mrow> </semantics></math> and a stride of 1, where the output retains the same shape as the input. Global Affine: Further scaling and translation applied to the global output. Channel permute: Shuffling the order of the channel dimension.</p>
Full article ">Figure 5
<p>The architecture of the multi-cognitive affine coupling layer. Multi-cognitive Conv: Multi-cognitive convolutional module in <a href="#information-15-00791-f003" class="html-fig">Figure 3</a>b. <math display="inline"><semantics> <mrow> <mn>3</mn> <mo>×</mo> <mn>3</mn> </mrow> </semantics></math> Conv: A convolutional layer with a kernel size of <math display="inline"><semantics> <mrow> <mn>3</mn> <mo>×</mo> <mn>3</mn> </mrow> </semantics></math> and a stride of 1, where the output has the same spatial dimensions as the input but with twice the number of channels. Global Affine: Further scaling and translation applied to the global output. Channel permute: Shuffling the order of the channel dimension.</p>
Full article ">Figure 6
<p>Visualization results of anomaly localization for examples from MVTec AD [<a href="#B18-information-15-00791" class="html-bibr">18</a>] and BTAD [<a href="#B20-information-15-00791" class="html-bibr">20</a>].</p>
Full article ">Figure 7
<p>Visualization results of anomaly localization for examples from VisA [<a href="#B19-information-15-00791" class="html-bibr">19</a>].</p>
Full article ">Figure 8
<p>The impact of varying feature jittering probability on anomaly detection and localization performance on MVTec AD.</p>
Full article ">Figure 9
<p>Framework of the convolutional feature adaptation layer.</p>
Full article ">
27 pages, 8858 KiB  
Article
Fractals as Pre-Training Datasets for Anomaly Detection and Localization
by Cynthia I. Ugwu, Emanuele Caruso and Oswald Lanz
Fractal Fract. 2024, 8(11), 661; https://doi.org/10.3390/fractalfract8110661 - 13 Nov 2024
Viewed by 804
Abstract
Anomaly detection is crucial in large-scale industrial manufacturing as it helps to detect and localize defective parts. Pre-training feature extractors on large-scale datasets is a popular approach for this task. Stringent data security, privacy regulations, high costs, and long acquisition time hinder the [...] Read more.
Anomaly detection is crucial in large-scale industrial manufacturing as it helps to detect and localize defective parts. Pre-training feature extractors on large-scale datasets is a popular approach for this task. Stringent data security, privacy regulations, high costs, and long acquisition time hinder the development of large-scale datasets for training and benchmarking. Despite recent work focusing primarily on the development of new anomaly detection methods based on such extractors, not much attention has been paid to the importance of the data used for pre-training. This study compares representative models pre-trained with fractal images against those pre-trained with ImageNet, without subsequent task-specific fine-tuning. We evaluated the performance of eleven state-of-the-art methods on MVTecAD, MVTec LOCO AD, and VisA, well-known benchmark datasets inspired by real-world industrial inspection scenarios. Further, we propose a novel method to create a dataset by combining the dynamically generated fractal images creating a “Multi-Formula” dataset. Even though pre-training with ImageNet leads to better results, fractals can achieve close performance to ImageNet under proper parametrization. This opens up the possibility for a new research direction where feature extractors could be trained on synthetically generated abstract datasets mitigating the ever-increasing demand for data in machine learning while circumventing privacy and security concerns. Full article
Show Figures

Figure 1

Figure 1
<p>Examples of samples we generated from a class of fractals. Note that in [<a href="#B9-fractalfract-08-00661" class="html-bibr">9</a>], fractals belonging to the same class share similar geometric properties, as they are sampled by slightly perturbing on one of the parameters of the linear operator <span class="html-italic">A</span>. Contrary to [<a href="#B15-fractalfract-08-00661" class="html-bibr">15</a>], different fractals are grouped under the same class, lacking geometric continuity within samples from the same class.</p>
Full article ">Figure 2
<p>Examples of samples we generated from a class of MandelbulbVAR-1k. We can observe that a class is composed of the same Mandelbulb taken from different perspectives and with various colour patterns, ensuring geometric continuity between objects of the same class.</p>
Full article ">Figure 3
<p>Overview of the proposed “Multi-Formula” dataset. Fractals from different classes from the source dataset are grouped to be the features of new classes, where a variable number of fractals are present in a sample of a class.</p>
Full article ">Figure 4
<p>The left-hand box (“Dataset Generation”) illustrates two distinct IFS, each defining unique codes obtained by sampling the parameters of the system which are used to generate both Fractal and Mandelbulb datasets. In the middle box ("Pre-Training") a computer vision model for multi-class classification is trained from the generated images, either with a single sample or multiple samples per image. Finally, in the last box (“Anomaly Detection”), the model is used as a feature extractor for unsupervised anomaly detection.</p>
Full article ">Figure 5
<p>Spider chart representing average image-level AUROC grouping MVTecAD and VisA classes into different object categories.</p>
Full article ">Figure 6
<p>Comparison between ImageNet and Fractals pre-training when using different feature hierarchies.</p>
Full article ">Figure 7
<p>Comparison between ImageNet, Fractals, and MandelbulbVAR-1k pre-training when using different feature hierarchies on PaDiM.</p>
Full article ">Figure 8
<p>Comparison of the filters from the first convolutional layer of WideResNet50 pre-trained with different datasets.</p>
Full article ">Figure 9
<p>Qualitative visualization for the MVTecAD’s classes: <span class="html-italic">bottle</span>, <span class="html-italic">cable</span>, <span class="html-italic">carpet</span>, <span class="html-italic">hazelnut</span>, and <span class="html-italic">wood</span>. In the first column, we have the original image and the ground-truth. In the <span style="color: #0000FF">blue</span> box, we have the anomaly score and predicted segmentation mask for ImageNet pre-training, in the <span style="color: #FF0000">red</span> box for Fractals, and the <span style="color: #5F04B4">purple</span> box for MandelbulbVAR-1k.</p>
Full article ">Figure 10
<p>Top-1 classification accuracy during training for different generated datasets.</p>
Full article ">Figure 11
<p>Comparison of the filters from the first convolutional layer of WideResNet50 pre-trained with different dataset configurations.</p>
Full article ">Figure 12
<p>The t-SNE plot of the CIFAR-10 validation set, using WideResNet-50 pre-trained on different datasets, is presented. We extracted feature vectors from the penultimate layers, prior to the final classification layers, without any fine-tuning. (Note: The legend in each t-SNE plot is intentionally small, as our focus is on illustrating the structure of the latent space rather than the classification of each individual point).</p>
Full article ">Figure 13
<p>Image- (<b>left</b>) and pixel-level (<b>right</b>) AUROC scores achieved with PatchCore at various epochs of the pre-training stage using different training configurations.</p>
Full article ">Figure 14
<p>Comparison of the filters from the first convolutional layer of WideResNet-50 that give the results reported in <a href="#fractalfract-08-00661-t014" class="html-table">Table 14</a>. Some of the “dot-like” filters are framed in red.</p>
Full article ">
17 pages, 24201 KiB  
Article
An Echo State Network-Based Light Framework for Online Anomaly Detection: An Approach to Using AI at the Edge
by Andrea Bonci, Renat Kermenov, Lorenzo Longarini, Sauro Longhi, Geremia Pompei, Mariorosario Prist and Carlo Verdini
Machines 2024, 12(10), 743; https://doi.org/10.3390/machines12100743 - 21 Oct 2024
Viewed by 845
Abstract
Production efficiency is used to determine the best conditions for manufacturing goods at the lowest possible unit cost. When achieved, production efficiency leads to increased revenues for the manufacturer, enhanced employee safety, and a satisfied customer base. Production efficiency not only measures the [...] Read more.
Production efficiency is used to determine the best conditions for manufacturing goods at the lowest possible unit cost. When achieved, production efficiency leads to increased revenues for the manufacturer, enhanced employee safety, and a satisfied customer base. Production efficiency not only measures the amount of resources that are needed for production but also considers the productivity levels and the state of the production lines. In this context, online anomaly detection (AD) is an important tool for maintaining the reliability of the production ecosystem. With advancements in artificial intelligence and the growing significance of identifying and mitigating anomalies across different fields, approaches based on artificial neural networks facilitate the recognition of intricate types of anomalies by taking into account both temporal and contextual attributes. In this paper, a lightweight framework based on the Echo State Network (ESN) model running at the edge is introduced for online AD. Compared to other AD methods, such as Long Short-Term Memory (LSTM), it achieves superior precision, accuracy, and recall metrics while reducing training time, CO2 emissions, and the need for high computational resources. The preliminary evaluation of the proposed solution was conducted using a low-resource computing device at the edge of the real production machine through an Industrial Internet of Things (IIoT) smart meter module. The machine used to test the proposed solution was provided by the Italian company SIFIM Srl, which manufactures filter mats for industrial kitchens. Experimental results demonstrate the feasibility of developing an AD method that achieves high accuracy, with the ESN-based framework reaching 85% compared to 80.88% for the LSTM-based model. Furthermore, this method requires minimal hardware resources, with a training time of 9.5 s compared to 2.100 s for the other model. Full article
Show Figures

Figure 1

Figure 1
<p>ESN diagram.</p>
Full article ">Figure 2
<p>LSTM diagram.</p>
Full article ">Figure 3
<p>Architecture for the computation of the standard deviation of the error in the training set.</p>
Full article ">Figure 4
<p>Framework architecture for the inference phase.</p>
Full article ">Figure 5
<p>Architecture diagram.</p>
Full article ">Figure 6
<p>Production machine and production layout.</p>
Full article ">Figure 7
<p>Cloud dashboard—CO<sub>2</sub> production.</p>
Full article ">Figure 8
<p>Standard deviation of the error of both the ESN and LSTM model-based approaches.</p>
Full article ">Figure 9
<p>Comparison between real time series and time-series prediction (ESN-based model on the left and LSTM-based model on the right).</p>
Full article ">Figure 10
<p>Accuracy for each epoch (LSTM-based and ESN-based methods).</p>
Full article ">
24 pages, 7534 KiB  
Article
DeepESN Neural Networks for Industrial Predictive Maintenance through Anomaly Detection from Production Energy Data
by Andrea Bonci, Luca Fredianelli, Renat Kermenov, Lorenzo Longarini, Sauro Longhi, Geremia Pompei, Mariorosario Prist and Carlo Verdini
Appl. Sci. 2024, 14(19), 8686; https://doi.org/10.3390/app14198686 - 26 Sep 2024
Viewed by 1298
Abstract
Optimizing energy consumption is an important aspect of industrial competitiveness, as it directly impacts operational efficiency, cost reduction, and sustainability goals. In this context, anomaly detection (AD) becomes a valuable methodology, as it supports maintenance activities in the manufacturing sector, allowing for early [...] Read more.
Optimizing energy consumption is an important aspect of industrial competitiveness, as it directly impacts operational efficiency, cost reduction, and sustainability goals. In this context, anomaly detection (AD) becomes a valuable methodology, as it supports maintenance activities in the manufacturing sector, allowing for early intervention to prevent energy waste and maintain optimal performance. Here, an AD-based method is proposed and studied to support energy-saving predictive maintenance of production lines using time series acquired directly from the field. This paper proposes a deep echo state network (DeepESN)-based method for anomaly detection by analyzing energy consumption data sets from production lines. Compared with traditional prediction methods, such as recurrent neural networks with long short-term memory (LSTM), although both models show similar time series trends, the DeepESN-based method studied here appears to have some advantages, such as timelier error detection and higher prediction accuracy. In addition, the DeepESN-based method has been shown to be more accurate in predicting the occurrence of failure. The proposed solution has been extensively tested in a real-world pilot case consisting of an automated metal filter production line equipped with industrial smart meters to acquire energy data during production phases; the time series, composed of 88 variables associated with energy parameters, was then processed using the techniques introduced earlier. The results show that our method enables earlier error detection and achieves higher prediction accuracy when running on an edge device. Full article
(This article belongs to the Special Issue Digital and Sustainable Manufacturing in Industry 4.0)
Show Figures

Figure 1

Figure 1
<p>Echo state network architecture.</p>
Full article ">Figure 2
<p>Long short-term memory architecture.</p>
Full article ">Figure 3
<p>Input layer and gates architecture.</p>
Full article ">Figure 4
<p>Deep echo state network architecture.</p>
Full article ">Figure 5
<p>Anomaly detector architecture.</p>
Full article ">Figure 6
<p>The sub-phases in the global architecture of the proposed AD methodology.</p>
Full article ">Figure 7
<p>System architecture.</p>
Full article ">Figure 8
<p>Sifim’s production line. (<b>1</b>) Loading station. (<b>2</b>) Working area. (<b>3</b>) Unloading station. (<b>4</b>) Complete overview.</p>
Full article ">Figure 9
<p>Seneca S604’s IoT module. (<b>1</b>) Electric schema. (<b>2</b>) Installed module.</p>
Full article ">Figure 10
<p>Example of acquired data.</p>
Full article ">Figure 11
<p>Example of <math display="inline"><semantics> <mi mathvariant="bold-italic">σ</mi> </semantics></math> and <math display="inline"><semantics> <mi mathvariant="bold">q</mi> </semantics></math> vectors.</p>
Full article ">Figure 12
<p>Development of the accuracy, F1 score, time, and CO<sub>2</sub> emissions metrics for each epoch of LSTM model training compared with the one-shot DeepESN results.</p>
Full article ">Figure 13
<p>Current system anomaly detection.</p>
Full article ">Figure 14
<p>DeepESN receiver operating characteristic (ROC) curve.</p>
Full article ">
25 pages, 2612 KiB  
Article
Measuring the Effectiveness of Carbon-Aware AI Training Strategies in Cloud Instances: A Confirmation Study
by Roberto Vergallo and Luca Mainetti
Future Internet 2024, 16(9), 334; https://doi.org/10.3390/fi16090334 - 13 Sep 2024
Viewed by 1092
Abstract
While the massive adoption of Artificial Intelligence (AI) is threatening the environment, new research efforts begin to be employed to measure and mitigate the carbon footprint of both training and inference phases. In this domain, two carbon-aware training strategies have been proposed in [...] Read more.
While the massive adoption of Artificial Intelligence (AI) is threatening the environment, new research efforts begin to be employed to measure and mitigate the carbon footprint of both training and inference phases. In this domain, two carbon-aware training strategies have been proposed in the literature: Flexible Start and Pause & Resume. Such strategies—natively Cloud-based—use the time resource to postpone or pause the training algorithm when the carbon intensity reaches a threshold. While such strategies have proved to achieve interesting results on a benchmark of modern models covering Natural Language Processing (NLP) and computer vision applications and a wide range of model sizes (up to 6.1B parameters), it is still unclear whether such results may hold also with different algorithms and in different geographical regions. In this confirmation study, we use the same methodology as the state-of-the-art strategies to recompute the saving in carbon emissions of Flexible Start and Pause & Resume in the Anomaly Detection (AD) domain. Results confirm their effectiveness in two specific conditions, but the percentage reduction behaves differently compared with what is stated in the existing literature. Full article
(This article belongs to the Special Issue Generative Artificial Intelligence in Smart Societies)
Show Figures

Figure 1

Figure 1
<p>Number of publications per type of Green AI definition (source [<a href="#B11-futureinternet-16-00334" class="html-bibr">11</a>]).</p>
Full article ">Figure 2
<p>Research methodology adopted in this paper.</p>
Full article ">Figure 3
<p>An example of a dataset provided by WattTime. In this case, it is the .csv file for the Italian region.</p>
Full article ">Figure 4
<p>Average emissions of the trainings during the year 2021 across different regions. These emissions exceed 1.6 kg CO<sub>2</sub>eq, comparable to emissions of CO<sub>2</sub> per litre of fuel consumed by a car.</p>
Full article ">Figure 5
<p>The carbon emissions from training HF-SCA (AD on 8 × A100 GPUs for 16 h) in seven different regions (one region per line) at various times throughout the year are shown. Each line is relatively flat, indicating that emissions in a single region are consistent across different months. However, there is significant variation between the least carbon-intensive regions (represented by the lowest lines) and the most carbon-intensive regions (represented by the top lines). This confirms that selecting the region in which experiments are conducted can have a substantial impact on emissions, with differences ranging from 0.25 kg in the most efficient regions to 2.5 kg in the least efficient regions.</p>
Full article ">Figure 6
<p>Emission reduction percentage for the four AI workloads using Flexible Start strategy and the hours set in Northern Carolina. The <span class="html-italic">x</span> axis represents the time extension (6, 12, 18, 24 h) assigned to the workload to complete the job. The <span class="html-italic">y</span> axis represents the checking time (15, 30, 60, 120 min) for carbon intensity. The <span class="html-italic">z</span> axis represents the emission reduction percentages for each specific combination of time extension and checking time. (<b>a</b>) Isolation Forest workload; (<b>b</b>) SVM workload; (<b>c</b>) HF-SCA workload; (<b>d</b>) autoencoder workload.</p>
Full article ">Figure 7
<p>Emission reduction percentage for the four AI workloads using Flexible Start strategy and the percentage set in Northern Carolina. The <span class="html-italic">x</span> axis represents the time extension (+25%, 50%, 75%, 100% of the original training time) assigned to the workload to complete the job. The <span class="html-italic">y</span> axis represents the checking time (15, 30, 60, 120 min) for carbon intensity. The <span class="html-italic">z</span> axis represents the emission reduction percentages for each specific combination of time extension and checking time: (<b>a</b>) Isolation Forest workload; (<b>b</b>) SVM workload; (<b>c</b>) HF-SCA workload; (<b>d</b>) autoencoder workload.</p>
Full article ">Figure 8
<p>Emission reduction percentage for the four AI workloads using Pause &amp; Resume strategy and the hours set in Northern Carolina. The <span class="html-italic">x</span> axis represents the time extension (6, 12, 18, 24 h) assigned to the workload to complete the job. The <span class="html-italic">y</span> axis represents the checking time (15, 30, 60, 120 min) for carbon intensity. The <span class="html-italic">z</span> axis represents the emission reduction percentages for each specific combination of time extension and checking time: (<b>a</b>) Isolation Forest workload; (<b>b</b>) SVM workload; (<b>c</b>) HF-SCA workload; (<b>d</b>) autoencoder workload.</p>
Full article ">Figure 9
<p>Emission reduction percentage for the four AI workloads using Pause &amp; Resume strategy and the percentage set in Northern Carolina. The <span class="html-italic">x</span> axis represents the time extension (+25%, 50%, 75%, 100% of the original training time) assigned to the workload to complete the job. The <span class="html-italic">y</span> axis represents the checking time (15, 30, 60, 120 min) for carbon intensity. The <span class="html-italic">z</span> axis represents the emission reduction percentages for each specific combination of time extension and checking time: (<b>a</b>) Isolation Forest workload; (<b>b</b>) SVM workload; (<b>c</b>) HF-SCA workload; (<b>d</b>) autoencoder workload.</p>
Full article ">
13 pages, 433 KiB  
Article
Developing a Hybrid Detection Approach to Mitigating Black Hole and Gray Hole Attacks in Mobile Ad Hoc Networks
by Mohammad Yazdanypoor, Stefano Cirillo and Giandomenico Solimando
Appl. Sci. 2024, 14(17), 7982; https://doi.org/10.3390/app14177982 - 6 Sep 2024
Cited by 1 | Viewed by 925
Abstract
Mobile ad hoc networks (MANETs) have revolutionized wireless communications by enabling dynamic, infrastructure-free connectivity across various applications, from disaster recovery to military operations. However, these networks are highly vulnerable to security threats, particularly black hole and gray hole attacks, which can severely disrupt [...] Read more.
Mobile ad hoc networks (MANETs) have revolutionized wireless communications by enabling dynamic, infrastructure-free connectivity across various applications, from disaster recovery to military operations. However, these networks are highly vulnerable to security threats, particularly black hole and gray hole attacks, which can severely disrupt network performance and reliability. This study addresses the critical challenge of detecting and mitigating these attacks within the framework of the dynamic source routing (DSR) protocol. To tackle this issue, we propose a robust hybrid detection method that significantly enhances the identification and mitigation of black hole and gray hole attacks. Our approach integrates anomaly detection, advanced data mining techniques, and cryptographic verification to establish a multi-layered defense mechanism. Extensive simulations demonstrate that the proposed hybrid method achieves superior detection accuracy, reduces false positives, and maintains high packet delivery ratios even under attack conditions. Compared to existing solutions, this method provides more reliable and resilient network performance, dynamically adapting to evolving threats. This research represents a significant advancement in MANET security, offering a scalable and effective solution for safeguarding critical MANET applications against sophisticated cyber-attacks. Full article
(This article belongs to the Special Issue Data Security in IoT Networks)
Show Figures

Figure 1

Figure 1
<p>An overview of the process underlying the proposed approach.</p>
Full article ">Figure 2
<p>Detection accuracy among Watchdog, DSR, and our hybrid approach.</p>
Full article ">Figure 3
<p>False positive rates among Watchdog, DSR, and our hybrid approach.</p>
Full article ">Figure 4
<p>Packet delivery ratio (PDR) among Watchdog, DSR, and our hybrid approach.</p>
Full article ">Figure 5
<p>Network latency among Watchdog, DSR, and our hybrid approach.</p>
Full article ">Figure 6
<p>System overhead among Watchdog, DSR, and our hybrid approach.</p>
Full article ">
23 pages, 2789 KiB  
Article
PSAU-Defender: A Lightweight and Low-Cost Comprehensive Framework for BeiDou Spoofing Mitigation in Vehicular Networks
by Usman Tariq
World Electr. Veh. J. 2024, 15(9), 407; https://doi.org/10.3390/wevj15090407 - 5 Sep 2024
Cited by 1 | Viewed by 814
Abstract
The increasing reliance of Vehicular Ad-hoc Networks (VANETs) on the BeiDou Navigation Satellite System (BDS) for precise positioning and timing information has raised significant concerns regarding their vulnerability to spoofing attacks. This research proposes a novel approach to mitigate BeiDou spoofing attacks in [...] Read more.
The increasing reliance of Vehicular Ad-hoc Networks (VANETs) on the BeiDou Navigation Satellite System (BDS) for precise positioning and timing information has raised significant concerns regarding their vulnerability to spoofing attacks. This research proposes a novel approach to mitigate BeiDou spoofing attacks in VANETs by leveraging a hybrid machine learning model that combines XGBoost and Random Forest with a Kalman Filter for real-time anomaly detection in BeiDou signals. It also introduces a geospatial message authentication mechanism to enhance vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication security. The research investigates low-cost and accessible countermeasures against spoofing attacks using COTS receivers and open-source SDRs. Spoofing attack scenarios are implemented in both software and hardware domains using an open-source BeiDou signal simulator to examine the effects of different spoofing attacks on victim receivers and identify detection methods for each type, focusing on pre-correlation techniques with power-related metrics and signal quality monitoring using correlator values. The emulation results demonstrate the effectiveness of the proposed approach in detecting and mitigating BeiDou spoofing attacks in VANETs, ensuring the integrity and reliability of safety-critical information. This research contributes to the development of robust security mechanisms for VANETs and has practical implications for enhancing the resilience of transportation systems against spoofing threats. Future research will focus on extending the proposed approach to other GNSS constellations and exploring the integration of additional security measures to further strengthen VANET security. Full article
Show Figures

Figure 1

Figure 1
<p>Differentiating legitimate and spoofed signals in navigation systems.</p>
Full article ">Figure 2
<p>PSAU-Defender framework to detect and mitigate BSD spoofing anomalies.</p>
Full article ">Figure 3
<p>Emulation assessment outcomes (selected sample extracted from data outcomes of fifty vehicles). (<b>a</b>) Link Establishment Requests between Source and Destination. (<b>b</b>) Network Connectivity Checks. (<b>c</b>) Session Connection Log. (<b>d</b>) Ad hoc Session Termination. (<b>e</b>) Critical System Event Alert. (<b>f</b>) Generation and Transmission of Localized Spoofing Signals. (<b>g</b>) Vehicle Disconnects from Ad Hoc Network. (<b>h</b>) Isolation of Malicious Nodes upon Detection.</p>
Full article ">Figure 4
<p>Performance evaluation of our proposed methodology in the context of current best practices [<a href="#B8-wevj-15-00407" class="html-bibr">8</a>,<a href="#B9-wevj-15-00407" class="html-bibr">9</a>,<a href="#B10-wevj-15-00407" class="html-bibr">10</a>,<a href="#B11-wevj-15-00407" class="html-bibr">11</a>,<a href="#B12-wevj-15-00407" class="html-bibr">12</a>,<a href="#B13-wevj-15-00407" class="html-bibr">13</a>].</p>
Full article ">Figure 5
<p>Performance outcome of PSAU-Defender at varying distances for enhanced VANET security [<a href="#B8-wevj-15-00407" class="html-bibr">8</a>,<a href="#B9-wevj-15-00407" class="html-bibr">9</a>,<a href="#B10-wevj-15-00407" class="html-bibr">10</a>,<a href="#B11-wevj-15-00407" class="html-bibr">11</a>,<a href="#B12-wevj-15-00407" class="html-bibr">12</a>,<a href="#B13-wevj-15-00407" class="html-bibr">13</a>].</p>
Full article ">
22 pages, 17310 KiB  
Article
Adjacent Image Augmentation and Its Framework for Self-Supervised Learning in Anomaly Detection
by Gi Seung Kwon and Yong Suk Choi
Sensors 2024, 24(17), 5616; https://doi.org/10.3390/s24175616 - 29 Aug 2024
Cited by 1 | Viewed by 951
Abstract
Anomaly detection has gained significant attention with the advancements in deep neural networks. Effective training requires both normal and anomalous data, but this often leads to a class imbalance, as anomalous data is scarce. Traditional augmentation methods struggle to maintain the correlation between [...] Read more.
Anomaly detection has gained significant attention with the advancements in deep neural networks. Effective training requires both normal and anomalous data, but this often leads to a class imbalance, as anomalous data is scarce. Traditional augmentation methods struggle to maintain the correlation between anomalous patterns and their surroundings. To address this, we propose an adjacent augmentation technique that generates synthetic anomaly images, preserving object shapes while distorting contours to enhance correlation. Experimental results show that adjacent augmentation captures high-quality anomaly features, achieving superior AU-ROC and AU-PR scores compared to existing methods. Additionally, our technique produces synthetic normal images, aiding in learning detailed normal data features and reducing sensitivity to minor variations. Our framework considers all training images within a batch as positive pairs, pairing them with synthetic normal images as positive pairs and with synthetic anomaly images as negative pairs. This compensates for the lack of anomalous features and effectively distinguishes between normal and anomalous features, mitigating class imbalance. Using the ResNet50 network, our model achieved perfect AU-ROC and AU-PR scores of 100% in the bottle category of the MVTec-AD dataset. We are also investigating the relationship between anomalous pattern size and detection performance. Full article
Show Figures

Figure 1

Figure 1
<p>Images from the MVTec-AD dataset. This dataset comprises object and texture classes. Normal images feature a green border, while anomaly images are outlined in red. Defects in this dataset are indicated by a red border.</p>
Full article ">Figure 2
<p>Utilizing deep one-class classification for anomaly detection. This algorithm determines the normalcy of input data by assessing whether they resides within a hypersphere formed by normal data. The figure illustrates the process of constructing such a hypersphere using a neural net1work to discern the features characteristic of normal data.</p>
Full article ">Figure 3
<p>This figure compares <math display="inline"><semantics> <mrow> <msup> <mrow> <mi mathvariant="script">l</mi> </mrow> <mrow> <mn>2</mn> </mrow> </msup> </mrow> </semantics></math>-autoencoder and SSIM-autoencoder for anomaly detection using autoencoders. An autoencoder trained on normal data compresses input fabric textures and then reconstructs them as normal fabric textures. The <math display="inline"><semantics> <mrow> <msup> <mrow> <mi mathvariant="script">l</mi> </mrow> <mrow> <mn>2</mn> </mrow> </msup> </mrow> </semantics></math>-autoencoder removes defects during reconstruction, while the SSIM-autoencoder retains defects. Therefore, the SSIM-autoencoder shows better anomaly detection performance than the <math display="inline"><semantics> <mrow> <msup> <mrow> <mi mathvariant="script">l</mi> </mrow> <mrow> <mn>2</mn> </mrow> </msup> </mrow> </semantics></math>-autoencoder.</p>
Full article ">Figure 4
<p>Utilizing memory bank for anomaly detection. The memory bank retains features extracted from normal patches. The model then compares the features of the input image with those stored in the memory bank. If there’s at least one discrepancy between the input patches and the stored normal patches, the model classifies the input image as an anomaly.</p>
Full article ">Figure 5
<p>Illustration of the class imbalance problem. In anomaly detection, class imbalance occurs when the quantity of normal data points greatly surpasses that of anomaly data points. This imbalance poses challenges for both model training and performance assessment. Particularly, when anomaly data are scarce, the model may struggle to differentiate between normal and anomaly instances.</p>
Full article ">Figure 6
<p>The difference between (<b>a</b>) SimCLR framework and (<b>b</b>) adjacent framework. While the SimCLR framework designates the training data within the batch as negative pairs, the adjacent framework pairs them as positive pairs. Notably, the adjacent framework embeds the features of normal data into the hypersphere space, resulting in improved discrimination between the features of normal data and those of anomaly data.</p>
Full article ">Figure 7
<p>Depicted here is an image with Weak Overall augmentation. Weak Overall augmentation involves subtle adjustments to the anchor’s size and a mild application of Gaussian blur. Additionally, horizontal flipping occurs randomly with a specific probability. These Weak Overall samples aid in reducing sensitivity to minor overall changes.</p>
Full article ">Figure 8
<p>Depicted here is an image with Strong Overall augmentation. Strong Overall augmentation significantly alters the size and color of anchor images. Moreover, Gaussian blur, horizontal flipping, and grayscale are applied with varying probabilities. Strong Overall samples promote the learning of intricate features within normal images.</p>
Full article ">Figure 9
<p>Depicted here is an image with CutPaste augmentation. CutPaste augmentation entails cutting a square patch from the anchor image and pasting it onto the original image. These CutPaste samples, which distort continuous patterns in normal images, facilitate the learning of discontinuous features present in anomaly data.</p>
Full article ">Figure 10
<p>Depicted here is an image with SmoothBlend augmentation. SmoothBlend augmentation involves cutting a small, round patch from the anchor image and pasting it onto the original image. These SmoothBlend samples, which distort local detailed patterns in normal images, encourage the learning of detailed features found in anomaly data.</p>
Full article ">Figure 11
<p>Depicted here is an image with Mosaic (ζ = 20) augmentation. Mosaic augmentation transforms color and resolution by specifying circular areas in anchor images. These Mosaic samples, which distort the resolution and color patterns of normal images, encourage the learning of natural and small defects present in anomaly data.</p>
Full article ">Figure 12
<p>Depicted here is an image with Liquify (η = 0.03) augmentation. Liquify augmentation randomly selects a point on the training image and transforms its contours as they move. These Liquify samples maintain the shape of the normal image while distorting the contours, facilitating the learning of unnatural contours present in anomaly data.</p>
Full article ">Figure 13
<p>Depicted here is an image with Mosiquify augmentation. Mosiquify augmentation applies both Mosaic (ζ = 20) and Liquify (η = 0.03) augmentations to images. These Mosiquify samples, including two distorted anomalous patterns, promote the learning of various features from the anomaly images.</p>
Full article ">Figure 14
<p>Presented here are images generated by adjacent augmentations. This figure showcases images created through adjacent augmentations, where Strong Overall augmentation and Weak Overall augmentation produce synthetic normal data, while Mosaic augmentation, Liquify augmentation, and Mosiquify augmentation generate synthetic anomaly data.</p>
Full article ">Figure 15
<p>Overview of the adjacent augmentation and its framework. <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>x</mi> </mrow> <mrow> <mi>i</mi> </mrow> </msub> </mrow> </semantics></math>: normal image serving as anchor. <math display="inline"><semantics> <mrow> <msub> <mrow> <mover accent="true"> <mrow> <mi>x</mi> </mrow> <mo>^</mo> </mover> </mrow> <mrow> <mi>i</mi> </mrow> </msub> </mrow> </semantics></math>: positive sample generated by Strong Overall. <math display="inline"><semantics> <mrow> <msubsup> <mrow> <mover accent="true"> <mrow> <mi>x</mi> </mrow> <mo>~</mo> </mover> </mrow> <mrow> <mi>i</mi> </mrow> <mrow> <mo>+</mo> </mrow> </msubsup> </mrow> </semantics></math>: positive sample generated by Weak Overall. <math display="inline"><semantics> <mrow> <msubsup> <mrow> <mover accent="true"> <mrow> <mi>z</mi> </mrow> <mo>~</mo> </mover> </mrow> <mrow> <mi>i</mi> </mrow> <mrow> <mo>−</mo> </mrow> </msubsup> </mrow> </semantics></math>: negative sample generated by mimicking actual defects. The image passes through the encoder (f (∙)) to become a representation (h). The representation (h) passes through the projector (g (∙)), and then l2 normalization is applied to the projection (z).</p>
Full article ">Figure 16
<p>Comparison of real-world defects and synthetic anomaly images.</p>
Full article ">Figure 17
<p>Liquify anomalous pattern size according to η.</p>
Full article ">Figure A1
<p>Learning curves for each augmentation in the Bottle category.</p>
Full article ">Figure A2
<p>Accuracy curves for each augmentation in the Bottle category.</p>
Full article ">Figure A3
<p>ROC curves for each augmentation in the Bottle category.</p>
Full article ">Figure A4
<p>The confusion matrix is used to calculate performance metrics such as accuracy, precision, recall, and F1 score.</p>
Full article ">
Back to TopTop