Attention in Rule-Based Machine Learning: Exploiting Learning Classifier Systems' Generalization for Image Classification

Deep learning is a cutting-edge methodology that has been widely used in real-world applications to solve computer vision tasks. Deep learning models are typically seen as black boxes, opaque, and difficult to interpret. Recently, attention-based vision transformers have been introduced to overcome the black-box behavior of deep networks. However, the decision-making process of the vision transformer is still not interpretable. Moreover, these models require a large amount of memory, huge computational resources, and enormous training data.

Learning classifier systems is a state-of-the-art rule-based evolutionary machine learning technique that stands out for its ability to provide interpretable decisions. These systems generate niche-based solutions, require less memory, and can be trained using small data sets. We hypothesize to wangle attention in learning classifier systems to identify critical components of the problem instance, link features to create simple patterns, and model hierarchical relationships in the data. The experimental results for binary-class image classification (cat and dog) tasks demonstrate that the novel system successfully ignores the irrelevant parts and pays attention to the salient features of cats and dogs. Crucially, the novel system exhibits almost the same performance accuracy as that of the state-of-the-art learning classifier systems.