[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

Liu et al., 2024 - Google Patents

Computation and parameter efficient multi-modal fusion transformer for cued speech recognition

Liu et al., 2024

View PDF
Document ID
14475571601227378580
Author
Liu L
Liu L
Li H
Publication year
Publication venue
IEEE/ACM Transactions on Audio, Speech, and Language Processing

External Links

Snippet

Cued Speech (CS) is a pure visual coding method used by hearing-impaired people that combines lip reading with several specific hand shapes to make the spoken language visible. Automatic CS recognition (ACSR) seeks to transcribe visual cues of speech into text …
Continue reading at arxiv.org (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62—Methods or arrangements for recognition using electronic means
    • G06K9/6217—Design or setup of recognition systems and techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06K9/6232—Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods
    • G06K9/6247—Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods based on an approximation criterion, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62—Methods or arrangements for recognition using electronic means
    • G06K9/6267—Classification techniques
    • G06K9/6268—Classification techniques relating to the classification paradigm, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62—Methods or arrangements for recognition using electronic means
    • G06K9/6267—Classification techniques
    • G06K9/6279—Classification techniques relating to the number of classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62—Methods or arrangements for recognition using electronic means
    • G06K9/68—Methods or arrangements for recognition using electronic means using sequential comparisons of the image signals with a plurality of references in which the sequence of the image signals or the references is relevant, e.g. addressable memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00221—Acquiring or recognising human faces, facial parts, facial sketches, facial expressions
    • G06K9/00268—Feature extraction; Face representation
    • G06K9/00281—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/36—Image preprocessing, i.e. processing the image information without deciding about the identity of the image
    • G06K9/46—Extraction of features or characteristics of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30781—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F17/30784—Information retrieval; Database structures therefor; File system structures therefor of video data using features automatically derived from the video content, e.g. descriptors, fingerprints, signatures, genre
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/3061—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00852—Recognising whole cursive words
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass
    • G06N99/005—Learning machines, i.e. computer in which a programme is changed according to experience gained by the machine itself during a complete run
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computer systems based on biological models
    • G06N3/02—Computer systems based on biological models using neural network models

Similar Documents

Publication Publication Date Title
Gao et al. Deep label distribution learning with label ambiguity
CN113420807A (en) Multi-mode fusion emotion recognition system and method based on multi-task learning and attention mechanism and experimental evaluation method
Peng et al. Recognition of handwritten Chinese text by segmentation: a segment-annotation-free approach
Huang et al. Multimodal continuous emotion recognition with data augmentation using recurrent neural networks
Wang et al. Stroke constrained attention network for online handwritten mathematical expression recognition
Yan et al. ConvMath: a convolutional sequence network for mathematical expression recognition
Xing et al. A convolutional neural network for aspect-level sentiment classification
Zhou et al. DPNet: Dual-path network for real-time object detection with lightweight attention
Ma et al. Tagging the web: Building a robust web tagger with neural network
Du et al. Expressive voice conversion: A joint framework for speaker identity and emotional style transfer
Liu et al. Multimodal emotion recognition based on cascaded multichannel and hierarchical fusion
Gajurel et al. A fine-grained visual attention approach for fingerspelling recognition in the wild
Liu et al. Computation and parameter efficient multi-modal fusion transformer for cued speech recognition
Gan et al. Unsupervised learning of sentence representations using convolutional neural networks
Liang et al. Learning atomic human actions using variable-length Markov models
Chaudhary et al. Signnet ii: A transformer-based two-way sign language translation model
Yin et al. Spatial temporal enhanced network for continuous sign language recognition
Xie et al. A multimodal fusion emotion recognition method based on multitask learning and attention mechanism
Xue et al. Lcsnet: End-to-end lipreading with channel-aware feature selection
Boukdir et al. Character-level Arabic text generation from sign language video using encoder–decoder model
CN116701996A (en) Multi-modal emotion analysis method, system, equipment and medium based on multiple loss functions
Mu et al. MOCOLNet: A Momentum Contrastive Learning Network for Multimodal Aspect-Level Sentiment Analysis
CN112651225B (en) Multi-item selection machine reading understanding method based on multi-stage maximum attention
Wang et al. MT-TCCT: Multi-task learning for multimodal emotion recognition
Vashistha et al. Active learning for neural machine translation