[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

Nguyen et al., 2021 - Google Patents

Automatic language identification in code-switched Hindi-English social media text

Nguyen et al., 2021

View HTML
Document ID
1830057927422720527
Author
Nguyen L
Bryant C
Kidwai S
Biberauer T
Publication year
Publication venue
Journal of Open Humanities Data

External Links

Snippet

Abstract Natural Language Processing (NLP) tools typically struggle to process code- switched data and so linguists are commonly forced to annotate such data manually. As this data becomes more readily available, automatic tools are increasingly needed to help …
Continue reading at openhumanitiesdata.metajnl.com (HTML) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2765—Recognition
    • G06F17/277—Lexical analysis, e.g. tokenisation, collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/21—Text processing
    • G06F17/22—Manipulating or registering by use of codes, e.g. in sequence of text characters
    • G06F17/2217—Character encodings
    • G06F17/2223—Handling non-latin characters, e.g. kana-to-kanji conversion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/3061—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F17/30634—Querying
    • G06F17/30657—Query processing
    • G06F17/30675—Query execution
    • G06F17/30684—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2765—Recognition
    • G06F17/2775—Phrasal analysis, e.g. finite state techniques, chunking
    • G06F17/278—Named entity recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/2809—Data driven translation
    • G06F17/2827—Example based machine translation; Alignment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2705—Parsing
    • G06F17/271—Syntactic parsing, e.g. based on context-free grammar [CFG], unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2795—Thesaurus; Synonyms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/2863—Processing of non-latin text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/274—Grammatical analysis; Style critique
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/2872—Rule based translation
    • G06F17/2881—Natural language generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/21—Text processing
    • G06F17/22—Manipulating or registering by use of codes, e.g. in sequence of text characters
    • G06F17/2247—Tree structured documents; Markup, e.g. Standard Generalized Markup Language [SGML], Document Type Definition [DTD]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2735—Dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/21—Text processing
    • G06F17/24—Editing, e.g. insert/delete
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/21—Text processing
    • G06F17/211—Formatting, i.e. changing of presentation of document
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2785—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2755—Morphological analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/289—Use of machine translation, e.g. multi-lingual retrieval, server side translation for client devices, real-time translation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/273—Orthographic correction, e.g. spelling checkers, vowelisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/3061—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F17/30613—Indexing

Similar Documents

Publication Publication Date Title
Zaidan et al. Arabic dialect identification
Sharma et al. Text normalization of code mix and sentiment analysis
Nguyen et al. Automatic language identification in code-switched Hindi-English social media text
Avner et al. Identifying translationese at the word and sub-word level
Simon et al. Introducing NYTK-NerKor, a gold standard Hungarian named entity annotated corpus
Mosavi Miangah FarsiSpell: A spell-checking system for Persian using a large monolingual corpus
Şeker et al. Extending a CRF-based named entity recognition model for Turkish well formed text and user generated content 1
Sharma et al. Sentiment analysis of code-mix script
Van Der Goot et al. Lexical normalization for code-switched data and its effect on POS-tagging
Hkiri et al. Arabic-English text translation leveraging hybrid NER
Korre et al. ELERRANT: Automatic grammatical error type classification for Greek
Jacksi et al. The Kurdish Language corpus: state of the art
Fashwan et al. A morphologically annotated corpus and a morphological analyzer for Egyptian Arabic
Salam et al. Developing the bangladeshi national corpus-a balanced and representative bangla corpus
Stankovic et al. Sentiment Analysis of Sentences from Serbian ELTeC corpus
Başıbüyük et al. Usage disambiguation of Turkish discourse connectives
Borin et al. Language technology for digital linguistics: Turning the linguistic survey of India into a rich source of linguistic information
Xia et al. Accurate Pinyin-English codeswitched language identification
Eyecioglu et al. Knowledge-lean paraphrase identification using character-based features
Seretan et al. Syntactic concordancing and multi-word expression detection
Klimov et al. Towards Intelligent Correction of Collocational Errors in Russian L2 Academic Texts in the CAT&kittens Writing Support Platform
Masanti et al. Novel Benchmark Data Set for Automatic Error Detection and Correction
Gupta Automatic Normalization of Punjabi Words
Maynard et al. MUSE: a MUlti-Source Entity recognition system
Verulkar et al. Transliterated search of Hindi lyrics