Short Note on Linguistic Feature

Example of Linguistic Features

Lexical Features

• word
  
• pos
• syllables (estimated based on distribution patterns of vowels and consonants)
• position (begin/end/middle)

Syntactic Features

• depending on tool but mostly features from parse tree?

Example Tool for Generating Syntactic Features

• Link Grammar: context-free lexicalized grammar. Rules are link requirements (set of disjuncts of possible usage of word). Word sequence belongs to grammar if linkage is planar (connected graph where at most one link between each word pair and no cross link)

Some open issues in NLP

• Still lack of semantic parsers for general domain [Shi05EMNLP]

Note Don't forget that "A journey of a thousand miles begin with a single step": Confucius

Some Trends of Asian Language Technology

• segmentation and tokenization: segment into unique word tokens   
• Lemmatization: dictionary base form for an inflected verb or adjective 
• Noun Decompounding: separate compound nouns
• POS tagging
• Sentence boundary detection
• Base NP analysis: identify sets of words including a noun which describes a single nominal expression 
•