Natural Language Processing (CS4990)
Why Python? - answer- Shallow learning curve
- Good string handling
- Combines OO, aspect-oriented and FP paradigms
- Extensive standard libraries (e.g. NLTK)
- Great support for Deep Learning
Human language - answer Ultimate interface for interaction and communication.
But something to understand, because it's:
- highly ambiguous at all level
- complex and a subtle use of context to convey meaning
- fuzzy and probabilistic
Understanding a language requires domain knowledge, discourse knowledge, world
knowledge and linguistic knowledge
Word level ambiguity - answer- Spelling (e.g. colour vs color)
- Pronunciation
• 1 word can have multiple pronunciations (e.g. abstract, desert)
• Multiple words can share the same pronunciation (e.g. flower/flour)
- Meaning (1 word can have multiple meanings, i.e. homonyms; e.g. date, crane,
leaves)
Natural Language Processing (NLP) - answerA subfield of linguistics, CompSci,
Information Engineering and AI concerned with the interactions between computers and
human (natural) languages, in particular how to program computers to process and
analyse large amounts of natural language data
NLP tasks & applications - answer- Writing assistance (spell/grammar/style checking,
auto completion).
- Text classification (spam detection, sentiment analyses, fake news/propaganda
detection, news topic classification, customer reviews category classification).
- Information retrieval (search engine)
- NL Understanding (argumentation mining, question-answering, NL inference,
humorous/ironic/metaphoric language analysis).
- NL generation (document summarisation, machine translation, sentence
paraphrasing/simplification, dialogue/exercise generation)
NLP limits & outlook - answer- Language problems are hard - for most of them, there's
still no fully accurate solution (like Physics, History and Psychology).
Data types (based on structures) - answer- Structured data
,- Semi-structured data
- Unstructured data
Corpus (=body) - answerA large body of text.
It usually contains raw text and any metadata associated with the text (e.g. timestamp,
source, index, ...).
It's also known as a dataset
Text cleaning & normalisation - answerRemove useless information (e.g. email
headers) and extract useful information (e.g. words, word sequences, verbs, nouns,
adjectives, names, locations, orgs, ...).
1. Tokenization (sentence, words)
2. Stemming / Lemmatization
3. Stop-words removal
Tokenization - answerProcess of splitting sentences into their constituents, i.e. tokens
(generally done by white-space or punctuation character separation in English), which
are meaningful segments.
Type - answerElement in the vocabulary. Also known as the form or spelling of the
token (including words and punctuation) independently of its specific occurrences in a
text.
Token - answerInstance of a type in a text, which is a sequence of characters that is
treated as a single group (i.e. words and punctuation).
E.g. To be or not to be
- 2x to, be
- 1x or, not
Simple tokenization - answerSplit with white-space (for English texts).
Pros: simple and natively supported by Python.
Cons: it fails to tokenize punctuation and hyphenated words (e.g. "state-of-the-art").
Natural Language Tool Kit (NLTK) - answer(FOSS) Python library to make programs
that work with NL.
It can perform different operations such as tokenization, stemming, classification,
parsing, tagging and semantic reasoning.
Word tokenizer (from NLTK) - answerNLTK' standard tokenizer.
Pros: successfully tokenizes punctuations, split hashtags into separate words (e.g.
#70thRepublic_Day into "#" and "70thRepublic_Day")
Cons: it fails to identify widely used symbol combinations (e.g. ":)" is split into 2
symbols)
, Tweet tokenizer (from NLTK) - answerPros: correctly handles hashtags and mentions
(`@somone`)
Cons: it fails at abbreviations (e.g. U.K)
Sentence tokenization - answerFor long documents, we may not be interested in words
but instead in sentences therein:
- Check whether a sentence's sentiment is positive or negative.
- Check whether a sentence contains propaganda content.
- Check the grammatical correctness of a sentence
- ...
Stemming - answerProcess of reducing the inflection in words to their root forms such
as mapping a group of words to the same stem even if the stem itself isn't a valid word
in the language.
NLTK includes 2 widely used ones: Port Stemmer and Lancaster Stemmer (younger
and more aggressive); they both regard an input text as a single word.
Pros: quick to run (because it's based on simple rules) and suitable for processing a
large amount of text
Cons: the resulting words may not carry any meaning (or be actual words)
Inflection (in grammar) - answerModification of a word to express different grammatical
categories such as tense, case, voice, aspect, person, number, gender and mood.
It expresses 1+ grammatical categories with a prefix, suffix, infix or another internal
modification such as vowel change.
Examples: removing ..ed, ..ize, ..s, ..de, mid from words.
Lemmatization - answerProcess of reducing the inflection in words to their dictionary
root forms (called lemma), ensuring that the root word belongs to the language.
Pros: the derived root word is meaningful
Cons: it's more expensive (hence slow) to run
Lemma - answerCanonical/dictionary/citation form of a set of words
Stop-words - answerSmall, unimportant words in a NL query, (such as "am", "the", "to"
and "are") which support words and sentences and help to construct sentences. But
they don't affect the meaning of the sentence.
Regular Expression (RegEx or RegExp) - answerA sequence of characters that define a
search pattern.
Usually, such patterns are used by string searching algorithms for "find" and "find and
replace" operations on strings or for input validation.
A key characteristic of it is its economical script (especially with the Kleene closures
with + and *).
Why Python? - answer- Shallow learning curve
- Good string handling
- Combines OO, aspect-oriented and FP paradigms
- Extensive standard libraries (e.g. NLTK)
- Great support for Deep Learning
Human language - answer Ultimate interface for interaction and communication.
But something to understand, because it's:
- highly ambiguous at all level
- complex and a subtle use of context to convey meaning
- fuzzy and probabilistic
Understanding a language requires domain knowledge, discourse knowledge, world
knowledge and linguistic knowledge
Word level ambiguity - answer- Spelling (e.g. colour vs color)
- Pronunciation
• 1 word can have multiple pronunciations (e.g. abstract, desert)
• Multiple words can share the same pronunciation (e.g. flower/flour)
- Meaning (1 word can have multiple meanings, i.e. homonyms; e.g. date, crane,
leaves)
Natural Language Processing (NLP) - answerA subfield of linguistics, CompSci,
Information Engineering and AI concerned with the interactions between computers and
human (natural) languages, in particular how to program computers to process and
analyse large amounts of natural language data
NLP tasks & applications - answer- Writing assistance (spell/grammar/style checking,
auto completion).
- Text classification (spam detection, sentiment analyses, fake news/propaganda
detection, news topic classification, customer reviews category classification).
- Information retrieval (search engine)
- NL Understanding (argumentation mining, question-answering, NL inference,
humorous/ironic/metaphoric language analysis).
- NL generation (document summarisation, machine translation, sentence
paraphrasing/simplification, dialogue/exercise generation)
NLP limits & outlook - answer- Language problems are hard - for most of them, there's
still no fully accurate solution (like Physics, History and Psychology).
Data types (based on structures) - answer- Structured data
,- Semi-structured data
- Unstructured data
Corpus (=body) - answerA large body of text.
It usually contains raw text and any metadata associated with the text (e.g. timestamp,
source, index, ...).
It's also known as a dataset
Text cleaning & normalisation - answerRemove useless information (e.g. email
headers) and extract useful information (e.g. words, word sequences, verbs, nouns,
adjectives, names, locations, orgs, ...).
1. Tokenization (sentence, words)
2. Stemming / Lemmatization
3. Stop-words removal
Tokenization - answerProcess of splitting sentences into their constituents, i.e. tokens
(generally done by white-space or punctuation character separation in English), which
are meaningful segments.
Type - answerElement in the vocabulary. Also known as the form or spelling of the
token (including words and punctuation) independently of its specific occurrences in a
text.
Token - answerInstance of a type in a text, which is a sequence of characters that is
treated as a single group (i.e. words and punctuation).
E.g. To be or not to be
- 2x to, be
- 1x or, not
Simple tokenization - answerSplit with white-space (for English texts).
Pros: simple and natively supported by Python.
Cons: it fails to tokenize punctuation and hyphenated words (e.g. "state-of-the-art").
Natural Language Tool Kit (NLTK) - answer(FOSS) Python library to make programs
that work with NL.
It can perform different operations such as tokenization, stemming, classification,
parsing, tagging and semantic reasoning.
Word tokenizer (from NLTK) - answerNLTK' standard tokenizer.
Pros: successfully tokenizes punctuations, split hashtags into separate words (e.g.
#70thRepublic_Day into "#" and "70thRepublic_Day")
Cons: it fails to identify widely used symbol combinations (e.g. ":)" is split into 2
symbols)
, Tweet tokenizer (from NLTK) - answerPros: correctly handles hashtags and mentions
(`@somone`)
Cons: it fails at abbreviations (e.g. U.K)
Sentence tokenization - answerFor long documents, we may not be interested in words
but instead in sentences therein:
- Check whether a sentence's sentiment is positive or negative.
- Check whether a sentence contains propaganda content.
- Check the grammatical correctness of a sentence
- ...
Stemming - answerProcess of reducing the inflection in words to their root forms such
as mapping a group of words to the same stem even if the stem itself isn't a valid word
in the language.
NLTK includes 2 widely used ones: Port Stemmer and Lancaster Stemmer (younger
and more aggressive); they both regard an input text as a single word.
Pros: quick to run (because it's based on simple rules) and suitable for processing a
large amount of text
Cons: the resulting words may not carry any meaning (or be actual words)
Inflection (in grammar) - answerModification of a word to express different grammatical
categories such as tense, case, voice, aspect, person, number, gender and mood.
It expresses 1+ grammatical categories with a prefix, suffix, infix or another internal
modification such as vowel change.
Examples: removing ..ed, ..ize, ..s, ..de, mid from words.
Lemmatization - answerProcess of reducing the inflection in words to their dictionary
root forms (called lemma), ensuring that the root word belongs to the language.
Pros: the derived root word is meaningful
Cons: it's more expensive (hence slow) to run
Lemma - answerCanonical/dictionary/citation form of a set of words
Stop-words - answerSmall, unimportant words in a NL query, (such as "am", "the", "to"
and "are") which support words and sentences and help to construct sentences. But
they don't affect the meaning of the sentence.
Regular Expression (RegEx or RegExp) - answerA sequence of characters that define a
search pattern.
Usually, such patterns are used by string searching algorithms for "find" and "find and
replace" operations on strings or for input validation.
A key characteristic of it is its economical script (especially with the Kleene closures
with + and *).
