CS7643 Quiz 4 Questions and Answers| |100%
Correct-GT 5|New 49 Questions and Answers|2025
Updated
Embedding - answer-A learned map from entities to vectors that encodes
similarity
Graph Embedding - answer-Optimize the objective that connected nodes have
more similar embeddings than unconnected nodes.
Task: convert nodes to vectors
- effectively unsupervised learning where nearest neighbors are similar
- these learned vectors are useful for downstream tasks
Long Short-Term Memory Network Gates and States - answer-- f(t) = forget
gate
- i(t) = input gate
- u(t) = candidate update gate
- o(t) = output gate
, - c(t) = cell state
- c(t) = f(t) * c(t - 1) + i(t) * u(t)
- h(t) = hidden state
- h(t) = o(t) * tanh(c(t))
Perplexity(s) - answer-= product( 1 / P(w(i) | w(i-1), ...) ) ^ (1 / N)
= b ^ (-1/N sum( log(b) (P(w(i) | w(i-1), ...) ) )
- note exponent of b is per word CE loss
- perplexity of a discrete uniform distribution over k events is k
Language Model Goal - answer-- estimate the probability of sequences of
words
- p(s) = p(w1, w2, ..., wn)
Masked Language Modeling - answer-- pre-training task - an auxiliary task
different from the final task we're really interested in, but which can help us
achieve better performance finding good initial parameters for the model
- By pre-training on masked language modeling before training on our final
task, it is usually possible to obtain higher performance than by simply training
on the final task
Correct-GT 5|New 49 Questions and Answers|2025
Updated
Embedding - answer-A learned map from entities to vectors that encodes
similarity
Graph Embedding - answer-Optimize the objective that connected nodes have
more similar embeddings than unconnected nodes.
Task: convert nodes to vectors
- effectively unsupervised learning where nearest neighbors are similar
- these learned vectors are useful for downstream tasks
Long Short-Term Memory Network Gates and States - answer-- f(t) = forget
gate
- i(t) = input gate
- u(t) = candidate update gate
- o(t) = output gate
, - c(t) = cell state
- c(t) = f(t) * c(t - 1) + i(t) * u(t)
- h(t) = hidden state
- h(t) = o(t) * tanh(c(t))
Perplexity(s) - answer-= product( 1 / P(w(i) | w(i-1), ...) ) ^ (1 / N)
= b ^ (-1/N sum( log(b) (P(w(i) | w(i-1), ...) ) )
- note exponent of b is per word CE loss
- perplexity of a discrete uniform distribution over k events is k
Language Model Goal - answer-- estimate the probability of sequences of
words
- p(s) = p(w1, w2, ..., wn)
Masked Language Modeling - answer-- pre-training task - an auxiliary task
different from the final task we're really interested in, but which can help us
achieve better performance finding good initial parameters for the model
- By pre-training on masked language modeling before training on our final
task, it is usually possible to obtain higher performance than by simply training
on the final task
