CS 7641 MIDTERM 1 EXAM QUESTIONS WELL ANSWERED LATEST UPDATE 2026

CS 7641 MIDTERM 1 EXAM QUESTIONS WELL ANSWERED LATEST UPDATE 2026 Induction - Answers the process that moves from a given series of specifics to a generalization Deduction - Answers the process of moving from a general rule to a specific example Supervised Learning - Answers Use labeled training data to generalize labels to new instances (function approximation) Unsupervised Learning - Answers Make sense out of unlabeled data (data description) Reinforcement Learning - Answers Learning from delayed reward Classification versus Regression - Answers Classification is process of mapping x to a discrete label (e.g., T/F, M/F, 0/1,red/blue/green); regression is mapping of x to continuous values in R Instances - Answers Vectors of attributes to describe input Concept - Answers Function that maps inputs to outputs Target Concept - Answers The concept that we are trying to find Hypothesis Class - Answers All functions I'm willing to consider Candidate - Answers Concept that might be the target concept Decision Tree Algorithm - Answers 1. Pick "Best" Attribute 2. Ask question 3. Follow the answer path 4. Go to 1 until got answer ID3 algorithm - Answers Loop: A- best attribute Maximize information Gain(S,A)=Entropy(S)-∑|S_v|/|S|Entropy(S_v)) Assign A as decision attribute for node For face value of A, create descendant of node Sort Training Examples to Leaves If examples perfectly classified, stop. Else, iterate over leaves Entropy - Answers A measure of disorder or randomness. Entropy=-∑p(v)log p(v) where p is the probability of observing the value v. ID3 Inductive bias - Answers - Good splits at top rather than bottom - Correct over incorrect - Shorter trees to longer trees (follows from 1st 2) Inductive bias - Answers Set of assumptions that the learner uses to predict outputs given inputs that it has not encountered. Restriction bias is where the set of hypothesis considered is restricted to a smaller set. Preference bias is where some hypothesis are preferred over others. Where do errors come from? - Answers - Measurement/sensor - Malicious - Transcription error - Unmodeled influences Perceptron - Answers The (binary) linear classifier that has: - Input values or One input layer - Weights and Bias - Net sum - Activation Function Perceptron Rule - Answers wₖ=wₖ+∆wₖ ∆wₖ=η(y-ŷ)xₖ ŷ=∑wₖxₖ≥0 Gradient Descent Update - Answers More robust to nonlinear separability a=∑wₖxₖ Minimize error metric E(w)=½∑(y-a)² Sigmoid - Answers σ(a)=1/(1+e^-a) a→∞ , then σ(a)→0 a→-∞ , then σ(a)→1 Derivative Dσ(a)=σ(a)(1-σ(a)) Restriction bias of perceptron - Answers Half spaces Curse of Dimensionality - Answers As the number of features or dimensions grows, the amount of data we need to generalize accurately grow exponentially, e.g. kNN. Bagging (Bootstrap Aggregating) - Answers Take random subsets and combine by the mean. Ensemble Boosting - Answers TBD Difference between Bagging and Boosting - Answers 1. In Boosting, each model is built on top of the previous ones. Whereas in bagging each model is built independently. 2. The final boosting ensemble uses weighted majority vote while bagging uses a simple majority vote. 3. Bagging is a method of reducing variance while boosting can reduce the variance and bias of the base classifier 4. Boosting is better than bagging on non-noisy data 5. Bagging is effective more often than boosting The main advantages of Ensemble learning methods are : - Answers 1. Reduced variance : Overcome overfitting problem. Low variance means model independent of training data. Results are less dependent on features of a single model and training set. 2. Reduced bias : Overcome underfitting problem. Low bias means linear regression applied to linear data, second degree polynomial applied to quadratic data. Combination of multiple classifiers may produce more reliable classification than single classifier. Bayes Theorem Formula - Answers P(h|D)=P(D|h)P(h)/P(D) MAP (Maximum a posteriori) Hypothesis - Answers hₘₐₚ=argmax(h∈H) P(h|D)=argmax(h∈H) P(D|h)P(h) If all H are equally probable a priori, P(h) same for all h, then hₘₐₚ equals the maximum likelihood hypothesis hₘₗ hₘₐₚ=hₘₗ=argmax(h∈H) P(D|h) version space VS_(H,D) - Answers With respect to hypothesis space H and training examples D, the subset of hypotheses from H that are consistent with the training examples in D. Minimum length description principle - Answers One should prefer the model that yields the shortest description of the data when the complexity of the model itself is also accounted for hₘₐₚ=argmin(h∈H)[-log₂P(D|h)-log₂P(h)] = length of P(D|h) length(h) Interpretation: If hypothesis models data well, we don't need much to describe data. Otherwise, need to describe the error; length(D|h) is misclassification or "error" Bayes optimal classifier - Answers Answers "What is the most probably classification of the new instance given the training data?" argmax{v∈V} ∑P(v|hᵢ)P(hᵢ|D) Most probable classification of new instance is combination of predictions of all hypotheses weighted by posterior probabilities of the hypotheses given the data. Naive Bayes Classifier - Answers vₘₐₚ=argmax{v∈V} P(a1,a2,..,an|v)P(v) Under assumption that features are independent of each other: v_NB=argmax{v∈V} P(v) Π P(aᵢ|v) P(v) and P(aᵢ|v) are estimated based on their frequencies in the training data When assumption is satisfied, v_NB is identical to MAP classification Bayes Net / Belief Network - Answers TBD Randomized Restart Hill Climbing - Answers A standard hill climbing approach where optima are found by exploring a solution space and moving in the direction of increased fitness on each iteration. Repeat: