CS 7643 QUIZ 3 QUESTIONS WELL ANSWERED LATEST UPDATE 2026

CS 7643 QUIZ 3 QUESTIONS WELL ANSWERED LATEST UPDATE 2026 Modeling Error - Answers Given a particular NN architecture, the actual model that represents the real world may not be in that space. When model complexity increases, modeling error reduces, but optimization error increases. Estimation Error - Answers Even if finding the best hypothesis, weights, and parameters that minimize training error, may not generalize to test set Optimization Error - Answers Even if your NN can perfectly model the world, your algo may not find good weights that model the function. When model complexity increases, modeling error reduces, but optimization error increases. Effectiveness of transfer learning under certain conditions - Answers Remove last FC layer of CNN and initialize it randomly, then run new data through network to train only that layer In order to train the NN for transfer learning -freeze the CNN layers or early layers and learn parameters in the FC layers. Performs very well on very small amount of training, if similar to the original data Does not work very well if the target task's dataset is very different If you have enough data in the target domain, and is different than the source, better to just train on the new data Transfer learning = reuse features we learn on a very large dataset on a completely new thing Steps: Train on very large dataset Take custom dataset and initialize network with weights trained in Step 1 (replace last fully connected layer since classes in new network will be different) Final step - continue training on new dataset Can either retrain all weights ("finetune") or freeze (ie: not update) weights in certain layers (freezing reduces number of parameters that you need to learn) AlexNet - Answers 2x(CONV=MAXPOOL=NORM)=3xCONV=MAXPOOL=3xFC ReLU, specialized normalization layers, PCA-based data augmentation, Dropout, Ensembling (used 7 NN with different random weights) Critical development: More depth and ReLU VGGNet - Answers 2x(2xCONV=POOL)=3x(3xCONV=POOL)=3xFC Repeated Application of 3x3 Conv (stride of 1, padding of) & 2x2 Max Pooling (stride 2) blocks Very large number of parameters (most in FC) layers, most memory in Conv Layers (you are storing activation produced in forward pass) Critical Development: Blocks of repeated structures Inception Net - Answers Deeper and more complex than VGGNet Average Pooling before FC Layer Repeated blocks that are repeated over again to form NN Blocks are made of simple layers, FC, Conv, MaxPool, and softmax Parallel filters of different sizes to get features at multiple scales Critical Development: Blocks of parallel paths Uses Network In Network concept i.e 1x1 Convolution -sort of Dimensionality reduction see slide Negative things: Increased Computational Work ResNet - Answers Allow information from a layer to propagate to a future layer Passes residuals of a layer at depth x and adds it to the output of the layer at x+1 Averaging block at end Critical Development: Passing residuals of previous layers forward Convolutional layers and how they work (forward/backward) - Answers (Don't have a good short summary) Equivariance - Answers If the input changes, the output changes in the same way if f(g(x) =g(f(x). If the beak of a bird in a picture moves a bit, the output values will move in the same way change to input causes equal change to output Invariance - Answers If the input changes, the output stays the same. That is f(g(x)) = f(x) E.g. rotating/scaling a number will still result in that number being classified the same.. change to input does not affect output Useful if we care more about if a feature is present than exactly where it is saliency maps - Answers Instead of using deconvnets, can, instead of taking the error gradient wrt model parameters, take gradient of class score we're trying to visualize wrt to the image itself (the input of the network) gets the degree to which a pixel contributed to that class score take absolute value of score because we care about degree, not direction helps us understand why model gives response they did Another method to make saliency maps: guided backpropagation algorithm (combination of deconvnet and gradient of class score wrt input of network) Sensitivity of loss to individual pixel changes , uses pre-softmax scores (gradient, then absval, then sum across channels) CAM = Class Activation Mapping - Answers use Global Average Pooling layer as final layer to average the activations of each feature map and run through softmax loss layer to highlight the important regions of the image by projecting back the weights of the output on the convolutional feature maps Grad-CAM - Answers more versatile version of CAM that can produce visual explanations for any arbitrary CNN, even if the network contains a stack of fully connected layers too let the gradients of any target concept score flow into the final convolutional layer; then compute an importance score based on the gradients and produce a coarse localization map highlighting the important regions in the image for predicting that concept What regions of image is model looking at to make prediction? Which individual regions have highest class activation as you extract layer from CNN? Direction/magnitude of gradients to determine which gradients are causing the most updates to the NN Objective: inspective given layer of CNN and correlate to output Task specific (if asked what is a dog - dog pixels are more important) Adversarial examples - Answers Inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Guided Backprop - Answers Layer by layer (deconvolution is similar to backprop) From details to more abstracted representations L1 Loss - Answers Sum of Absolute Value of (true - predicted) L2 Loss - Answers Sum of Absolute Value of (true - predicted)^2