AI Notes
Master image classification using CNNs. Learn ResNet, VGG, Inception, transfer learning, fine-tuning, ImageNet. Complete computer vision guide 2024.
Problem Statement
Image classification assigns an input image to one of N predefined categories. Given a photograph, the system must determine its class—dog breed, vehicle type, medical condition, or any other categorization scheme. This seemingly simple task requires the model to handle enormous variation in viewpoint, lighting, scale, occlusion, and intra-class diversity.
Image classification is the fundamental computer vision task upon which more complex problems build: object detection (classification + localization), segmentation (per-pixel classification), and image captioning all begin with classification capabilities. Mastering classification architectures provides the foundation for all modern computer vision.
The Deep Learning Pipeline
Pretrained Architectures
ResNet (Residual Networks, 2015)
The architecture that enabled training very deep networks through skip connections:
| Key Innovation | Residual Learning |
| x ── | [Conv→BN→ReLU→Conv→BN] → (+) → ReLU → output |
| ResNet-18 | 69.8% top-1 (11.7M params) |
| ResNet-50 | 76.1% top-1 (25.6M params) ← most popular |
| ResNet-101 | 77.4% top-1 (44.5M params) |
| ResNet-152 | 78.3% top-1 (60.2M params) |
VGG Networks (2014)
Demonstrated that depth with small filters works:
| Architecture philosophy | Use ONLY 3×3 filters |
| Conv3-64 × 2 | MaxPool |
| Conv3-128 × 2 | MaxPool |
| Conv3-256 × 3 | MaxPool |
| Conv3-512 × 3 | MaxPool |
| Conv3-512 × 3 | MaxPool |
| FC-4096 | FC-4096 → FC-1000 → Softmax |
| Performance | 71-74% ImageNet top-1 |
| Limitation | 138M parameters (heavy for deployment) |
Inception/GoogLeNet (2014)
Multi-scale feature extraction within each layer:
| / | \ |
|---|---|
| \ | / |
EfficientNet (2019)
Systematic scaling of depth, width, and resolution:
| depth | d = α^φ |
| width | w = β^φ |
| resolution | r = γ^φ |
| α × β² × γ² ≈ 2 (constraint | doubling compute) |
| EfficientNet-B0 | 77.1% accuracy, 5.3M params |
| EfficientNet-B7 | 84.3% accuracy, 66M params |
| Key insight | Scaling all three dimensions proportionally |
Transfer Learning
Why Transfer Learning Works
ImageNet pretrained networks learn general visual features
Layer 1-2: Edges, textures, colors (universal)
Layer 3-5: Parts, patterns (somewhat general)
Layer 6+: Object-specific features (task-dependent)
Transfer strategy
- Keep early layers frozen (general features)
- Fine-tune later layers (task-specific adaptation)
- Replace final classifier (new task's classes)
Fine-Tuning Protocol
| Step 1 | Replace classifier head |
| Step 2 | Freeze backbone initially |
| Step 3 | Train head only (5-10 epochs, lr=0.001) |
| Step 4 | Unfreeze and fine-tune entire model (10-20 epochs, lr=0.0001) |
| Step 5 | Learning rate scheduling |
| Discriminative lr | lower layers get smaller lr |
| Layer 1 | lr=1e-6, Layer 2: lr=1e-5, ... Head: lr=1e-3 |
Data Augmentation
Training-time augmentation (generates variety)
- Random horizontal flip (50% probability)
- Random rotation (±15 degrees)
- Random crop (resize to 256, random crop to 224)
- Color jitter (brightness, contrast, saturation)
- Random erasing (occlude random patches)
- Mixup: blend two images + blend labels
- CutMix: paste patch from one image onto another
Test-time augmentation (TTA)
- Classify original + flipped + multi-crop
- Average predictions → more robust
- Typical improvement: 1-2% accuracy
Training Best Practices
| Optimizer | AdamW (weight decay decoupled from gradient) |
| Learning rate | Cosine annealing with warmup |
| Warmup | linear increase over first 5 epochs |
| Then | cosine decay to near zero |
| Batch size | Largest that fits in GPU memory (32-256) |
| Larger batch | more stable gradients → higher lr possible |
| - Label smoothing (0.1) | soft targets prevent overconfidence |
| - Early stopping | monitor validation loss |
Evaluation
Metrics
Top-1 accuracy: correct if highest probability = true class
Top-5 accuracy: correct if true class in top 5 predictions
Confusion matrix: shows which classes are confused
Per-class precision/recall: identifies weak categories
Common issues
- Class imbalance: use weighted loss or oversampling
- Fine-grained classification: birds/cars subspecies need attention mechanisms
- Domain shift: model trained on one style fails on another
Modern Trends
| Split image into 16×16 patches | treat as sequence |
| Apply transformer encoder | classification |
| Zero-shot classification | describe classes in text |
| "A photo of a [class]" | no fine-tuning needed! |
| DINOv2, SAM | pretrained on billions of images |
| Emerging | few-shot and zero-shot classification |
Interview Questions
Q: Why does ResNet work better than plain deep networks? A: Without skip connections, deep networks suffer from degradation—adding layers paradoxically increases training error. Skip connections solve this by allowing gradients to flow directly backward and enabling each block to learn residual corrections rather than complete transformations. The identity mapping provides a "gradient highway."
Q: When should you fine-tune vs. train from scratch? A: Fine-tune when: your dataset is small (<10k images), your domain is similar to ImageNet (natural photos), or you need quick results. Train from scratch when: you have millions of labeled images, your domain is very different (medical imaging, satellite), or you need a custom architecture.
Q: What is the role of Global Average Pooling? A: GAP replaces the fully connected layers that traditional CNNs used between features and classifier. It averages each feature map to a single number, creating a vector regardless of spatial dimensions. Benefits: no learnable parameters (reduces overfitting), works with any input size, provides spatial invariance.
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Image Classification with Deep Learning - CNN Guide.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Artificial Intelligence topic.
Search Terms
artificial-intelligence, artificial intelligence, artificial, intelligence, computer, vision, image, classification
Related Artificial Intelligence Topics