Common ML Architectures Reference: CNN, RNN, Transformer, MoE

Architecture choice shapes what a model can learn. This reference covers the major neural network families - their mechanisms, practical tradeoffs, and the problems they were built to solve.

Convolutional Neural Networks (CNN)

What it does

A CNN processes data with local spatial structure (images, audio spectrograms, 1D signals) by applying learned filters that detect local patterns regardless of position.

Core mechanism

Input image (H × W × C)
   ↓ Convolution layer: applies K filters of size (f × f)
   ↓ Activation (ReLU)
   ↓ Pooling (reduces spatial size)
   ↓ ... repeat ...
   ↓ Flatten + Dense layers
   ↓ Output (classes / embedding)

Convolution: a filter slides across the input, computing dot products at each position. The filter learns to detect edges, textures, shapes at different scales.

Pooling: reduces spatial dimensions. Max pooling takes the maximum in each window, introducing translation invariance.

Parameter sharing: the same filter is applied everywhere in the input. A filter detecting a horizontal edge works at any position.

Key architectures

Architecture	Year	Key Innovation
AlexNet	2012	ReLU activation, dropout, GPU training
VGG	2014	Very deep networks with 3×3 filters only
ResNet	2015	Skip connections, enables 100+ layer networks
EfficientNet	2019	Neural architecture search, compound scaling
ViT	2020	Patches + Transformer, no convolution

When to use

• Image classification, object detection, segmentation
• Audio processing (spectrograms)
• 1D signals with local patterns (time series, DNA sequences)
• Transfer learning: fine-tune a pre-trained ResNet or EfficientNet for your task

Tradeoffs

• Strong for 2D spatial data; poor for long-range dependencies
• Requires more data than classical methods for training from scratch
• Pre-trained models via transfer learning dramatically reduce data requirements

Recurrent Neural Networks (RNN) and LSTMs

What it does

Processes sequential data by maintaining a hidden state that carries information from previous time steps.

Core mechanism

At each time step t:
h_t = tanh(W_h · h_{t-1} + W_x · x_t + b)

Problem: vanilla RNNs suffer from vanishing gradients. Gradients shrink exponentially through many time steps - the model forgets distant context.

LSTM (Long Short-Term Memory)

Adds a cell state and gating mechanisms to selectively remember and forget:

Forget gate:  f_t = σ(W_f · [h_{t-1}, x_t] + b_f)
Input gate:   i_t = σ(W_i · [h_{t-1}, x_t] + b_i)
Cell update:  c̃_t = tanh(W_c · [h_{t-1}, x_t] + b_c)
Cell state:   c_t = f_t * c_{t-1} + i_t * c̃_t
Output gate:  o_t = σ(W_o · [h_{t-1}, x_t] + b_o)
Hidden state: h_t = o_t * tanh(c_t)

The cell state acts as a long-term memory that can be preserved over many steps.

When to use

• Time series forecasting (short horizons, < 200 steps)
• Sequence labeling where Transformers are overkill (small data)
• On-device inference where Transformer compute is prohibitive

Tradeoffs

• Slow to train: sequential computation cannot be parallelized
• Largely superseded by Transformers for NLP
• Still competitive for short time series with small datasets (< 50K)

Transformer

What it does

Processes sequences by computing attention between every pair of positions simultaneously, enabling full parallelization and long-range dependency modeling.

Core mechanism: Self-Attention

For each position in the sequence, compute:

Q = X · W_Q    (Query)
K = X · W_K    (Key)
V = X · W_V    (Value)

Attention(Q, K, V) = softmax(Q · Kᵀ / √d_k) · V

Each position attends to every other position. The attention weight tells the model how much to focus on each position when computing the output.

Multi-head attention: run H attention heads in parallel, each learning different relationships. Concatenate and project.

Positional encoding: since attention is permutation-invariant, positions are injected via sinusoidal or learned embeddings.

Feed-forward sublayer: after attention, each position goes through an MLP independently:

FFN(x) = max(0, x · W₁ + b₁) · W₂ + b₂

Encoder vs. Decoder

Component	Purpose	Examples
Encoder	Builds contextual representations	BERT, RoBERTa
Decoder	Generates tokens autoregressively	GPT, Claude
Encoder-Decoder	Seq2seq tasks	T5, BART

Causal masking in decoders: each position can only attend to previous positions (not future) - essential for autoregressive generation.

Complexity

Self-attention is O(n²d) in time and space. For long sequences, this becomes the bottleneck. Solutions: sliding window attention (Longformer), linear attention approximations, Flash Attention (efficient GPU implementation).

When to use

• Any NLP task - Transformer is the default
• Images with sufficient data (ViT, CLIP)
• Audio (Whisper, AudioLM)
• Code (Codex, StarCoder)
• Tabular with complex feature interactions (TabTransformer)

Key architectures

Model	Type	Use
BERT	Encoder	Embeddings, classification, NER
GPT / Claude / Llama	Decoder	Generation, chat, reasoning
T5	Enc-Dec	Translation, summarization, Q&A
ViT	Encoder (patches)	Image classification
Whisper	Enc-Dec	Speech-to-text

Mixture of Experts (MoE)

What it does

Routes each input token to a subset of "expert" networks instead of running all parameters, achieving high model capacity with lower per-token compute.

Core mechanism

Router: for each token, compute scores over N experts
Top-K selection: route to the K experts with highest scores (typically K=2)
Expert output: each selected expert processes the token independently
Combine: weighted sum of expert outputs

The key insight: a model with 100 experts and K=2 uses 2% of its parameters per token - but the total model has 100× the capacity.

Load balancing loss: an auxiliary loss encourages uniform expert utilization. Without it, the router collapses to using 1–2 experts exclusively.

When it matters

MoE is a training and architecture decision, not an inference decision. At inference time, MoE models require all experts in memory (large memory footprint) even though only K are used per token.

• GPT-4 is rumored to use MoE
• Mixtral 8×7B: 8 experts, uses 2 per token, effective capacity of ~45B params with ~12B active compute
• Google's Switch Transformer and GLaM demonstrated MoE scaling advantages

Tradeoffs

• Higher total parameter count → requires more memory
• Better parameter efficiency at training: same FLOPs, more capacity
• Expert specialization is not always interpretable or guaranteed
• Load balancing requires careful tuning

Architecture Comparison

Architecture	Sequential	Parallel	Long-range	Data needed	Best for
MLP	No	Yes	Limited	Low	Tabular, embeddings
CNN	Partial	Yes (spatial)	No	Medium	Images, signals
RNN/LSTM	Yes	No	Moderate	Low-medium	Short sequences
Transformer	No	Yes	Yes	High	Text, code, large-scale
MoE	No	Yes	Yes	Very high	Scale efficiency

Practical Guidance

1. Images: start with a pre-trained ResNet or EfficientNet. Fine-tune on your data. Only consider ViT if you have > 1M training images.
2. Text: always use a pre-trained Transformer (BERT family for classification, GPT family for generation).
3. Time series: try gradient boosting first (XGBoost with lag features). Add LSTM or Transformer only if gradient boosting plateaus.
4. Mixed modality: CLIP-style contrastive training or cross-attention between modalities.
5. Scale: if you need to scale parameters beyond 10B, consider MoE architectures for compute efficiency.

Common Mistakes

Choosing architecture before defining the task type. CNNs excel at spatially local patterns; RNNs and transformers at sequences; GNNs at relational structures. Jumping straight to "use a transformer" without asking whether spatial locality, sequential ordering, or relational structure matters most leads to unnecessarily complex models and harder debugging. Write down the task's structural assumptions before picking an architecture.

Applying sequence models to tabular data. Tabular features do not have a meaningful natural ordering, so treating a row as a token sequence provides no benefit and adds significant complexity. Tree-based models (XGBoost, LightGBM) still outperform neural architectures on most tabular benchmarks with less tuning. Reserve sequence models for data where position carries semantic meaning.

Using encoder-only models for generation tasks. BERT-style encoder-only transformers produce contextual embeddings but have no mechanism for autoregressive text generation - they see the full input at once and are not trained to predict the next token. Attempting to use them for generation produces incoherent output. For generation, use a decoder-only (GPT-style) or encoder-decoder (T5-style) architecture.

What to Practice Next

• For any new task you encounter this week, write down which architecture family is appropriate and your reasoning before looking at any benchmarks or existing implementations.
• Compare XGBoost and a simple MLP on the same tabular dataset; measure accuracy and training time and reflect on why one dominates.
• Identify one encoder-only and one decoder-only model in a codebase you have access to; confirm their architectural differences by inspecting the attention mask patterns used during training.