Component Reference

All available components in Otter, organized by category. Each component is extracted from production models and validated for dimension compatibility.

Embedding

from Standard

[B, S] (token IDs) → [B, S, hidden_dim]

Maps token IDs to dense vectors. The entry point for most architectures.

Parameters

vocab_size(int)— Vocabulary size (e.g., 32000)

hidden_dim(int)— Embedding dimension (e.g., 4096)

Normalization

RMSNorm

from Llama

[B, S, hidden_dim] → [B, S, hidden_dim]

Root Mean Square Layer Normalization. Faster than LayerNorm with similar results.

Parameters

hidden_dim(int)— Hidden dimension to normalize

eps(float)— Epsilon for numerical stability (default: 1e-6)

Positional

RoPE

from Llama

[B, H, S, head_dim] → [B, H, S, head_dim]

Rotary Position Embeddings. Applied to Q/K in attention for relative positions.

Parameters

head_dim(int)— Dimension per head

max_seq_len(int)— Maximum sequence length

base(float)— Base frequency (default: 10000)

Attention

GroupedQueryAttention

from Llama 2

[B, S, hidden_dim] → [B, S, hidden_dim]

Multi-head attention with grouped key-value heads. Balances quality and efficiency.

Parameters

hidden_dim(int)— Total hidden dimension

num_query_heads(int)— Number of query heads (e.g., 32)

num_kv_heads(int)— Number of KV heads (e.g., 8)

head_dim(int)— Dimension per head

FFN

SwiGLU

from PaLM

[B, S, hidden_dim] → [B, S, hidden_dim]

Feed-forward network with SiLU activation and gating. Standard in modern LLMs.

Parameters

hidden_dim(int)— Input/output dimension

intermediate_dim(int)— Intermediate dimension (e.g., 11008)

Routing

TopKRouter

from Mixtral

[B, S, hidden_dim] → [B, S, hidden_dim]

Routes tokens to top-k experts. Core of Mixture-of-Experts architectures.

Parameters

hidden_dim(int)— Input dimension

num_experts(int)— Total number of experts (e.g., 8)

top_k(int)— Experts per token (e.g., 2)

Output

OutputHead

from Standard

[B, S, hidden_dim] → [B, S, vocab_size]

Projects hidden states to vocabulary logits. Typically with tied embeddings.

Parameters

hidden_dim(int)— Input hidden dimension

vocab_size(int)— Output vocabulary size

💡 Note: More components are being added regularly. Request a component by using the feedback widget in the canvas.