Training Guide

Your exported model is ready to train. This guide covers running the included training script on various platforms.

Prerequisites

• Python 3.9+ with pip
• PyTorch 2.0+ with CUDA support (for GPU training)
• At least 16GB GPU memory for small models, 40GB+ for larger ones
• Training data in text format (one document per line recommended)

Quick Start (Local GPU)

Terminal

# Unzip your export
unzip Otter_export.zip
cd Otter_model

# Install dependencies
pip install -r requirements.txt

# Prepare your data (example: tiny shakespeare)
curl -O https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
mv input.txt data/train.txt

# Start training
python train.py --data_dir data --epochs 10

Google Colab

Free GPU access with some limitations. Good for experimentation.

Colab Cell

# Upload your ZIP to Colab, then:
!unzip Otter_export.zip
%cd Otter_model
!pip install -r requirements.txt

# Download sample data
!curl -O https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
!mkdir -p data && mv input.txt data/train.txt

# Train (use Colab's GPU)
!python train.py --data_dir data --epochs 5 --batch_size 4

⚠️ Colab limitations: Free tier has session timeouts (90 min), limited GPU memory (T4 = 16GB), and may disconnect during training. Use Colab Pro for longer runs.

Lambda Labs / Cloud GPU

For serious training runs, rent an A100 or H100 instance. Lambda Labs, RunPod, and Vast.ai are good options.

SSH Session

# SSH into your instance
ssh ubuntu@<instance-ip>

# Clone your data repo or upload via scp
scp -r ./data ubuntu@<instance-ip>:~/data

# Upload and extract the model
scp Otter_export.zip ubuntu@<instance-ip>:~/
unzip Otter_export.zip && cd Otter_model

# Install and train
pip install -r requirements.txt
python train.py \
  --data_dir ~/data \
  --epochs 100 \
  --batch_size 32 \
  --learning_rate 3e-4 \
  --gradient_accumulation_steps 4

Training Arguments

Argument	Default	Description
--data_dir	data/	Directory containing train.txt
--epochs	10	Number of training epochs
--batch_size	8	Batch size per GPU
--learning_rate	1e-4	Peak learning rate
--warmup_steps	100	LR warmup steps
--gradient_accumulation_steps	1	Gradient accumulation
--max_seq_len	512	Maximum sequence length
--output_dir	checkpoints/	Save checkpoints here

Monitoring Training

The training script logs metrics to the console. For better visualization, enable Weights & Biases:

pip install wandb
wandb login

python train.py --data_dir data --use_wandb --wandb_project my-llm

After Training

Your trained model checkpoint can be:

• Loaded with torch.load() for inference
• Converted to HuggingFace format with the included script
• Quantized with llama.cpp for local deployment
• Fine-tuned further on domain-specific data