Interactive Cosine Annealing with Warmup Visualizer

by Kurious Fox
July 8, 2025July 8, 2025

Interactive Cosine Annealing with Warmup Visualizer

See also Example of Knowledge Distillation (KD) using PyTorch on the MNIST dataset

Cosine Annealing with Linear Warmup

Explore the two-phase learning rate schedule by adjusting the parameters.

Controls

Warmup Ratio 10%

Peak Learning Rate (η_max) 0.01

Min Learning Rate (η_min) 0.0001

Total Epochs (T_max) 200

Schedule Phases

1. Warmup: Linear increase to Peak LR.

2. Decay: Cosine anneal from Peak to Min LR.

Related

Discover more from Knowledge sparks

Subscribe to get the latest posts sent to your email.

Leave a ReplyCancel reply

error: Content is protected !!