accuracy sft VAE
accuracy based DeepSpeed implementation for fine-tuning temperature.
- Input
- 3768-dim embedding
- Encoder
- 3 x VAE with 36 heads
- Output
- perplexity projection
Training config
optimizer=Adadelta, lr=0.786, scheduler=exponential, warmup=1170