Loading devreal.ai…

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training. — devreal.ai