kyegomez/Sophia

This is stars and forks stats for /kyegomez/Sophia repository. As of 03 May, 2024 this repository has 317 stars and 22 forks.

Sophia Optimizer PAPER LINK: Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training Author Implementation Cut Model Training Cost by 50%? with this all-new simple plug in and play Optimizer: Sophia Usage Download with pip pip install Sophia-Optimizer import torch from torch import nn from Sophia import SophiaG class MyModel(nn.Module): def __init__(self): super(MyModel, self).__init__() self.fc1 = nn.Linear(784, 128) self.fc2 = nn.Linear(128,...