facebookresearch/dadaptation

This is stars and forks stats for /facebookresearch/dadaptation repository. As of 05 May, 2024 this repository has 428 stars and 14 forks.

D-Adaptation Learning rate free learning for SGD, AdaGrad and Adam! by Aaron Defazio and Konstantin Mishchenko (Arxiv) pip install dadaptation NEW V3.0 release uses an improved algorithm that may give different results from past versions. The old version is still availiable under experimental/d_adapt_adam_preprint. NEW: Prodigy We have recently released the Prodigy method, which grows the adapted learning rate faster than D-Adaptation in theory and practice. Try it out if D-Adaptation is under-estimating...