A new zeroth-order optimization algorithm called Dust has been introduced that is competitive with backpropagation for pretraining transformer language models.
register or log in to get more services such as weather report, extended news articles and web search