|
Seminarium 14.03.2023
dodano: 12/03/2023
przez: Dariusz Zawisza |
- Referent : Jarosław Duda
- Tytuł referatu : SGD-OGR 2nd order optimizers with Online Gradient Regression for Hessian estimation
- Abstrakt : Neural network training is currently dominated by 1st order SGD (stochastic gradient descent) methods with heuristic modifications like ADAM updating 2 average vectors. I will talk about OGR (online linear regression of gradients) approaches updating e.g. 4 average vectors instead, this way providing real 2nd order method: ideally optimizing parabolas/paraboloids in a single steps, at least in low dimensions providing much faster convergence.
- Slajdy: Link
- Termin: Wtorek 14.03.2023, 12:15, sala 1093
