Otherwise compute the normalized search direction to p k gx kkgx kk.
Example of descent direction. R2R given by fxy 1 2 p 16 24x2 y. Not recommended for large training samples. Stochastic Gradient Descent is a stochastic as in probabilistic spin on Gradient Descent.
A single training sample is used. We rst compute the steepest descent direction from rfxy 8x 4y4y 4x to obtain rfx 0 rf23 44. A classic example is of course ordinary gradient ascent whose search direction is simply the gradient.
1Select starting point x 0 and convergence parameters g aand r. Lets take the function. Steepest descent methods 4 are usually used for solving for example.
Check for the Descent Condition. Find the curves of steepest descent for the ellipsoid 4x2 y2 4z2 16 for z 0. The function value at the starting point is.
In particular r fx is a descent direction if nonzero. This ellipsoid is the graph of a function f. Diagonally Scaled Steepest Descent.
We then minimize the function t f23 t44. Then any direction dthat satis es rTfxddescent direction. Common choices of descent direction.