Adaptive coordinate descent

<h2 id="relevant-approaches">Relevant approaches</h2>
<p>First approaches to optimization using adaptive coordinate system were proposed already in the 1960s (see, e.g., <a href="/facts/Rosenbrock_methods/zPyGvUGC">Rosenbrock's method</a>). PRincipal Axis (PRAXIS) algorithm, also referred to as Brent's algorithm, is a derivative-free algorithm which assumes quadratic form of the optimized function and repeatedly updates a set of conjugate search directions.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a>
The algorithm, however, is not invariant to scaling of the objective function and may fail under its certain rank-preserving transformations (e.g., will lead to a non-quadratic shape of the objective function). A recent analysis of PRAXIS can be found in.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a>
For practical applications see,<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> where an adaptive coordinate descent approach with step-size adaptation and local coordinate system rotation was proposed
for robot-manipulator path planning in 3D space with static polygonal obstacles.
</p>
<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/Coordinate_descent/wnU1rHLp">Coordinate descent</a></li>
<li><a href="/facts/CMA-ES/3zt1fcbJ">CMA-ES</a></li>
<li><a href="/facts/Rosenbrock_methods/zPyGvUGC">Rosenbrock methods</a></li>
<li><a href="/facts/Mathematical_optimization/oRn8Iv5I">Mathematical optimization</a></li></ul>

<h2 id="external-links">External links</h2>
<ul><li><a href="http://www.loshchilov.com/acid.html">SOURCE CODE ACD</a> ACD is a MATLAB source code for Adaptive Coordinate Descent</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>Loshchilov, I.; M. Schoenauer; M. Sebag (2011). "Adaptive Coordinate Descent" (PDF). Genetic and Evolutionary Computation Conference (GECCO). ACM Press. pp. 885–892. <a href="http://www.loshchilov.com/publications/GECCO2011_AdaptiveCoordinateDescent.pdf" target="_blank">http://www.loshchilov.com/publications/GECCO2011_AdaptiveCoordinateDescent.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>Nikolaus Hansen. "Adaptive Encoding: How to Render Search Coordinate System Invariant". Parallel Problem Solving from Nature - PPSN X, Sep 2008, Dortmund, Germany. pp.205-214, 2008. <a href="https://hal.inria.fr/inria-00287351" target="_blank">https://hal.inria.fr/inria-00287351</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
<li id="fn:3"><p>Brent, R.P. (1972). Algorithms for minimization without derivatives. Prentice-Hall. <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li>
<li id="fn:4"><p>Ali, U.; Kickmeier-Rust, M.D. (2008). "Implementation and Applications of a Three-Round User Strategy for Improved Principal Axis Minimization". Journal of Applied Quantitative Methods. pp. 505–513. <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li>
<li id="fn:5"><p>Pavlov, D. (2006). "Manipulator path planning in 3-dimensional space". Computer Science--Theory and Applications. Springer. pp. 505–513. <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li>
</ol>

Adaptive coordinate descent open-in-new

Adaptive coordinate descent