Scaling of objective function in optimization problems.

When you study a optimization problem theoretically, scaling does not matter. However, sometimes one adds a scaling factor, such that the derivative is a little bit simpler, e.g., instead of $x^2$ one likes to minimize $\frac12 x^2$.

However, scaling is of importance for the numerically solution. If your optimization method is not scaling invariant, then you get a different sequence of iterates (not only due to rounding errors). Newton's method, e.g., is affine invariant, i.e., if you do an affine transformation of your optimization problem, you get (up to the transformation) the same sequence of iterates.

Tags:

Optimization

Related

Let $A$ be any uncountable set, and let $B$ be a countable subset of $A$. Prove that the cardinality of $A = A - B $ Finding the value of $\sum\limits_{k=0}^{\infty}\frac{2^{k}}{2^{2^{k}}+1}$ Another Series $\sum\limits_{k=2}^\infty \frac{\log(k)}{k}\sin(2k \mu \pi)$ A linear transformation $T:V\to V$ is one-to-one if and only if it is onto Cardinality of a set A is strictly less than the cardinality of the power set of A Factoring the ideal $(8)$ into a product of prime ideals in $\mathbb{Q}(\sqrt{-7})$ Deriving the sub-differential of the nuclear norm Why do elementary row operations preserve linear dependence between matrix columns? What is the negation of "$A\subseteq B$"? $f \in K[X]$ of degree $n$ with Galois group $S_n$, why are there no non-trivial intermediate fields of $K \subset K(a)$ with $a$ a root of $f$? How many nonisomorphic directed simple graphs are there with $n$ vertices, when $n$ is $2$,$3$, or $4$? Hartshorne Problem 1.2.14 on Segre Embedding

Recent Posts