Homepage › Solution manuals › Gilbert Strang › Linear Algebra and Learning from Data › Exercise 6.4.6
f(x) = 1 2∥Ax−b∥2 = 1 2(Ax−b)T(Ax−b) = 1 2(xTAT−bT)(Ax−b) = 1 2(xTATAx−xTATb−bTAx+bTb)
Then we have ∇ f(xk) = 1 2(2ATAx k − 2ATb) = ATAx k − ATb, so xk+1 = xk − sk(ATAxk − ATb)
