The first chapter of machine learning

conducted the first chapter of machine learning today. I think you can first recognize these concepts

(1)Models, strategies, algorithms

(2) The concept of return

(3) Supervision, unsupervised, semi-supervised issues

(2) Over-fitting, under-fitting

The following are some concepts of expansion:


Hessian Matrix, also translated as Hessen matrix, Hessian matrix, Hesse matrix, etc., is a square matrix composed of second-order partial derivatives of a multivariate function, describing the local curvature of the function. The black matrix is ​​often used in the Newton method to solve the optimization problem. The black matrix can be used to determine the extremum problem of the multivariate function. In the optimization design of engineering practical problems, the listed objective functions are often very complicated. In order to simplify the problem, the objective function is often expanded into a Taylor polynomial at a certain point to approximate the original function. At this time, the function expands at a certain point. The black matrix is ​​involved in the matrix form.


In vector analysis, the Jacobian matrix is ​​a matrix in which the first-order partial derivatives are arranged in a certain way, and the determinant is called the Jacobian determinant. Also, in algebraic geometry, The Jacobian of the algebraic curve represents the Jacobian cluster: an algebraic group that accompanies the curve, the curve can be embedded in it. They are all mathematically written by Carl Jacob, October 4, 1804, February 18, 1851. Japanese) naming; English Jacobian "Jacobian" can be pronounced as [ja ˈko bi ən] or [ʤə ˈko bi ən].

The importance of the Jacobian matrix is ​​that it embodies a differentiable equation and gives a point The optimal linear approximation. Therefore, the Jacobian matrix is ​​similar to the derivative of the multivariate function.

Newton method optimization problem:

In the optimization problem, linear optimization can at least use the simplex method (or The fixed point algorithm is solved, but for the nonlinear optimization problem, the Newton method provides a solution. It is assumed that the task is to optimize an objective function ff, and the maximum and minimum problem of the function ff can be converted into a solution function ff. The problem of the derivative f' = 0f' = 0 The problem is considered as an equation solving problem (f'=0f'=0). The remaining problems are very similar to the Newton method mentioned in the first part.

This time to solve f'=0f'=0 The root, expands the Taylor of f(x)f(x), and expands to the second-order form:

gradient descent is just a specific operation of the numerical solution, and the optimization problem below the least squares criterion can use a stochastic gradient. Falling solution