What are the optimization methods for deep learning TensorFlow's basic theory

1, deep learning framework TensorFlow? GradientDescentOptimizer, AdagradOptimizer, AdagradDAOptimizer, MomentumOptimizer AdamOptimizer, FtrlOptimizer, RMSPropOptimizer 2. What are the common activation functions in the deep learning framework TensorFlow? Sigmoid, tanh, relu, relu6, elu, softplus 3. What are the four common cross entropies in TensorFlow? 1.tf.nn.sigmoid_cross_entropy_with_logits( _sentinel=None, labels=None, logits=None, name=None) 2.tf.nn.softmax_cross_entropy_with_logits( _sentinel=None, labels=None, logits=None, dim=-1, name=None) 3.tf.nn.sparse_softmax_cross_entropy_with_logits( _sentinel=None, labels=None, logits=None, name=None) 4.tf.nn.weighted_cross_entropy_with_logits( labels, logits, pos_weight, name=None)

4, what is it? Over-fitting, what measures are taken to avoid overfitting? 1. What is over-fitting When training the model, it is possible to encounter insufficient training data, that is, when the training data cannot estimate the distribution of the entire data, or when the model is overtrained, it often leads to over-fitting of the model. 2. How to solve the overfitting In order to prevent overfitting, we need to use some methods, data enhancement, adding Dropout layer, regularization, Reduce the number of features and use cross-validation to find the optimal model