Machine learning week 6(Andrew Ng)

Advice for applying machine learning

1、Advice

1.1、What to try next

1.2、Evaluating model

• To test your model’s performance on new data before deploying it? we can split the training set into “Training” and “Test” sets.
  - For the linear regression :
  

1.3、Model selection and training/cross validation/test sets

The original thought is to choose the least $J_{test}(w,b)$.
But the test set J test is now overly optimistic that is lower than the actual estimate of the generalization error.
We divide the data set into three parts and pick the least $J_{cv}(w,b)$.

2、Bias and variance

2.1、Diagnosing bias and variance

Underfit produces high bias while overfit produces high variance.


The notion of high bias and high variance, it doesn’t really happen for linear models applied to one deep. But it is possible sometimes they’re both at the same time.

2.2、Regularization and bias/variance

2.3、Establishing a baseline level of performance

Determine the baseline first.

2.4、Learning curves

2.5、Deciding what to try next revisited

2.6、Bias/variance and neural networks

if you have a small neural network like this, and you were to switch to a much larger neural network like this, you would think that the risk of overfitting goes up significantly. But it turns out that if you were to regularize this larger neural network appropriately, then this larger neural network usually will do at least as well or better than the smaller one. So long as the regularization has been chosen appropriately.

3、Machine learning development process

3.1、Iterative loop of ML development

3.2、Error analysis

Hopefully looking through maybe around 100 examples will give you enough statistics about whether the most common types of errors and therefore where maybe most fruitful to focus your attention. After this analysis, if you find that a lot of errors are pharmaceutical spam emails then this might give you some ideas or inspiration for things to do next.

3.3、Adding data

There are many ways to enhance data based on the data we had before.

3.4、Transfer learning: using data from a different task

3.5、Full cycles of a machine learning project

4、Skewed datasets

4.1、Error metrics（误差度量） for skewed datasets

The ratio of positive to negative examples is very skewed, very far from 50-50, then it turns out that the usual error metrics like accuracy don’t work that well.

4.2、Trading off precision and recall