I reworked my experiment as a classification problem rather than a regression problem. Classification is predicting what group an instance belongs to, regression predicting a number, such as todays return. For a class I went with whether or not a return of at least 1% was achieved. So, binary classification, yes or no.
I was getting about 70% accuracy, because every time the result was no, the prediction was no, and this was most of the time. However nearly all the yes results were also classified as no.
This is where precision and recall come in. Precision – how many results that are classified positive actually are positive. And recall – how many examples that actually are positive are classified as such. So precision has to do with false positives, and recall with false negatives. Some results on my test data:
| Predicted No | Predicted Yes | |
| True No | 334 | 128 |
| True Yes | 146 | 73 |
So, the worst possible result. Most of the true yes examples were predicted to be No, and most of the examples predicted to be Yes were not. I guess there’s plenty of room for improvement.