Training Loss Evaluation

[rufex2001]

This PR adds an evaluation job that is simply the training loss. Currently, support is only for computing the loss on the training split. This is part of the following PR:

#99

[rgemulla]

Also, the configuraiton should be updated (saying that there is now this other evaluation type).

[rufex2001]

@rgemulla All comments addressed in latest commit

[rufex2001]

@rgemulla Please wait until I add sum_loss to tracing for merging

[rufex2001]

@rgemulla I've added sum loss tracing to trainers and the training loss eval job. Two things:

1. Should we do this as simple as it is how? Or should we add this also at the batch trace level? This would mean trainers add sum_loss to their results object.
2. The solution to preparing the train job when creating it as an eval job (eval.py line 172) calls a private method from outside. Not sure this is what you meant, but this is partly why I didn't do it like this directly, but I forgot to bring it up (until now).

[rufex2001]

@rgemulla Ok, num_examples was there but called size. Renamed it to num_examples because I think that makes more sense than the notion of "epoch size". We do call it size at the batch level, which of course makes sense. If you want, I can revert this.

Also, not sure what keys to include in the trace produced by this eval job. I manually add avg_loss and num_examples from the internal training job for now. When adding everything, some things are overwritten, e.g. the event key, type key, epoch number key, etc. Also, this causes an error during the trace event, e.g. because there is already a key for job_id that comes from the already traced entry from the training job. We need to either decide which keys from the train job to include (avg cost too? avg penalty?), or which keys to exclude to prevent errors and overwritten keys. Ideally, it would be nice if all relevant keys from the train job were available to design custom metrics.

[rufex2001]

@rgemulla Tracing done as suggested.

[rgemulla]

Other than this, looks good to me. Please add an entry into the CHANGELOG before merging.

[rgemulla]

Since we use size everywhere else (e.g., for the batch size), I suggest to continue using "size" here.

[rufex2001]

@rgemulla Fixed!

[rgemulla]

Alright, please add the CHANGELOG and merge