Improved Hyperparameter Tuning API

Now, the ga_tune method performs a fit operation on the pipeline passed to it, fitting using the best hyperparameter configuration the tuner was able to find.

SKPlumber Fit Result Data

The SKPlumber.fit method now returns a SearchResult named tuple which contains data about how the AutoML search went. The tuple members are:

• time: The time in seconds it took the fit method to complete
• n_sample_iters: The total number of pipelines tried in the sampling stage
• n_tune_iters: The total number of pipelines tried in the hyperparameter tuning stage
• best_score: The best score the evaluator was able to find during the AutoML search

Time-Constrained AutoML

This release simplifies the API of SKPlumber, making it do more under the hood. Now the only required parameters are problem type (e.g. "classification"), and a budget argument that specifies the number of seconds skplumber is allowed to search for. It and also changes the SKPlumber class to accept parameters to its constructor, instead of the crank method. It renames the crank method to fit, which only accepts the data to fit to/search for solutions for.

Hyperparameter Optimization & Custom Evaluation

Two main features this release:

Hyperparameter Optimization

Hyperparameters can be optimized on the best found pipeline via the skplumber.SKPlumber.crank(..., tune=True) API or the on any single pipeline using the skplumber.tuners.ga.ga_tune method. This is accomplished via the flexga package and hyperparameter annotations which have been added to all machine learning primitives.

Custom Evaluation

Previously, skplumber.SKPlumber.crank could only do k-fold cross validation. Now, by passing in a custom evaluator e.g. skplumber.SKPlumber.crank(..., evaluator=my_evaluator), any other pipeline evaluation method can be used. skplumber provides evaluators for k-fold cross validation, simple train/test splitting, and down-sampled train/test splitting.