Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for conditional/hierarchical design spaces #30

Open
bbudescu opened this issue Nov 12, 2022 · 1 comment
Open

Support for conditional/hierarchical design spaces #30

bbudescu opened this issue Nov 12, 2022 · 1 comment

Comments

@bbudescu
Copy link

bbudescu commented Nov 12, 2022

Can one specify a conditional/hierarchical search space to HEBO? I.e., something similar to SMAC3?

E.g., only sample the number of convoutional filters in the second layer of a neural net only if we decide (by another parameter )to have a network with at least two layers.

I'm thinking that this can be somewhat circumvented by returning a high cost value for infeasible combinations, but I imagine this might be a suboptimal approach as it might affect optimization performance since, e.g., some optima might lie on the edge of feasible regions, and the cost estimator, if it has some sort of smoothness prior (which they usually do), has a chance of assigning unfaithfully high cost values near the infeasible configurations - at least initially (i.e., the a priori probability of a kink in the error surface is generally lower).

Some optimizers are able to address this by training a different model - a feasability predictor (which has the advantage of being able to work with unknown feasibility constraints).

So how should one deal with this in HEBO?

@artiko88
Copy link

Has anyone come up with any solution for hierarchical searches using HEBO? Would hate to switch to different optimizer because of not being able to do something similar to how SMAC3 performs hierarchical searches.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants