If r != 0 then could be working for negative reward as well?

[charalak]

In the following line
https://github.com/david-cortes/contextualbandits/blob/7b87efe31ffefdbc95a125d557920ab5b13105c5/contextualbandits/utils.py#L912C13-L912C34

if we replace arms_w_rew = (r > 0.) with arms_w_rew = (r != 0.) then "BootstrappedUCB" and "BootstrappedTS" could work and incorporate negative rewards (I guess).

[david-cortes]

Thanks for the suggestion.

There's unfortunately many other places scattered throughout the code where the assumption about binary rewards is used, such as this one:

contextualbandits/contextualbandits/utils.py

Line 480 in 7b87efe

self.bs_algos[sample].partial_fit(xsample, ysample, classes=[0,1])

Plus other methodological aspects which specifically exploit the assumption of rewards being binary, like the usage of a beta distribution whose priors are updated iteratively, or the calculation of uncertainty in active learning criteria.

Hence, you might find that changing that line might be sufficient for allowing negative rewards for some particular combinations of input parameters, but it will not guarantee that everything else will work along with them.