Mask mode behaves strangely when changing layer selection

[iwehrman]

1. Select a graphic layer, enter mask mode.
2. Deselect. You're still in mask mode, which is kind of weird but I guess OK?
3. Reselect the same layer. Now you're out of mask mode. Why?

I think if you change your layer selection in a way that is incompatible with mask mode, mask mode should be disabled and you should have to explicitly re-enable it.

[iwehrman]

Bonus bug:

1. Select a graphic layer, enter mask mode.
2. Shift click a vector layer to add it to the selection.
3. Click the mask button to leave mask mode.

Result: still in mask mode.

[pineapplespatula]

changing selections should exit mask mode.