Automatically calibrate cursor position to match head tilt (Bonus: generalizes head tilt mode) #45
Labels
affects:desktop-app
Affects the Electron app
affects:web-library
Affects usage of the tracky-mouse.js library
enhancement
New feature or request
Currently, the user's face is detected in order to place tracking points, and optical flow is then used to track these points on your face, and only the optical flow influences the final cursor movement.
In this existing scheme:
What if we auto-calibrated based on the head tracker's face orientation, perhaps making adjustments only during otherwise-detected movement?
Like a perpetual motion machine's secret magnetic "kick", or a magician's slight of hand, it would subtly adjust the mouse position so that it ends up at the edges of the screen when tilted a certain amount, and centered when facing forward.
Drawbacks:
Formula:
Assuming the tilt can be normalized, or assuming the camera and screen and face are directly in line with each other,
a formula for the other part might be fairly simple, something like
adjusted_x = x + (abs(delta_x) * auto_calibrate_strength * (x_implied_by_tilt - x)
where
x_implied_by_tilt
is the x position on the screen that would be mapped purely from the head tiltwhere
delta_x = x - x_previous
where
x
is the latest x position from optical flow trackingIt's probably a little more complicated, like maybe the
delta_x
factor should be raised to some power, or clamped, etc.On the plus side, it could be made so that with
auto_calibrate_strength = 1
, it purely uses the head tilt, so a separate head tilt mode wouldn't be needed. (Again, I'm not sure about thedelta_x
part in regard to this.)The text was updated successfully, but these errors were encountered: