Fix loading 200 Hz gaze data in Pupil Invisible recordings #2204
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When Pupil Player opens a Pupil Invisible recording for the first time, it performs an in-place upgrade procedure due to compatibility reasons. In this process, it loads PI gaze data and generates Pupil Player compatible gaze data based on it.
Due to a flaw in the previous gaze conversion procedure, Pupil Cloud post-processed 200 Hz gaze data could be loaded multiple times or not at all.
This PR moves the prior implementation from
video_capture.utils
topupil_recording.update.invisible
and restructures the code flow such that 200 Hz gaze data is processed separately from the case where gaze data is loaded from the real-time recorded files.Below a detailed description of what should be happening:
Pupil Invisible Companion records gaze data into three different sets of files. Their
names can be matched by the following regex patterns:
^gaze ps[0-9]+.raw$
^gaze ps[0-9]+.time$
^worn ps[0-9]+.raw$
The worn data is a stream of values of either 0 or 255, indicating that the glasses
were (not) worn. Pupil Player maps these to gaze confidence values of 0.0 and 1.0
respectively.
Since all
*.time
files are converted to Pupil Player before this function is beingcalled, we match the
^gaze ps[0-9]+_timestamps.npy$
pattern on the recording filesinstead. When looking for the location and worn data, the function just replaces the
necessary parts of the timestamp file names instead of performing separate regex
matches.
If the recording was successfully post-processed and downloaded from Pupil Cloud, it
will contain 200Hz-densified gaze data. This data replaces the real-time recorded
data by Pupil Invisible Companion and is stored in three files:
gaze_200hz.raw
gaze_200hz.time
(orgaze_200hz_timestamps.npy
if upgraded)worn_200hz.raw
The worn data is a special case as it was introduced at different points in time to
Pupil Invisible Companion and Pupil Cloud. In other words, it is possible that there
is no worn data, only real-time recorded worn data, or 200 Hz worn data. The latter
is preferred. If 200 Hz gaze data is only available with real-time recorded worn
data, the latter is interpolated to 200 Hz using a k-nearest-neighbour (k=1)
approach. If no worn data is available, or the numbers of worn samples and gaze
timestamps are not consistent, Pupil Player assumes a confidence value of 1.0 for
every gaze point.