Skip to content

Commit

Permalink
Add FrameProcessor functionality to release a frame immediately
Browse files Browse the repository at this point in the history
Currently `FrameProcessor.releaseOutputFrame()` method supports

Release at a specific system time
Drops the frame
This API is not that convenient to use when the caller wants to release a frame, now, regardless of the release time. A use case is to release (present) a frame when no frame is shown for a while, and it's thus better to just release the frame, now.

Currently if MCVR wants a frame to be rendered now, MCVR calls release frame with a set offset like 10us: `releaseOutputFrame(System.nanoTime() + 10_000)`. The 10us offset is to prevent the frame processor dropping the frame, due to thread hopping delays.

To make the API better usable, consider adding a mode for releasing the frame now, like (bold marks the new mode)

- Use C.TIME_UNSET to drop
- **Use -1 to release the frame immediately, or**
- Use an actual release time.

PiperOrigin-RevId: 479044215
  • Loading branch information
claincly authored and marcbaechinger committed Oct 20, 2022
1 parent 20c1ae1 commit ff8dd0b
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,29 @@ public void controlledFrameRelease_withOneFrame_usesGivenTimestamp() throws Exce
assertThat(outputReleaseTimesNs).containsExactly(releaseTimesNs);
}

@Test
public void controlledFrameRelease_withOneFrameRequestImmediateRelease_releasesFrame()
throws Exception {
long originalPresentationTimeUs = 1234;
long releaseTimesNs = FrameProcessor.RELEASE_OUTPUT_FRAME_IMMEDIATELY;
AtomicLong actualPresentationTimeUs = new AtomicLong();
setupGlEffectsFrameProcessorWithBlankFrameProducer(
/* inputPresentationTimesUs= */ new long[] {originalPresentationTimeUs},
/* onFrameAvailableListener= */ presentationTimeUs -> {
actualPresentationTimeUs.set(presentationTimeUs);
checkNotNull(glEffectsFrameProcessor).releaseOutputFrame(releaseTimesNs);
},
/* releaseFramesAutomatically= */ false);

checkNotNull(produceBlankFramesTask).run();
Thread.sleep(FRAME_PROCESSING_WAIT_MS);

assertThat(frameProcessingException.get()).isNull();
assertThat(actualPresentationTimeUs.get()).isEqualTo(originalPresentationTimeUs);
// The actual release time is determined by the FrameProcessor when releasing the frame.
assertThat(outputReleaseTimesNs).hasSize(1);
}

@Test
public void controlledFrameRelease_withLateFrame_dropsFrame() throws Exception {
long originalPresentationTimeUs = 1234;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,12 +170,18 @@ public void releaseOutputFrame(TextureInfo outputTexture) {
public void releaseOutputFrame(long releaseTimeNs) {
checkState(!releaseFramesAutomatically);

boolean dropLateFrame = true;
if (releaseTimeNs == FrameProcessor.RELEASE_OUTPUT_FRAME_IMMEDIATELY) {
dropLateFrame = false;
releaseTimeNs = System.nanoTime();
}

Pair<TextureInfo, Long> oldestAvailableFrame = availableFrames.remove();
renderFrameToSurfaces(
/* inputTexture= */ oldestAvailableFrame.first,
/* presentationTimeUs= */ oldestAvailableFrame.second,
releaseTimeNs,
/* dropLateFrame= */ true);
dropLateFrame);
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,12 @@ interface Listener {
void onFrameProcessingEnded();
}

/**
* Indicates the frame should be released immediately after {@link #releaseOutputFrame(long)} is
* invoked.
*/
long RELEASE_OUTPUT_FRAME_IMMEDIATELY = -1;

/** Returns the input {@link Surface}, where {@link FrameProcessor} consumes input frames from. */
Surface getInputSurface();

Expand Down Expand Up @@ -164,8 +170,9 @@ interface Listener {
* {@linkplain Listener#onOutputFrameAvailable(long) available}.
*
* @param releaseTimeNs The release time to use for the frame, in nanoseconds. Use {@link
* C#TIME_UNSET} to drop the frame. If {@code releaseTimeNs} is after {@link
* System#nanoTime()} at the time of the release, the frame is also dropped.
* C#TIME_UNSET} to drop the frame, or {@link #RELEASE_OUTPUT_FRAME_IMMEDIATELY} to release
* the frame immediately. If {@code releaseTimeNs} is after {@link System#nanoTime()} at the
* time of the release, the frame is also dropped.
*/
void releaseOutputFrame(long releaseTimeNs);

Expand Down

0 comments on commit ff8dd0b

Please sign in to comment.