Skip to content

Commit

Permalink
whisper : make beam candidate sort more stable (ggerganov#1943)
Browse files Browse the repository at this point in the history
All else being otherwise equal, this encourages the beam candidate
selection to re-use the same decoder, which slightly
reduces the cache size.

I wouldn't expect it to make much of a performance difference,
but it helps when debug printing the cache and beam.

Added as part of understanding ggerganov#1941.
  • Loading branch information
josharian authored and jiahansu committed Apr 17, 2024
1 parent 7458943 commit 2cc52ba
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion whisper.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5357,7 +5357,10 @@ int whisper_full_with_state(
beam_candidates.begin(),
beam_candidates.end(),
[](const beam_candidate & a, const beam_candidate & b) {
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
if (a.sequence.sum_logprobs_all != b.sequence.sum_logprobs_all) {
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
}
return a.decoder_idx < b.decoder_idx;
});

uint32_t cur_c = 0;
Expand Down

0 comments on commit 2cc52ba

Please sign in to comment.