fix(spooler): Fix datetime comparison #4025
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
iambriccardo
changed the title
iambriccardo
commented
Sep 11, 2024
| // Unfortunately we have to do this because SQLite `DELETE` with `RETURNING` doesn't | ||
| // return deleted rows in a specific order. | ||
| extracted_envelopes.sort_by_key(|a| a.received_at()); | ||
| extracted_envelopes.sort_by_key(|a| { | ||
| Reverse(UnixTimestamp::from_datetime(a.received_at()).unwrap_or(UnixTimestamp::now())) |
There was a problem hiding this comment.
I forgot to return the dates in reverse sorted order to honor the API. This is now fixed, the envelopes are returning in descending timestamp order.
iambriccardo
commented
Sep 11, 2024
| assert!(envelope_store | ||
| .insert_many(envelopes.iter().map(|e| e.as_ref().try_into().unwrap())) | ||
| .await | ||
| .is_ok()); | ||
|
|
||
| // We check that if we load more than the limit, we still get back at most 10. | ||
| // We check that if we load 5, we get the newest 5. | ||
| let extracted_envelopes = envelope_store |
There was a problem hiding this comment.
Fixed and improved this test.
jjbayer
reviewed
Sep 11, 2024
| // Since the store returns the envelopes sorted in descending order, we want to put them | ||
| // in reverse into the vector in the buffer, because we want to pop the last element always, | ||
| // which has to be the newest (aka with the biggest timestamp). | ||
| envelopes.reverse(); |
There was a problem hiding this comment.
Why do we reverse twice, once here and once in delete_many?
There was a problem hiding this comment.
It's explained in the comment above, I wanted to return descending values from the store but when concatenating in the buffer, we pop from the bottom (which we assume has the newest timestamp).
jjbayer
approved these changes
Sep 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR uses
UnixTimestampinstead ofDateTimeto compare timestamps when sorting data from disk.This fixes an issue that raised the following panic (https://doc.rust-lang.org/src/core/slice/sort/shared/smallsort.rs.html#862) which caused the envelope buffer service to crash and stop handling envelopes.
ref: rust-lang/rust#129561