Skip to content

Commit

Permalink
Separate solvable_orders_after query (#2954)
Browse files Browse the repository at this point in the history
# Description
A suggestion to unify SQL queries mentioned in the
#2923 didn't work out.
Postgres converts common table expressions into subqueries, which
affects the performance. Also, with this approach, indexes cease to be
utilized. As a result, it executes in 1.2-1.5s instead of ~200ms.

Tried so far(nothing helped):
- JOIN with CTE instead of using `WHERE IN (rowset)`
- Use `WITH cte AS NOT MATERIALIZED`
- Replace CTE with a subquery.
- Convert CTE result into ARRAY(has the best performance, but still 2-3
times slower than separated queries).

# Changes
Revert to the separated queries that execute in ~200ms total.

## How to test
Restored DB tests.
  • Loading branch information
squadgazzz authored Sep 6, 2024
1 parent ae3ef29 commit 2678178
Show file tree
Hide file tree
Showing 3 changed files with 304 additions and 277 deletions.
30 changes: 22 additions & 8 deletions crates/autopilot/src/infra/persistence/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,16 @@ impl Persistence {
.execute(tx.deref_mut())
.await?;

// Find order uids for orders that were updated after the given block.
let updated_order_uids = {
let _timer = Metrics::get()
.database_queries
.with_label_values(&["updated_order_uids"])
.start_timer();

database::orders::updated_order_uids_after(&mut tx, after_block).await?
};

// Fetch the orders that were updated after the given block and were created or
// cancelled after the given timestamp.
let next_orders: HashMap<domain::OrderUid, model::order::Order> = {
Expand All @@ -427,14 +437,18 @@ impl Persistence {
.with_label_values(&["open_orders_after"])
.start_timer();

database::orders::open_orders_after(&mut tx, after_block, after_timestamp)
.map(|result| match result {
Ok(order) => full_order_into_model_order(order)
.map(|order| (domain::OrderUid(order.metadata.uid.0), order)),
Err(err) => Err(anyhow::Error::from(err)),
})
.try_collect()
.await?
database::orders::open_orders_by_time_or_uids(
&mut tx,
&updated_order_uids,
after_timestamp,
)
.map(|result| match result {
Ok(order) => full_order_into_model_order(order)
.map(|order| (domain::OrderUid(order.metadata.uid.0), order)),
Err(err) => Err(anyhow::Error::from(err)),
})
.try_collect()
.await?
};

// Fetch quotes for new orders and also update them for the cached ones since
Expand Down
2 changes: 1 addition & 1 deletion crates/database/src/byte_array.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ use {

/// Wrapper type for fixed size byte arrays compatible with sqlx's Postgres
/// implementation.
#[derive(Clone, Copy, Eq, PartialEq, Hash)]
#[derive(Clone, Copy, Eq, PartialEq, Hash, sqlx::FromRow)]
pub struct ByteArray<const N: usize>(pub [u8; N]);

impl<const N: usize> Debug for ByteArray<N> {
Expand Down
Loading

0 comments on commit 2678178

Please sign in to comment.