Skip to content

Commit

Permalink
[#23859] YSQL: Remove redundant Bitmap Scan filters on partial indexes
Browse files Browse the repository at this point in the history
Summary:
One of the things that `create_bitmap_subplan` does is generates a list of conditions that are guaranteed by the tree of Bitmap Ands, Bitmap Ors, and Bitmap Index Scans. This is used to determine the clauses that remain to be checked by the Bitmap Heap Scan / YB Bitmap Table Scan.

4ea354b / D36484 introduced `yb_get_bitmap_index_quals` that just generates the list of conditions. This makes the filter information available for use during the costing phase, before the actual plan nodes need to be constructed.

`create_bitmap_subplan` accounted for the clauses of partial indexes. `yb_get_bitmap_index_quals` missed including this part, which caused the planner to believe the the conditions that are implied by the partial index conditions still need to be rechecked. For example,

```lang=sql
create table t (a int, b int);
create index idx_a_partial on t (a asc) where b < 10;
create index idx_b_partial on t (b asc) where a < 10;

 /*+ Set(yb_enable_bitmapscan true) BitmapScan(t) */ explain select * from t where b < 10 and a < 10;
```

Generates a bitmap subplan that looks as follows:
```
   ->  Bitmap Index Scan on idx_b_partial  (cost=0.00..0.92 rows=8 width=0)
         Index Cond: (b < 10)
```

We know that `b < 10` is satisfied (by index condition) and `a < 10` is satisfied (by partial index clause). However, `yb_get_bitmap_index_quals` did not report the partial index clause, so it told the Bitmap Table Scan that only `b < 10` was satisfied - requiring the Bitmap Table Scan to use a Storage Filter to check `a < 10`.
```
                QUERY PLAN
------------------------------------------
 YB Bitmap Table Scan on t
   Storage Filter: (a < 10)
   ->  Bitmap Index Scan on idx_b_partial
         Index Cond: (b < 10)
(4 rows)
```

With the change in this diff, `yb_get_bitmap_index_quals` reports both clauses, so the Bitmap Table Scan knows that both conditions are already met and it does not need to add a remote filter.
```
                QUERY PLAN
------------------------------------------
 YB Bitmap Table Scan on t
   ->  Bitmap Index Scan on idx_b_partial
         Index Cond: (b < 10)
(3 rows)
```

**How did this regression get in?** I noticed `yb_pg_select` was failing in D36484. I saw the problem was with Storage Filters but did not pay much attention to whether the storage filters were added or removed. The diff was able to remove some redundant storage filters from other tests, so I assumed it was doing the same here.

Jira: DB-12766

Test Plan:
```
ybd --java-test 'org.yb.pgsql.TestPgRegressPgSelect#testPgRegressPgSelect'
```

Reviewers: mtakahara

Reviewed By: mtakahara

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D37983
  • Loading branch information
timothy-e committed Sep 16, 2024
1 parent 063dbe5 commit a913524
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 6 deletions.
16 changes: 16 additions & 0 deletions src/postgres/src/backend/optimizer/path/costsize.c
Original file line number Diff line number Diff line change
Expand Up @@ -7455,6 +7455,7 @@ yb_get_bitmap_index_quals(PlannerInfo *root, Path *bitmapqual,
IndexPath *ipath = (IndexPath *) bitmapqual;
IndexScan *iscan;
List *indexqual;
ListCell *l;

/* Use the regular indexscan plan build machinery... */
iscan = castNode(IndexScan,
Expand All @@ -7469,6 +7470,21 @@ yb_get_bitmap_index_quals(PlannerInfo *root, Path *bitmapqual,
indexqual = lappend(indexqual,
make_ands_explicit(iscan->yb_idx_pushdown.quals));

/* We can add any index predicate conditions, too */
foreach(l, ipath->indexinfo->indpred)
{
Expr *pred = (Expr *) lfirst(l);

/*
* We know that the index predicate must have been implied by the
* query condition as a whole, but it may or may not be implied by
* the conditions that got pushed into the bitmapqual. Avoid
* generating redundant conditions.
*/
if (!predicate_implied_by(list_make1(pred), indexqual, false))
indexqual = lappend(indexqual, pred);
}

pfree(iscan);

return indexqual;
Expand Down
10 changes: 4 additions & 6 deletions src/postgres/src/test/regress/expected/yb_pg_select.out
Original file line number Diff line number Diff line change
Expand Up @@ -626,10 +626,9 @@ select unique2 from onek2 where unique2 = 11 and stringu1 < 'B';
QUERY PLAN
------------------------------------------
YB Bitmap Table Scan on onek2
Storage Filter: (stringu1 < 'B'::name)
-> Bitmap Index Scan on onek2_u2_prtl
Index Cond: (unique2 = 11)
(4 rows)
(3 rows)

select unique2 from onek2 where unique2 = 11 and stringu1 < 'B';
unique2
Expand Down Expand Up @@ -674,18 +673,17 @@ SELECT * FROM (
select unique1, unique2 from onek2
where (unique2 = 11 and stringu1 < 'B') or unique1 = 0
LIMIT ALL) ybview ORDER BY unique2;
QUERY PLAN
----------------------------------------------------------------------------------------
QUERY PLAN
------------------------------------------------------
Sort
Sort Key: onek2.unique2
-> YB Bitmap Table Scan on onek2
Storage Filter: (((unique2 = 11) AND (stringu1 < 'B'::name)) OR (unique1 = 0))
-> BitmapOr
-> Bitmap Index Scan on onek2_u2_prtl
Index Cond: (unique2 = 11)
-> Bitmap Index Scan on onek2_u1_prtl
Index Cond: (unique1 = 0)
(9 rows)
(8 rows)

SELECT * FROM (
select unique1, unique2 from onek2
Expand Down

0 comments on commit a913524

Please sign in to comment.