Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Splitting Test Directories to Increase Test Concurrency and Optimize Cache #44940

Open
hawkingrei opened this issue Jun 26, 2023 · 0 comments · Fixed by #45001, #45020, #45833, #45834 or #46780
Open

Splitting Test Directories to Increase Test Concurrency and Optimize Cache #44940

hawkingrei opened this issue Jun 26, 2023 · 0 comments · Fixed by #45001, #45020, #45833, #45834 or #46780
Labels
good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. type/enhancement The issue or PR belongs to an enhancement.

Comments

@hawkingrei
Copy link
Member

Enhancement

we are using Bazel's shard_count to increase test concurrency and speed up our testing process. However, in some cases, we have found that certain directories have too many tests, which can result in slower test execution times, decreased efficiency, and less optimized cache usage.

To address this issue, we need to split the test directories into smaller, more manageable chunks. This will help increase test concurrency, optimize cache usage, and ultimately improve our testing process.
@hawkingrei hawkingrei added the type/enhancement The issue or PR belongs to an enhancement. label Jun 26, 2023
@hawkingrei hawkingrei mentioned this issue Jun 26, 2023
Merged
12 tasks
@hawkingrei hawkingrei added the good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. label Jun 26, 2023
ti-chi-bot bot pushed a commit that referenced this issue Jun 26, 2023
@hawkingrei
executor: split test and create hash_join_test to improve ci (#44939)
b367d5f 
ref #44940
@hawkingrei hawkingrei mentioned this issue Jun 26, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jun 27, 2023
@hawkingrei
server: split code and improve ci performance (#44958)
d5d5817 
ref #44940
@hawkingrei hawkingrei mentioned this issue Jun 27, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jun 27, 2023
@hawkingrei
executor: split test for aggregate (#44987)
8ef5b4a 
ref #44940
@hawkingrei hawkingrei mentioned this issue Jun 27, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jun 28, 2023
@hawkingrei
server: split the code and create parse (#45001)
ceccc1f 
close #44940
@hawkingrei hawkingrei reopened this Jun 28, 2023
@hawkingrei hawkingrei mentioned this issue Jun 28, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jun 28, 2023
@hawkingrei
expression: split multi valued index test (#45008)
7e808b3 
ref #44940
@hawkingrei hawkingrei mentioned this issue Jun 28, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jun 28, 2023
@hawkingrei
executor: split executor_test (#45020)
9fc80a9 
close #44940
@hawkingrei hawkingrei reopened this Jun 30, 2023
@hawkingrei hawkingrei mentioned this issue Jun 30, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jul 3, 2023
@hawkingrei
executor: refactor executor and baseExecutor (#45065)
30cafb3 
ref #44940
@hawkingrei hawkingrei mentioned this issue Jul 6, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jul 6, 2023
@hawkingrei
sessiontest: split test for improve ci (#45198)
c6f5151 
ref #44940
This was referenced Jul 7, 2023
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jul 7, 2023
@hawkingrei
session: update sessiontest for improve ci (#45227)
c63ff77 
ref #44940
ti-chi-bot bot pushed a commit that referenced this issue Jul 7, 2023
@hawkingrei
server: refact the code structure (#45228)
cf0ae34 
ref #44940
This was referenced Jul 8, 2023
Merged
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jul 10, 2023
@hawkingrei
kazel: enable shard_count for util/chunk (#45250)
40dc190 
ref #44940
ti-chi-bot bot pushed a commit that referenced this issue Aug 11, 2023
@hawkingrei
ddl: split partition test (#45994)
837afe3 
ref #44940
@hawkingrei hawkingrei mentioned this issue Aug 21, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 28, 2023
@hawkingrei
tables: split partiton test (#46252)
e7e9fb6 
ref #44940
@hawkingrei hawkingrei mentioned this issue Sep 4, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 4, 2023
@hawkingrei
casetest: split DAG test (#46625)
08024e7 
ref #44940
@hawkingrei hawkingrei mentioned this issue Sep 8, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 8, 2023
@hawkingrei
planner: split index test (#46780)
a41f5ea 
close #44940
@hawkingrei hawkingrei reopened this Sep 12, 2023
@hawkingrei hawkingrei mentioned this issue Sep 12, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 12, 2023
@hawkingrei
executor: merge some tests into one (#46882)
d583ed6 
ref #44940
@hawkingrei hawkingrei mentioned this issue Sep 12, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 12, 2023
@hawkingrei
planner: merge tests into one (#46919)
f0e4086 
ref #44940
@hawkingrei hawkingrei mentioned this issue Oct 9, 2023
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 9, 2023
@hawkingrei
statistics: move tests into GlobalStats test (#47473)
c474445 
ref #44940
@hawkingrei hawkingrei mentioned this issue Oct 13, 2023
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 14, 2023
@hawkingrei
statistics: refactor test package (#47608)
1b4383a 
ref #44940
@hawkingrei hawkingrei mentioned this issue Oct 30, 2023
Closed
13 tasks
This was referenced Nov 22, 2023
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 22, 2023
@hawkingrei
session: move some test into new package (#48784)
66f3a25 
ref #44940
ti-chi-bot bot pushed a commit that referenced this issue Nov 23, 2023
@hawkingrei
session: move session interface into new package (#48792)
b8515ae 
ref #44940
@hawkingrei hawkingrei mentioned this issue Dec 1, 2023
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Dec 1, 2023
@hawkingrei
server: fix spell (#49095)
330a5d5 
ref #44940
@ti-chi-bot ti-chi-bot mentioned this issue Dec 5, 2023
Open
13 tasks
@hawkingrei hawkingrei mentioned this issue Feb 1, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Feb 1, 2024
@hawkingrei
server: split tests (#50860)
b816816 
ref #44940
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
1 participant
@hawkingrei