Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-8675][CH] Rewrite union of multiple aggregates into one #8676

Merged
merged 9 commits into from
Feb 11, 2025
Merged

[GLUTEN-8675][CH] Rewrite union of multiple aggregates into one #8676

merged 9 commits into from
Feb 11, 2025
+1,217 −0

Conversation

lgbo-ustc
Copy link
Contributor

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

Fixes: #8675

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

unit tests

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

#8675

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI on x86

@lgbo-ustc lgbo-ustc requested a review from zzcclp February 10, 2025 06:03
@lgbo-ustc
Copy link
Contributor Author

lgbo-ustc commented Feb 11, 2025 •
edited
Loading

An example to show the improvement.

disable this rule
image

enable this rule
image

zzcclp
zzcclp approved these changes Feb 11, 2025
View reviewed changes
Copy link
Contributor

@zzcclp zzcclp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lgbo-ustc lgbo-ustc merged commit bb16109 into apache:main Feb 11, 2025
6 checks passed
baibaichen added a commit to baibaichen/gluten that referenced this pull request Feb 11, 2025
@baibaichen
Fix build due to apache#8676
3e03543
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_02_11_2025_time.csv log/native_master_02_10_2025_39950cf26a_time.csv difference percentage
q1 11.24 11.28 0.045 100.40%
q2 12.01 12.65 0.638 105.31%
q3 3.39 2.51 -0.879 74.07%
q4 52.52 50.97 -1.543 97.06%
q5 7.93 9.52 1.589 120.04%
q6 4.89 2.84 -2.056 57.99%
q7 4.79 4.47 -0.318 93.36%
q8 3.01 4.74 1.734 157.64%
q9 14.65 16.23 1.576 110.75%
q10 12.39 10.65 -1.734 86.00%
q11 28.75 28.89 0.148 100.51%
q12 1.63 3.35 1.725 206.06%
q13 5.00 6.15 1.147 122.94%
q14a 44.13 44.63 0.499 101.13%
q14b 42.23 41.41 -0.816 98.07%
q15 3.57 2.54 -1.028 71.18%
q16 6.10 6.13 0.028 100.46%
q17 5.47 6.58 1.114 120.38%
q18 8.71 8.43 -0.280 96.78%
q19 3.83 3.33 -0.503 86.88%
q20 1.94 2.42 0.484 124.93%
q21 2.05 0.78 -1.268 38.17%
q22 4.16 2.92 -1.234 70.31%
q23a 61.53 61.72 0.188 100.30%
q23b 71.77 72.87 1.098 101.53%
q24a 71.43 74.14 2.711 103.80%
q24b 69.17 67.60 -1.563 97.74%
q25 5.15 5.30 0.157 103.05%
q26 2.69 4.38 1.696 163.09%
q27 2.22 2.56 0.339 115.25%
q28 16.94 18.35 1.414 108.35%
q29 7.95 7.41 -0.533 93.29%
q30 5.48 5.48 0.005 100.08%
q31 7.27 8.86 1.594 121.93%
q32 2.19 1.49 -0.694 68.26%
q33 4.26 3.97 -0.296 93.05%
q34 3.84 3.41 -0.422 88.99%
q35 6.37 7.23 0.857 113.44%
q36 2.67 2.31 -0.363 86.42%
q37 3.26 4.07 0.811 124.91%
q38 11.70 11.77 0.072 100.62%
q39a 4.14 4.25 0.111 102.68%
q39b 3.55 3.54 -0.006 99.83%
q40 3.89 3.12 -0.766 80.27%
q41 0.72 0.62 -0.105 85.41%
q42 0.92 1.12 0.206 122.39%
q43 2.99 2.10 -0.888 70.26%
q44 5.86 6.24 0.378 106.46%
q45 3.37 3.62 0.244 107.25%
q46 4.16 5.38 1.225 129.47%
q47 9.56 10.35 0.790 108.26%
q48 3.65 3.51 -0.143 96.08%
q49 5.48 5.37 -0.109 98.01%
q50 17.39 17.67 0.279 101.61%
q51 7.04 7.96 0.919 113.05%
q52 1.68 0.97 -0.704 58.02%
q53 1.86 1.62 -0.240 87.09%
q54 5.66 5.69 0.032 100.56%
q55 0.66 1.37 0.709 207.02%
q56 4.42 3.94 -0.472 89.30%
q57 7.08 6.57 -0.508 92.83%
q58 2.83 3.30 0.470 116.59%
q59 4.40 3.87 -0.528 88.01%
q60 4.57 5.69 1.115 124.39%
q61 3.77 4.95 1.174 131.14%
q62 3.61 2.82 -0.793 78.04%
q63 1.28 1.30 0.023 101.81%
q64 35.73 36.37 0.642 101.80%
q65 11.72 11.30 -0.420 96.42%
q66 2.79 3.71 0.918 132.92%
q67 58.88 58.04 -0.840 98.57%
q68 2.98 2.80 -0.188 93.71%
q69 5.15 4.89 -0.261 94.93%
q70 5.41 5.77 0.357 106.59%
q71 5.12 4.54 -0.588 88.53%
q72 20.85 22.71 1.858 108.91%
q73 2.40 2.24 -0.158 93.39%
q74 17.29 17.25 -0.039 99.77%
q75 23.40 23.06 -0.337 98.56%
q76 6.46 6.24 -0.216 96.66%
q77 2.85 2.69 -0.160 94.38%
q78 33.11 34.18 1.061 103.20%
q79 3.39 4.21 0.816 124.04%
q80 10.65 11.84 1.190 111.18%
q81 8.01 6.55 -1.462 81.74%
q82 5.64 5.74 0.095 101.68%
q83 1.40 1.75 0.353 125.24%
q84 2.68 2.69 0.010 100.36%
q85 6.18 5.92 -0.263 95.75%
q86 1.98 2.24 0.255 112.83%
q87 11.41 11.84 0.426 103.74%
q88 15.59 15.78 0.191 101.22%
q89 2.28 2.50 0.222 109.74%
q90 1.95 2.07 0.124 106.35%
q91 4.43 4.48 0.051 101.15%
q92 2.15 1.50 -0.650 69.75%
q93 23.79 23.63 -0.161 99.32%
q94 8.65 8.68 0.024 100.27%
q9 59.93 57.15 -2.779 95.36%
q5 2.86 2.46 -0.401 86.00%
q96 10.96 10.57 -0.385 96.48%
q97 2.37 1.96 -0.412 82.62%
q98 5.25 5.35 0.101 101.92%
q99 0.26 0.47 0.218 185.29%
total 1180.75 1188.49 7.740 100.66%

baibaichen added a commit that referenced this pull request Feb 12, 2025
@baibaichen
[GLUTEN-6067] Open spark 35 ut (#8555)
4bb28f2 
* Sync ClickHouseTestSettings with VeloxTestSettings and ignore failed uts

* Fix Velox Build

* exclude failed uts on cicd

* Fix build due to #8676
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zzcclp zzcclp zzcclp approved these changes

Assignees
No one assigned
Labels
CLICKHOUSE performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CH] Rewrite union of multiple aggregates
3 participants
@lgbo-ustc @GlutenPerfBot @zzcclp