I compare different versions:
- main (29f1e778faaa31479e4ef84e2fb37a7a53984550)
- main+revert(+ https://github.com/sergey-miryanov/cpython/commit/efd2a4c0447496396445a5b5acc63a0482f1790d)
- main+gc-fix(+ https://github.com/sergey-miryanov/cpython/commits/gc-fix?author=sergey-miryanov)
I did comparison of pyperformance and gc-bench. gc-bench is pyperf-ed version of yours gc_big and gc_big_tree. Results are following:
gc-bench
| Benchmark | gc-fix-main | gc-fix-revert | gc-fix |
|---|---|---|---|
| Ubuntu | |||
| gc_big_array | 521 ms | not significant | 452 ms: 1.15x faster |
| gc_big_tree | 222 ms | 219 ms: 1.02x faster | 219 ms: 1.02x faster |
| Geometric mean | (ref) | 1.01x faster | 1.08x faster |
| Windows | |||
| gc_big_array | 560 ms | not significant | 520 ms: 1.08x faster |
| gc_big_tree | 238 ms | not significant | 235 ms: 1.01x faster |
| Geometric mean | (ref) | 1.00x slower | 1.05x faster |
pyperformance
| Benchmark | gc-fix-main | gc-fix-revert | gc-fix |
|---|---|---|---|
| Ubuntu | |||
| Geometric mean | (ref) | 1.00x faster | 1.03x slower |
| Windows | |||
| Geometric mean | (ref) | 1.00x slower | 1.04x slower |
Reversion of the work_to_do shows no difference in results nor for the pyperformance neither for the gc-bench. gc-fix shows different results - it wins for the gc-bench and loses for the pyperformance.
Also, I compared 3.14.1 and 3.14.1+revert. And got a bit strange results for the gc-bench:
| Benchmark | 3.14.1 | 3.14.1-revert |
|---|---|---|
| Ubuntu | ||
| gc_big_array | 525 ms | 399 ms: 1.31x faster |
| gc_big_tree | 223 ms | 217 ms: 1.03x faster |
| Geometric mean | (ref) | 1.16x faster |
| Windows | ||
| gc_big_array | 315 ms | 434 ms: 1.38x slower |
| gc_big_tree | 249 ms | 240 ms: 1.04x faster |
| Geometric mean | (ref) | 1.15x slower |
After this I decided to compare minor releases and main. Results for the gc-bench are following:
| Benchmark | 3.13.10 | 3.14.0 | 3.14.1 | main |
|---|---|---|---|---|
| Ubuntu | ||||
| gc_big_array | 65.7 ms | 495 ms: 7.53x slower | 525 ms: 7.98x slower | 521 ms: 7.92x slower |
| gc_big_tree | 249 ms | 219 ms: 1.14x faster | 223 ms: 1.11x faster | 222 ms: 1.12x faster |
| Geometric mean | (ref) | 2.57x slower | 2.68x slower | 2.66x slower |
| Windows | ||||
| gc_big_array | 76.0 ms | 539 ms: 7.08x slower | 315 ms: 4.15x slower | 560 ms: 7.37x slower |
| gc_big_tree | 272 ms | 238 ms: 1.14x faster | 249 ms: 1.09x faster | 238 ms: 1.14x faster |
| Geometric mean | (ref) | 2.49x slower | 1.95x slower | 2.54x slower |
3.14.* and main shows much worse results than 3.13.10.
I ran pyperformance only on windows and got that 3.14.* and main have worse results than 3.13.10 too:
| Benchmark | 3.13.10 | 3.14.0 | 3.14.1 | gc-fix-main |
|---|---|---|---|---|
| Windows | ||||
| Geometric mean | (ref) | 1.09x slower | 1.10x slower | 1.08x slower |