Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC)

@malemburg

BPO	45116
Nosy	@malemburg, @gvanrossum, @rhettinger, @pfmoore, @vstinner, @tjguk, @markshannon, @zware, @zooba, @animalize, @pablogsal, @brandtbucher, @neonene, @erlend-aasland, @Fidget-Spinner
PRs	bpo-45116: Add the Py_ALWAYS_INLINE macro #28390 bpo-45116: Py_DEBUG ignores Py_ALWAYS_INLINE #28419 bpo-45116: Use Py_ALWAYS_INLINE in object.h #28427 [3.10] bpo-45116: Shrink interpreter by targetted revert of #25244 #28475 bpo-45116 Fix another performance regression on Windows #28630 bpo-45116 Add a warm-up for PGO training on Windows #28631 bpo-45116: Make --inlinestat option available in build.bat for diagnostic logs #31436 [3.10] bpo-45116: Fix inlining regressions on Windows Release build #31459 gh-89279: In ceval.c, redefine some macros for speed #32387
Files	310rc1_confirm_overhead.patch ceval_310rc1_patched.c b98e-no-inline-in-all.diff b98e-no-inline-in-eval.diff b98e-no-inline-in-the-others.diff pyproject_inlinestat.patch x64_28d2.log x64_b98e.log 310rc2_benchmarks.txt 310a7_vs_310rc2_bench.txt PR28475_inline.log PR28475_vs_310rc2_vs_310a7.txt PR28475_skip1test_bench.txt 310rc2patched_vs_310rc2notrace.txt switch-case_unarranged_bench.txt ceval_PR29565_split_func.c

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-09-06.15:27:18.216>
labels = ['interpreter-core', '3.10', 'performance', 'expert-C-API', '3.11', 'OS-windows']
title = 'Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC)'
updated_at = <Date 2022-04-08.11:04:12.476>
user = 'https://github.com/neonene'

bugs.python.org fields:

activity = <Date 2022-04-08.11:04:12.476>
actor = 'steve.dower'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'Windows', 'C API']
creation = <Date 2021-09-06.15:27:18.216>
creator = 'neonene'
dependencies = []
files = ['50263', '50264', '50271', '50272', '50273', '50274', '50275', '50276', '50280', '50286', '50291', '50293', '50296', '50315', '50363', '50452']
hgrepos = []
issue_num = 45116
keywords = ['patch']
message_count = 82.0
messages = ['401143', '401152', '401154', '401182', '401183', '401319', '401329', '401346', '401364', '401623', '401624', '401628', '401743', '401964', '401970', '401972', '402025', '402040', '402043', '402044', '402063', '402064', '402065', '402067', '402068', '402071', '402090', '402091', '402092', '402098', '402099', '402117', '402135', '402143', '402189', '402190', '402217', '402229', '402230', '402287', '402289', '402296', '402307', '402308', '402320', '402480', '402856', '402857', '402858', '402864', '402867', '402871', '402878', '402886', '402891', '402893', '402928', '402930', '402954', '403403', '403409', '403430', '403432', '403464', '403559', '403587', '403609', '404089', '406354', '406386', '406407', '406416', '406471', '406474', '406479', '406487', '406613', '407188', '415378', '416911', '416950', '416977']
nosy_count = 15.0
nosy_names = ['lemburg', 'gvanrossum', 'rhettinger', 'paul.moore', 'vstinner', 'tim.golden', 'Mark.Shannon', 'zach.ware', 'steve.dower', 'malin', 'pablogsal', 'brandtbucher', 'neonene', 'erlendaasland', 'kj']
pr_nums = ['28390', '28419', '28427', '28475', '28630', '28631', '31436', '31459', '32387']
priority = None
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue45116'
versions = ['Python 3.10', 'Python 3.11']

pyperformance on Windows shows some gap between 3.10a7 and 3.10b1.
The following are the ratios compared with 3.10a7 (the higher the slower).

-------------------------------------------------
Windows x64 | PGO release official-binary
----------------+--------------------------------
20210405 |
3.10a7 | 1.00 1.24 1.00 (PGO?)
20210408-07:58 |
b98eba5 | 0.98
20210408-10:22 |

PR25244 | 1.04
20210503 |
3.10b1 | 1.07 1.21 1.07
-------------------------------------------------
Windows x86 | PGO release official-binary
----------------+--------------------------------
20210405 |
3.10a7 | 1.00 1.25 1.27 (release?)
20210408-07:58 |
b98eba5 | 1.00
20210408-10:22 |
PR25244 | 1.11
20210503 |
3.10b1 | 1.14 1.28 1.29

Since PR25244 (28d28e0),
_PyEval_EvalFrameDefault() in ceval.c has seemed to be unoptimized with PGO (msvc14.29.16.10).
At least the functions below have become un-inlined there at all.

(1) _Py_DECREF() (from Py_DECREF,Py_CLEAR,Py_SETREF)
(2) _Py_XDECREF() (from Py_XDECREF,SETLOCAL)
(3) _Py_IS_TYPE() (from PyXXX_CheckExact)
(4) _Py_atomic_load_32bit_impl() (from CHECK_EVAL_BREAKER)

I tried in vain other linker options like thread-safe-profiling, agressive-code-generation, /OPT:NOREF.
3.10a7 can inline them in the eval-loop even if profiling only test_array.py.

I measured overheads of (1)~(4) on my own build whose eval-loop uses macros instead of them.

-----------------------------------------------------------------
Windows x64 | PGO patched overhead in eval-loop
----------------+------------------------------------------------
3.10a7 | 1.00
20210802 |
3.10rc1 | 1.09 1.05 4% (slow 43, fast 5, same 10)
20210831-20:42 |
863154c | 0.95 0.90 5% (slow 48, fast 3, same 7)
(3.11a0+) |
-----------------------------------------------------------------
Windows x86 | PGO patched overhead in eval-loop
----------------+------------------------------------------------
3.10a7 | 1.00
20210802 |
3.10rc1 | 1.15 1.13 2% (slow 29, fast 14, same 15)
20210831-20:42 |
863154c | 1.05 1.02 3% (slow 44, fast 7, same 7)
(3.11a0+) |

Rather than defining again functions as macro, you should consider using __forceinline function attribute: see bpo-45094.

Perhaps these critical code sections should have been left as macros. It is difficult to assuring system wide inlining across modules.

If I understand correctly, x86 official binaries are non-PGO builds.

Yeah, this is correct. We're more likely to deprecate and drop the 32-bit binaries before we make any major effort to optimise them - they run under an emulation layer in the OS (practically all supported OS installs are 64-bit native), so aren't really going to be recommended for people who care about performance anyway.

Add comment referring to pythonGH-89279 for is_method()

gh-89279: In ceval.c, redefine some macros for speed (#32387)

I think this issue can be closed. (I can't after migration)

Most of my experiences are invalid after Guido's #91718 corrected the quirks of MSVC.
Another reasonable fix would be a good test which makes specialized sections hotter.

Thanks.

@neonene

Closing as requested by OP. Thanks for your investigations @neonene ! Thanks to Guido too for the fix.

@neonene

Thank you @neonene for your gentle pushes and encouragement and help to get this fixed!

@neonene

@neonene:

Most of my experiences are invalid after Guido's #91718 corrected the quirks of MSVC.

Do you mean that this merged change 2f233fc is now useless?

No they are complementary.

Do you mean that this merged change 2f233fc is now useless?

No. What I said is about the optimization, not the (force) inlining. And what I suggested before have been already fixed by f8dc618 (and 2f233fc):

tp_* or cfunc pointer in the eval-loop can inline multiple callees without conflict.
Moving LOAD_FAST out of switch according to the scores below has no advantage now.

TOP3 entries with current 44 tests
case 124  132522464  // LOAD_FAST
case 100   48956231  // LOAD_CONST
case  45   48318813  // LOAD_FAST__LOAD_FAST

What I understand is that PGO build of Python 3.11 on Windows will be faster thanks to these changes, and the Windows python.org binaries only use PGO for 64-bit, not for 32-bit.

You can read a bit more posts and links because you have changed this thread's title several times.

Can someone please try to write a summary of this long and complex issue? It seems like different but related topics have been discussed and it's hard to get an overview. I'm confused between sometimes someone said that a change fixed the fix and then wrote that no, it's not really fixing the issue.

@neonene

Let me give it a quick try.

Originally, @neonene observed a Windows-specific performance regression in 3.10 between the a7 and b1 release. This was eventually shown to be caused by the function _PyEval_EvalFrameDefault getting so long that the MSVC LTO gave up on inlining many things there. IIUC in 3.10 this was eventually fixed by making the function a bit smaller ([3.10] bpo-45116: Shrink interpreter by targetted revert of #25244 #28475).
Of course, the same issue was then observed in the main branch (3.11). We then went back and forth trying various approaches to fix it. This wasn't easy because (a) the code kept changing (because the "Faster CPython" team was very active -- mostly growing the function), and (b) we had no good hardware or strategy to run reliable benchmarks. The latter problem spawned Measure Windows performance (and improve if lacking) faster-cpython/ideas#321.
Eventually I settled on a fix which consisted of turning a few inline functions back into macros, but only in ceval.c, and not in debug mode (and one only for MSVC). This was PR Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC) #89279, commit 2f233fc.
Somewhat relatedly, I also figured out how to get MSVC to generate slightly faster switch code: if you switch on a one-byte value and all 256 cases exist, it skips a memory load. This was PR Improve performance for switch in ceval.c when using MSVC #91719, commit f8dc618.
Finally, I figured out how to get stable benchmark numbers (see Measure Windows performance (and improve if lacking) faster-cpython/ideas#321 (comment) and following comments) and showed that the macrofied inline functions gave us 10% performance back and the improved switch code gave 3%.

That's it.

Thanks for the summary. I would add that marking performance critical function with __forceinline (Py_ALWAYS_INLINE) was tested, but it didn't work.

Navigation Menu

Repository navigation

Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC) #89279

102 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Performance regression 3.10b1: inlining issue in the big _PyEval_EvalFrameDefault() function with Visual Studio (MSC) #89279

Description

Activity

neonene commented on Sep 6, 2021

vstinner commented on Sep 6, 2021

rhettinger commented on Sep 6, 2021

102 remaining items

zooba commented on Apr 13, 2022

neonene commented on Apr 23, 2022

Fidget-Spinner commented on Apr 23, 2022

gvanrossum commented on Apr 23, 2022

vstinner commented on Apr 25, 2022

gvanrossum commented on Apr 25, 2022

neonene commented on Apr 25, 2022

vstinner commented on Apr 25, 2022

neonene commented on Apr 25, 2022

vstinner commented on Apr 26, 2022

gvanrossum commented on Apr 28, 2022

vstinner commented on Apr 28, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions