Repository navigation
Add more details about the Python build in sys.version #100086
Description
Activity
I don't think this change should be made. It potentially affects all users of Python, for example, the REPL users, but is of no benefit to 99% of them. All of this information is readily available via multiple other mechanisms, in particular, via
python3 -m test.pythoninfoor viasysconfig.get_config_vars(). And as implemented in the PR, the information is truncated when displayed due to the string format limitation ingetversion.cand can make the header unnecessarily long:Python 3.12.0a3+ (heads/py_build_str-dirty:94b93efeed, Dec 7 2022, 16:23:18, release framework bu) [Clang 14.0.0 (clang-1400.0.29.202)] on darwinOne motivation for me is that many users don't use Python binaries from python.org or from their Linux packaging system, but build their own binary with their custom recipe and it's not trivial to guess how the binary was built. Was it properly optimized with LTO+PGO? Was it built in debug mode?
For a long time, the most popular Docker recipe to build Python used
./configure && make: no LTO, no PGO.Today, I cannot recall if the macOS binary is optimized with LTO or not.
Recently, @JulienPalard shared his recipe to build a custom Python with few lines of shell code: compile-python.sh. I was surprised to see
--with-pydebugin the middle of the script: why building Python in debug mode? Some users may use it without paying attention to this configure flag.in particular, via python3 -m test.pythoninfo or via sysconfig.get_config_vars()
Well, these APIs are not great :-(
test.pythoninfo is designed to be used on the command line, it doesn't have a simple API to query info about the current Python.
sysconfig.get_config_vars() is hard to use: you have to guess which variable is the best to get compiler flags: make your choice in this long list, it can depend on how Python was built, in case of doubt you might have to check multiple variables. Moreover, once you have compiler options, you have to parse this string which is not trivial (I hate parsing shell). For LTO, do you have to check compiler flags or linker flags, hum?
It potentially affects all users of Python, for example, the REPL users, but is of no benefit to 99% of them.
Ah? I'm not the 99%, I build Python on a daily basis and work on Python built in various ways. I know that :-)
I saw this "debug build" vs "release build" as an useful information, but maybe it should be omitted for the common case: "release build" string? What about "lto+pgo": is it useful for most users?
On my Fedora 37, sys.version is
Python 3.11.0 (main, Oct 24 2022, 00:00:00) [GCC 12.2.1 20220819 (Red Hat 12.2.1-2)] on linux.String length:
- Python version: 8% (6 characters)
- space + Build info: 39% (30 characters)
- space + compiler name and version: 53% (41 characters)
- Total: 77 characters
To make this string shorter, the compiler part should be shorter:
[GCC 12.2.1 20220819 (Red Hat 12.2.1-2)] on linux.At the beginning, I started by modifying
test.libregrtestto add some "build details" in the first lines of the test runner:== CPython 3.12.0a2+ (heads/main-dirty:9dc787ea96, Dec 6 2022, 11:51:12) [GCC 9.4.0] == Linux-5.15.0-1019-azure-x86_64-with-glibc2.31 little-endian == cwd: /opt/buildbot/bcannon-wasm/3.x.bcannon-wasm.emscripten-node-pthreads/build/build_oot/host/build/test_python_829492æ == CPU count: 8 == encodings: locale=UTF-8, FS=utf-8 Using random seed 7151121 0:00:00 load avg: 7.01 Run tests in parallel using 2 child processes (timeout: 15 min, worker timeout: 20 min)These lines don't say if Python was built in debug mode or not. The
test.pythonfodata is not far in CIs, but it's annoying to have to reach thetest.pythoninfostep, unfold it, scroll (this is especially painful in the buildbot web UI), just to check if Python was built in debug mode or not.The first line is coming from the code:
print("==", platform.python_implementation(), *sys.version.split())Since
sys.versionis used, I thought that maybe putting the info directly insys.versioncan benefit to more users.About the REPL,
python2andpypyput the info on two lines which makes it more readable IMO:$ python2 Python 2.7.18 (default, Aug 22 2022, 00:00:00) [GCC 12.2.1 20220819 (Red Hat 12.2.1-1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>and
$ pypy3 Python 3.9.12 (2fc6706f5902, Oct 12 2022, 21:40:58) [PyPy 7.3.9 with GCC 12.2.1 20220819 (Red Hat 12.2.1-2)] on linux Type "help", "copyright", "credits" or "license" for more information. >>>>The single line format (
sys.version) is also used by thepython -VVcommand:$ python3 -VV Python 3.11.0 (main, Oct 24 2022, 00:00:00) [GCC 12.2.1 20220819 (Red Hat 12.2.1-2)]- added a commit that references this issue
on Dec 8, 2022
Currently, it's not easy to guess how Python was built just by looking at
python -VVoutput orsys.version.I propose to enhance
sys.versionto include more details about how Python was configured for the build:It should help me to more quickly identify if a buildbot was built in debug mode or release mode.
It should help to see how Python was optimized: with or without LTO and PGO optimizations?
It should help to quickly have an idea of which ABI is used: the "pystats" ABI is different than the "debug" than the "release" ABI.
Linked PRs