Skip to content

PGO tests fail on 3.11 and 3.12 #111929

Description

@neonene

Bug report

Bug description:

Recent PGO tests on Python 3.11 and 3.12 result in failure.
For example, Linux Buildbots' logs (compile section) are like:

3.12 (AMD64 RHEL7 LTO + PGO)

Using random seed: 2130237982
0:00:00 load avg: 1.75 Run 44 tests sequentially (timeout: 20 min)
0:00:00 load avg: 1.75 [ 1/44] test_array
0:00:01 load avg: 1.75 [ 2/44] test_base64
0:00:02 load avg: 1.75 [ 3/44] test_binascii
0:00:02 load avg: 1.75 [ 4/44] test_binop
0:00:02 load avg: 1.75 [ 5/44] test_bisect
0:00:03 load avg: 1.75 [ 6/44] test_bytes
0:00:15 load avg: 1.58 [ 7/44] test_bz2
0:00:16 load avg: 1.58 [ 8/44] test_cmath
0:00:16 load avg: 1.58 [ 9/44] test_codecs
0:00:19 load avg: 1.54 [10/44] test_collections
0:00:22 load avg: 1.54 [11/44] test_complex
0:00:22 load avg: 1.54 [12/44] test_dataclasses
0:00:23 load avg: 1.49 [13/44] test_datetime
0:00:32 load avg: 1.45 [14/44] test_decimal
0:00:42 load avg: 1.38 [15/44] test_difflib
0:00:45 load avg: 1.35 [16/44] test_embed
0:01:00 load avg: 1.12 [17/44] test_float
0:01:00 load avg: 1.12 [18/44] test_fstring
0:01:03 load avg: 1.12 [19/44] test_functools
0:01:04 load avg: 1.11 [20/44] test_generators
0:01:04 load avg: 1.11 [21/44] test_hashlib
0:01:06 load avg: 1.11 [22/44] test_heapq
0:01:08 load avg: 1.11 [23/44] test_int
0:01:09 load avg: 1.10 [24/44] test_itertools
0:01:20 load avg: 1.09 [25/44] test_json
0:01:25 load avg: 1.00 [26/44] test_long
0:01:33 load avg: 1.00 [27/44] test_lzma
0:01:34 load avg: 1.00 [28/44] test_math
0:01:42 load avg: 1.00 [29/44] test_memoryview
0:01:43 load avg: 1.00 [30/44] test_operator
0:01:43 load avg: 1.00 [31/44] test_ordered_dict
0:01:45 load avg: 1.00 [32/44] test_patma
0:01:46 load avg: 1.00 [33/44] test_pickle
0:02:01 load avg: 1.00 [34/44] test_pprint
0:02:02 load avg: 1.00 [35/44] test_re
0:02:04 load avg: 1.00 [36/44] test_set
0:02:18 load avg: 1.00 [37/44] test_sqlite3
0:02:19 load avg: 1.00 [38/44] test_statistics
0:02:25 load avg: 1.00 [39/44] test_str
0:02:25 load avg: 1.00 [40/44] test_struct -- test_str failed (uncaught exception)
0:02:26 load avg: 1.00 [41/44] test_tabnanny
0:02:27 load avg: 1.00 [42/44] test_time
0:02:30 load avg: 0.92 [43/44] test_xml_etree
0:02:31 load avg: 0.92 [44/44] test_xml_etree_c

Total duration: 2 min 33 sec
Total tests: run=8,699 skipped=193
Total test files: run=44/44 failed=1
Result: FAILURE

3.11 (AMD64 RHEL7 LTO + PGO)

Using random seed: 2899545713
0:00:00 load avg: 2.37 Run 44 tests sequentially (timeout: 20 min)
0:00:00 load avg: 2.37 [ 1/44] test_array
0:00:02 load avg: 2.37 [ 2/44] test_base64
0:00:02 load avg: 2.37 [ 3/44] test_binascii
0:00:03 load avg: 2.37 [ 4/44] test_binop
0:00:03 load avg: 2.37 [ 5/44] test_bisect
0:00:03 load avg: 2.37 [ 6/44] test_bytes
0:00:12 load avg: 2.16 [ 7/44] test_bz2
0:00:13 load avg: 2.16 [ 8/44] test_cmath
0:00:14 load avg: 2.16 [ 9/44] test_codecs
0:00:14 load avg: 2.16 [10/44] test_collections -- test_codecs failed (uncaught exception)
0:00:14 load avg: 2.16 [11/44] test_complex -- test_collections failed (uncaught exception)
0:00:14 load avg: 2.16 [12/44] test_dataclasses -- test_complex failed (uncaught exception)
0:00:14 load avg: 2.16 [13/44] test_datetime -- test_dataclasses failed (uncaught exception)
0:00:14 load avg: 2.16 [14/44] test_decimal -- test_datetime failed (uncaught exception)
0:00:14 load avg: 2.16 [15/44] test_difflib -- test_decimal failed (uncaught exception)
0:00:14 load avg: 2.16 [16/44] test_embed -- test_difflib failed (uncaught exception)
0:00:14 load avg: 2.16 [17/44] test_float -- test_embed failed (uncaught exception)
0:00:14 load avg: 2.16 [18/44] test_fstring -- test_float failed (uncaught exception)
0:00:14 load avg: 2.16 [19/44] test_functools -- test_fstring failed (uncaught exception)
0:00:14 load avg: 2.16 [20/44] test_generators -- test_functools failed (uncaught exception)
0:00:14 load avg: 2.16 [21/44] test_hashlib -- test_generators failed (uncaught exception)
0:00:14 load avg: 2.16 [22/44] test_heapq -- test_hashlib failed (uncaught exception)
0:00:14 load avg: 2.16 [23/44] test_int -- test_heapq failed (uncaught exception)
0:00:14 load avg: 2.16 [24/44] test_itertools -- test_int failed (uncaught exception)
0:00:14 load avg: 2.16 [25/44] test_json -- test_itertools failed (uncaught exception)
0:00:14 load avg: 2.16 [26/44] test_long -- test_json failed (uncaught exception)
0:00:14 load avg: 2.16 [27/44] test_lzma -- test_long failed (uncaught exception)
0:00:14 load avg: 2.16 [28/44] test_math -- test_lzma failed (uncaught exception)
0:00:14 load avg: 2.16 [29/44] test_memoryview -- test_math failed (uncaught exception)
0:00:14 load avg: 2.16 [30/44] test_operator -- test_memoryview failed (uncaught exception)
0:00:14 load avg: 2.16 [31/44] test_ordered_dict -- test_operator failed (uncaught exception)
0:00:14 load avg: 2.16 [32/44] test_patma -- test_ordered_dict failed (uncaught exception)
0:00:14 load avg: 2.16 [33/44] test_pickle -- test_patma failed (uncaught exception)
0:00:14 load avg: 2.16 [34/44] test_pprint -- test_pickle failed (uncaught exception)
0:00:14 load avg: 2.16 [35/44] test_re -- test_pprint failed (uncaught exception)
0:00:14 load avg: 2.16 [36/44] test_set -- test_re failed (uncaught exception)
0:00:14 load avg: 2.16 [37/44] test_sqlite3 -- test_set failed (uncaught exception)
0:00:14 load avg: 2.16 [38/44] test_statistics -- test_sqlite3 failed (uncaught exception)
0:00:14 load avg: 2.16 [39/44] test_str -- test_statistics failed (uncaught exception)
0:00:14 load avg: 2.16 [40/44] test_struct -- test_str failed (uncaught exception)
0:00:14 load avg: 2.16 [41/44] test_tabnanny -- test_struct failed (uncaught exception)
0:00:14 load avg: 2.16 [42/44] test_time -- test_tabnanny failed (uncaught exception)
0:00:14 load avg: 2.16 [43/44] test_xml_etree -- test_time failed (uncaught exception)
0:00:14 load avg: 2.16 [44/44] test_xml_etree_c -- test_xml_etree failed (uncaught exception)
test_xml_etree_c failed (uncaught exception)

Total duration: 14.2 sec
Total tests: run=1,388 skipped=9
Total test files: run=44/44 failed=36
Result: FAILURE

Seems related to issue gh-110276, gh-110756.
cc: @vstinner @AlexWaygood

CPython versions tested on:

3.12

Operating systems tested on:

Windows

Linked PRs

Activity

  1. added 2 commits that reference this issue on Nov 10, 2023
  2. added a commit that references this issue on Nov 10, 2023
  3. vstinner commented on Nov 10, 2023

    @vstinner
    Member

    I'm working on fixes.

  4. AlexWaygood commented on Nov 10, 2023

    @AlexWaygood
    Member

    How do we ensure this isn't repeated the next time we try to sync libregrtest in the backport branches with the code on the main branch?

  5. added a commit that references this issue on Nov 10, 2023
  6. vstinner commented on Nov 10, 2023

    @vstinner
    Member

    I'm considering to backport the Makefile change which treats a failure... as a failure: don't ignore the failure.

  7. vstinner commented on Nov 10, 2023

    @vstinner
    Member

    I'm considering to backport the Makefile change which treats a failure... as a failure: don't ignore the failure.

    I created #111950 for that.

  8. added a commit that references this issue on Nov 12, 2023
  9. vstinner commented on Nov 15, 2023

    @vstinner
    Member

    Thanks @neonene for your bug report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions