AttributeError when copying flags in the re module.

Bug report

In CPython 3.11, I've noticed that an attempt to copy (combined) regular expression flags fails with an attribute error; like so:

>>> import copy, re
>>> copy.copy(re.A | re.I)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lib/python3.11/copy.py", line 102, in copy
    return _reconstruct(x, None, *rv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/copy.py", line 265, in _reconstruct
    y = func(*args)
        ^^^^^^^^^^^
  File "lib/python3.11/enum.py", line 862, in __getattr__
    raise AttributeError(name) from None
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: ASCII|IGNORECASE

Your environment

CPython versions tested on: 3.11.0b3
Operating system and architecture: macOS 10.15.7

Background

This behavior was detected from a CrossHair test failure here, in which we realize possibly symbolic arguments prior to regex compilation. (it's an esoteric use case, and I expect not very high priority)

@ethanfurman

Thanks for reporting! Confirmed that copy.copy(re.A | re.I) fails on 3.11 but works on 3.10. Marking as a release blocker since this is a regression.

cc @ethanfurman for enums

Also repros with enum.IntFlag directly:

$ ./python.exe
Python 3.11.0b3+ (heads/3.11:7aa4038a6e, Jun 14 2022, 09:35:10) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import copy, enum
>>> class F(enum.IntFlag):
...     a = 2
...     b = 4
... 
>>> copy.copy(F.a | F.b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jelle/py/cpython/Lib/copy.py", line 102, in copy
    return _reconstruct(x, None, *rv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jelle/py/cpython/Lib/copy.py", line 265, in _reconstruct
    y = func(*args)
        ^^^^^^^^^^^
  File "/Users/jelle/py/cpython/Lib/enum.py", line 862, in __getattr__
    raise AttributeError(name) from None
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: a|b

Also occurs on 3.12

In 3.11 the __reduce__ and __reduce_ex__ methods returns different values:

# 3.10
>>> import re
>>> flags = re.I | re.A
>>> flags.__reduce__()
(<function _reconstructor at 0x7f58bb775ea0>, (<enum 'RegexFlag'>, <class 'int'>, 258), {'_name_': None, '_value_': 258})
>>> flags.__reduce_ex__(1)
(<enum 'RegexFlag'>, (258,))
>>> flags._name_ is None
True
>>> flags.name is None
True

# 3.11
>>> import re
>>> flags = re.I | re.A
>>> flags.__reduce__()
(<function _reconstructor at 0x7fd0b8d4f920>, (<flag 'RegexFlag'>, <class 'int'>, 258), {'_value_': 258, '_name_': 'ASCII|IGNORECASE'})
>>> flags.__reduce_ex__(1)
(<built-in function getattr>, (<flag 'RegexFlag'>, 'ASCII|IGNORECASE'))
>>> flags._name_
'ASCII|IGNORECASE'
>>> flags.name
'ASCII|IGNORECASE'

I guess the regression was introduced by GH-26658 and commit 62f1d2b .

A quick fix would be to reduce flags by value instead of by name.

class Flag(Enum, boundary=STRICT):
    def __reduce_ex__(self, proto):
        return self.__class__, (self._value_, )

Running the example in 0507303 so works fine, so commit 62f1d2b introduced the issue

Oh, it looks like we came across it at the same time 😂

@ethanfurman

PR #93876 is a potential fix. Waiting for @ethanfurman to confirm.

gh-93820: Fix copy() regression in enum.Flag (GH-93876)

pythongh-93820: Fix copy() regression in enum.Flag (pythonGH-93876)

Ideally, re.I | re.A should be reconstructed as operator.or_(re.ASCII, re.IGNORECASE).

It gets a bit tricky when the flags have a residual member, e.g. re.A | re.I | 0x10000. IntFlag supports additional values that are not covered by an enum member.

pythongh-93820: Pickle enum.Flag by name

gh-93820: Fix copy() regression in enum.Flag (GH-93876) (#93886)

Revert 30935 defer bpo45162 to 312 (#94285)

gh-93820: Pickle enum.Flag by name (GH-93891)

[3.11] pythongh-93820: Pickle enum.Flag by name (pythonGH-93891).

[3.11] gh-93820: Pickle enum.Flag by name (GH-93891). (GH-94288)

Navigation Menu

Repository navigation

AttributeError when copying flags in the `re` module. #93820

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

AttributeError when copying flags in the re module. #93820

Description

Activity

JelleZijlstra commented on Jun 14, 2022

JelleZijlstra commented on Jun 14, 2022

dignissimus commented on Jun 15, 2022

tiran commented on Jun 15, 2022

tiran commented on Jun 15, 2022

dignissimus commented on Jun 15, 2022

dignissimus commented on Jun 15, 2022

tiran commented on Jun 15, 2022

serhiy-storchaka commented on Jun 16, 2022

tiran commented on Jun 16, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

AttributeError when copying flags in the `re` module. #93820