Skip to content

Introduce MetaPathFinder.discover and PathEntryFinder.discover #139899

Description

@FFY00

Proposal:

In certain scenarios, such as implementing error suggestions for ModuleNotFoundError, it would be helpful to have the ability to ask the finder to search for possible specs.

I propose adding a new optional discover method to MetaPathFinder and PathEntryFinder, as follows:

   .. method:: discover(parent=None)

      An optional method which searches for possible specs with given *parent*.
      If *parent* is *None*, :meth:`MetaPathFinder.discover` will search for
      top-level modules.

      Returns an iterable of possible specs.
Python 3.15.0a0 free-threading build (heads/main:744ec1d6c39, Oct 10 2025, 06:22:11) [GCC 15.2.1 20250813] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> path_finder = sys.meta_path[-1]
>>> import sysconfig
>>> for spec in path_finder.discover(sysconfig):
...     print(spec)
...
ModuleSpec(name='sysconfig.__init__', loader=<_frozen_importlib_external.SourceFileLoader object at 0x20000922c20>, origin='/home/anubis/git/cpython/Lib/sysconfig/__init__.py')
ModuleSpec(name='sysconfig.__main__', loader=<_frozen_importlib_external.SourceFileLoader object at 0x20000922ca0>, origin='/home/anubis/git/cpython/Lib/sysconfig/__main__.py')
>>> import urllib
>>> for spec in path_finder.discover(urllib):
...     print(spec)
...
ModuleSpec(name='urllib.robotparser', loader=<_frozen_importlib_external.SourceFileLoader object at 0x20000922220>, origin='/home/anubis/git/cpython/Lib/urllib/robotparser.py')
ModuleSpec(name='urllib.response', loader=<_frozen_importlib_external.SourceFileLoader object at 0x20000920d20>, origin='/home/anubis/git/cpython/Lib/urllib/response.py')
ModuleSpec(name='urllib.__init__', loader=<_frozen_importlib_external.SourceFileLoader object at 0x20000921ba0>, origin='/home/anubis/git/cpython/Lib/urllib/__init__.py')
ModuleSpec(name='urllib.request', loader=<_frozen_importlib_external.SourceFileLoader object at 0x20000920520>, origin='/home/anubis/git/cpython/Lib/urllib/request.py')
ModuleSpec(name='urllib.parse', loader=<_frozen_importlib_external.SourceFileLoader object at 0x20000921520>, origin='/home/anubis/git/cpython/Lib/urllib/parse.py')
ModuleSpec(name='urllib.error', loader=<_frozen_importlib_external.SourceFileLoader object at 0x200009214a0>, origin='/home/anubis/git/cpython/Lib/urllib/error.py')

Possibly, we could also add a helper function that searches sys.meta_path, similarly to importlib.util.find_spec. Although I don't think it is strictly necessary.

def discover(parent=None):
    for finder in sys.meta_path:
        try:
            discover = getattr(finder, 'discover')
        except AttributeError:
            continue
        yield from discover(parent)

This allows us to implement suggestions in traceback using the import system abstractions — see https://github.com/FFY00/cpython/tree/ModuleNotFoundError-suggestions.

Python 3.15.0a0 free-threading build (heads/main:744ec1d6c39, Oct 10 2025, 06:22:11) [GCC 15.2.1 20250813] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asd
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import asd
ModuleNotFoundError: No module named 'asd'. Did you mean: 'ast'?
>>> import importlib.utill
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    import importlib.utill
ModuleNotFoundError: No module named 'importlib.utill'. Did you mean: 'importlib.util'?

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

Activity

  1. added 2 commits that reference this issue on Oct 10, 2025
  2. Locked-chess-official commented on Oct 10, 2025

    @Locked-chess-official
    Contributor

    Great. This way is like the method named "__find__" in my first plan. In my full plan, the message "No module named 'importlib.utill'. Did you mean: 'importlib.util'?" changes to "module 'importlib' has no child module 'utill'. Did you mean 'util'?", but it is up to everyone.

  3. added
    stdlibStandard Library Python modules in the Lib/ directory
    on Oct 10, 2025
  4. Locked-chess-official commented on Oct 14, 2025

    @Locked-chess-official
    Contributor

    Now I use my test test_wrong_module_exception for the branch and use the folder structure like it (to avoid for long name):

    aa\
        __init__.py
        bbb.py
    aaa\
        __init__.py
        bbb\
            __init__.py
            cccc.py
    

    However, the result is not expect. I will use my tool in friendly_module_not_found_error to find out where is wrong.
    Image

  5. Locked-chess-official commented on Oct 14, 2025

    @Locked-chess-official
    Contributor

    My fault :-( . The suggestion code is not there. I will try for that code.

  6. Locked-chess-official commented on Oct 14, 2025

    @Locked-chess-official
    Contributor

    OK. Now I sure it is a bug in that branch:

    Image

    The right now implement on that branch cannot handle on relative import.

  7. Locked-chess-official commented on Oct 14, 2025

    @Locked-chess-official
    Contributor

    This is the question:

    Image

    I suggest that the method discover and the code when finding the suggestion for ModuleNotFoundError should never import any modules.

  8. FFY00 commented on Oct 14, 2025

    @FFY00
    MemberAuthor

    Yes, that's a bug on relative imports.

    I suggest that the method discover and the code when finding the suggestion for ModuleNotFoundError should never import any modules.

    It triggers an import for the parent module, but it should never actually result in a new module being loaded. If the ModuleNotFoundError is triggering on the child, it means the parent was successfully imported, so it should always be in sys.modules already.

  9. Locked-chess-official commented on Oct 14, 2025

    @Locked-chess-official
    Contributor

    If the ModuleNotFoundError is triggering on the child, it means the parent was successfully imported, so it should always be in sys.modules already.

    This statement is not always true. In relative import, when failed, the module will fail to load so finally it won't appear in sys.modules.

  10. FFY00 commented on Oct 14, 2025

    @FFY00
    Author
  11. Locked-chess-official commented on Oct 14, 2025

    @Locked-chess-official
    Contributor

    Which module? I am talking about the parent. In the current implementation, the parent is always in sys.modules, we need its __path__. Taking your example, aaa is in sys.modules.

    For example, aa.__init__ want's to import module bb from its __path__, but there is no bb there, so it failed. It means that the code in aa.__init__ failed. Any exceptions raised from the "__init__.py" will leads to that the module failed to load, so the parent module aa won't be in sys.modules.

    Image
  12. FFY00 commented on Oct 14, 2025

    @FFY00
    MemberAuthor

    Right, that'll be true for regular packages, I suppose, when the error happens while we are executing the module. We can change the API to take a module spec instead.

    I need to have a better look at this.

    Thanks for the feedback!

  13. Locked-chess-official commented on Oct 16, 2025

    @Locked-chess-official
    Contributor

    Now there is another bug here:

    Image If there is a module name that too long, the suggestion in this implement can give any suggestions on child module that exist, even if the name of the child module is vastly different from the suggestion. I suggest that the comparsion should be only limit to the child module name (because that the parent module can always be found). More over, I suggest that don't suggest on the fullname. In relative import, the full module's name will be spreaded across multiple code files so that full-name suggestion need to be split.
  14. brettcannon commented on Nov 7, 2025

    @brettcannon
    Member

    Is this basically a spec-like replacement for https://docs.python.org/3/library/pkgutil.html#pkgutil.iter_importers ?

  15. Locked-chess-official commented on Nov 8, 2025

    @Locked-chess-official
    Contributor

    Is this basically a spec-like replacement for https://docs.python.org/3/library/pkgutil.html#pkgutil.iter_importers ?

    Seems not. pkgutil.iter_importers find the finders instead of submodules. More exactly, pkgutil.iter_importers can simplify the logic to find "which finder find the parent package", but how to find the list of submodules depends on the finder itself.

  16. brettcannon commented on Nov 14, 2025

    @brettcannon
    Member

    I'm fine with the idea, although we might want to use this as an opportunity to introduce protocols to importlib to move away from ABCs.

  17. added a commit that references this issue on Dec 10, 2025
  18. Sachaa-Thanasius commented on Jan 14, 2026

    @Sachaa-Thanasius
    Contributor

    Would it be more accurate to say that this could act as a spec-like replacement for pkgutil.iter_modules and the non-standard finder.iter_modules API it works with?

    If so, there may be an opportunity here to further advance #64138 by deprecating pkgutil.iter_modules and potentially pkgutil.walk_packages while also turning them into thin wrappers around MetaPathFinder.discover calls. Assuming I understand this correctly, it would be much clearer how users could replace usage of those functions with this new API in their code, at least compared to now, imo.

  19. added a commit that references this issue on Feb 19, 2026
  20. FFY00 commented on Feb 25, 2026

    @FFY00
    MemberAuthor

    Implemented in GH-139900.

    Let's track the conversion to protocols in GH-82963, as it is not trivial, and whatever we do must follow the approach that will be decided there.

  21. added a commit that references this issue on Feb 28, 2026
  22. added a commit that references this issue on Apr 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytopic-importlibtype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions