Skip to content

Generated executable aliases do not work for packages that rely on multiprocessing (vermin) #361

@n67-exe

Description

@n67-exe

Install source and version

  • Installed from the Windows Store
  • Installed with the MSIX from python.org
  • Installed with the MSI from python.org
  • Installed with winget install 9NQ7512CXL7T

Version: 26.2

Describe the bug
I encountered the bug while trying to use vermin (package) through the executable aliases that pymanager generates in %USERPROFILE%\AppData\Local\Python\bin. However i believe this is a much broader problem related to the use of multiprocessing module.
When multiprocessing is used with the spawn and forkserver start methods, child processes must be able to safely import the main module without executing the same threading logic.
For example, vermin doesn't execute its main function, it is called explicitly by an executable, making the main module (vermin.main) safe to import (as far as i understand). However, from my experimentation, the executable alias that pymanager provides (vermin.exe) calls vermin.exe.__script__.py, which isn't safe to import as it calls vermin.main() unconditionally.
I expect executable aliases to fail in a similar manner with other packages that rely on multiprocessing module the same way. But I wasn't able to confirm it yet.

I can think of a few ways to fix this issue:

  • Calling the original executable directly if it exists in %USERPROFILE%\AppData\Local\Python\pythoncore-*\Scripts
    Probably the most reliable, but not very versatile.
  • Making *.__script__.py files safe to import by adding an explicit __name__ check:
    if __name__ == '__main__':
        from ... import main
        sys.exit(main())

To Reproduce
Steps to reproduce the behavior (an example using vermin):

  1. Install vermin 1.8.0
    py -m pip install vermin==1.8.0
  2. Refresh executable aliases in %USERPROFILE%\AppData\Local\Python\bin
    py install --refresh
  3. Ensure %USERPROFILE%\AppData\Local\Python\bin is in the PATH and %USERPROFILE%\AppData\Local\Python\pythoncore-*\Scripts is not (or has lower priority). Alternatively you can use %USERPROFILE%\AppData\Local\Python\bin\vermin in step 5 to emulate the same conditions.
    where vermin
  4. Create an empty directory with a single empty main.py file.
  5. From this directory run
    vermin --processes=2 .
  6. See vermin infinitely calling itself and displaying errors
    RuntimeError: If running `detect_paths()` outside of `if __name__ == "__main__":`
    it, or the code calling it, must be done within it instead.
    No files specified to analyze!
    

Expected behavior
If you run vermin with --processes=1 (it avoids using multithreading in that case), or if you run the original vermin executable in %USERPROFILE%\AppData\Local\Python\pythoncore-*\Scripts instead, it will display the expected report without any errors:

Tips:
- Since '# novm' or '# novermin' weren't used, a speedup can be achieved using: --no-parse-comments
(disable using: --no-tips)

Minimum required versions: ~2, ~3

Note: Not enough evidence to conclude it won't work with Python 2 or 3.

Additional context
vermin.exe.__script__.py, generated consistently with py install --refresh.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions