Skip to content

bpo-45188: Windows now regenerates frozen modules at the start of build instead of late #28322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Sep 15, 2021
Merged

bpo-45188: Windows now regenerates frozen modules at the start of build instead of late #28322

merged 4 commits into from
Sep 15, 2021

Conversation

zooba
Copy link
Member

@zooba zooba commented Sep 13, 2021 •
edited by bedevere-bot
Loading

@zooba zooba requested a review from a team as a code owner September 13, 2021 23:51
@zooba
Copy link
Member Author

zooba commented Sep 14, 2021

I haven't looked at how much slower this makes the build, but it ought to be a minute or so. It'd be possible to make the freeze_module project share a temp directory with pythoncore, which would speed it up, but that's going to hurt reliability so I'd rather not.

ericsnowcurrently
ericsnowcurrently approved these changes Sep 14, 2021
View reviewed changes
Copy link
Member

@ericsnowcurrently ericsnowcurrently left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks for doing this!

@ericsnowcurrently
Copy link
Member

I think you can also drop the special-case in regen_pcbuild() in Tools/scripts/freeze_modules.py, where we skip the module if it isn't in ESSENTIAL.

@zooba zooba added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 14, 2021
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @zooba for commit 4250dc7 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 14, 2021
@zooba
Copy link
Member Author

zooba commented Sep 14, 2021

I think you can also drop the special-case in regen_pcbuild() in Tools/scripts/freeze_modules.py, where we skip the module if it isn't in ESSENTIAL.

Potentially, but I'll leave that one to you ;) In case it breaks other things

zooba
zooba commented Sep 15, 2021
View reviewed changes
Tools/scripts/freeze_modules.py Outdated
@@ -523,7 +523,7 @@ def regen_pcbuild(modules):
continue
pyfile = os.path.relpath(src.pyfile, ROOT_DIR).replace('/', '\\')
header = os.path.relpath(src.frozenfile, ROOT_DIR).replace('/', '\\')
intfile = header.split('\\')[-1].strip('.h') + '.g.h'
intfile = os.path.splitext(os.path.basename(header))[0] + '.g.h'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @ericsnowcurrently I fixed a bug in your script here (should've been removesuffix('.h') rather than strip, but better to stick with os.path when dealing with paths anyway).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this. I was going to suggest using ntpath explicitly here, but in this case it really should be os.path. 🙂

@zooba zooba merged commit 09b4ad1 into python:main Sep 15, 2021
@zooba zooba deleted the bpo-45188 branch September 15, 2021 17:11
@ericsnowcurrently
Copy link
Member

Thanks again for doing this, @zooba!

@ericsnowcurrently ericsnowcurrently mentioned this pull request Sep 15, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ericsnowcurrently ericsnowcurrently ericsnowcurrently approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@zooba @ericsnowcurrently @bedevere-bot @the-knights-who-say-ni