Enable ninja to use > 64 CPUs on Windows

[jessemckenna]

This incorporates a fix to ninja's calculation of the number of processors on Windows systems with more than 64 cores.

Windows systems with more than 64 logical processors divide the processors into groups, with most applications using only one group. Previously, ninja was using a Windows API call that only counted the number of processors in the first processor group, which resulted in only partial utilization of high-processing-power systems when no -j value was specified.

When building Chrome on a 72-core Windows system, this change has the following effect:

Before:
ninja detects 36 cores (i.e. one processor group only)
Average time to build Chrome over 10 builds: 5628 s

After:
ninja detects all 72 cores
Average time to build Chrome over 10 builds: 3215 s (57% of the time needed before)

This fixes #1603 .

[atetubou]

You need to have -D_WIN32_WINNT=0x0601 in cygwin build?

[jessemckenna]

Thanks so much @atetubou for the helpful suggestion! : )

[jhasse]

What does that change mean in terms of the windows platforms that Ninja will execute on?

It means that Ninja requires Windows 7 (Windows Server 2008) now.

Does ninja target a specific windows platform? E.g. is Ninja still supporting windows XP? Vista? 8? 8.1?

At the moment: All of them. After this PR: Only 8 and 8.1.

For this PR to be merged, someone with a >64 core machine needs to test it. Also anyone with an unusual Windows setup (e.g. 2 CPUs) could give it a try.

[jessemckenna]

If losing support for older versions of Windows is not an option, the alternative would be for Windows users with > 64 processors to explicitly pass a -j value that reflects their actual number of processors. This could perhaps be reflected in the documentation somehow.

Re: testing, I believe there is a way to manually set the system group size (see: https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/boot-parameters-to-test-drivers-for-multiple-processor-group-support#create-multiple-processor-groups-by-changing-the-group-size). This change could be tested by setting group size to (number of processors / 2) to create two processor groups. I believe the ninja will then detect only half the system processors, i.e. one processor group.

[atetubou]

Maybe better to keep fallback to GetNativeSystemInfo so that ninja can run on older os for a while?

[jhasse]

Windows 7 reaches total end-of-life in January 2020.

Windows 7 would still be supported.

Maybe better to keep fallback to GetNativeSystemInfo so that ninja can run on older os for a while?

Do you still need XP/Vista support at Google? If not, I don't see who would honestly.

[jhasse]

which would be end-of-life by the time it was included in an official release

That's just speculation.

[atetubou]

Maybe better to keep fallback to GetNativeSystemInfo so that ninja can run on older os for a while?

Do you still need XP/Vista support at Google? If not, I don't see who would honestly.

No, we don't use XP/Vista anymore. So I'm fine with this patch.