Fix the msvc-build scripts (specifically those creating VS projects) for g4w-sdk / g4w #256
Conversation
|
Just wondering if anyone has had a chance to look at this within the new SDK (rather than the msysgit environment which was reported here-> msysgit#318). Philip |
|
I gave it a try to compile it with VS2013. But failed so far due to missing knowledge in that area. |
dscho
added this to the First release of Git for Windows generated using the new SDK milestone
|
I planed to invest some time mid next week to test it and get it merged. |
|
@PhilipOakley
|
|
second try: any idea with these error? |
|
BTW, these 2 patterns could be also added into .gitignore if using VS2013. |
|
@YueLinHo Thanks for the feedback. Yes the common-cmds.h should be build first (a pre-requisite of a regular build).
Yes, the command is 'msvc-build --vs', which should down load a couple of directories/repos if missing (as per the README), and then internally do a make-dryrun to create/generate the .sln and .vcproj files, which is where errors (hence patches) mainly were. Finally it then does the little command dance to start up the VS20xx version. I have had issues if I accidentally built the patch series on top of Junio's git/git because the setting weren't right (it still thought 'Ah Linux', rather than 'Oh Windows'). On the SafeSEH issue, that is one I haven't been able to test, and the VS GUI is incomplete in that it has two locations for SafeSEH, (a) the old option flag, and (b) the new xml flag which the gui displays. For backward compatibility I used the former, and to use it you should add SAFESEH=YesPlease to the script invocation, so that it is passed down the environments to the make-dry, and finally picked up as an integral part of the perl generation of the .sln & .vcproj files. Don't know about the VS2013 issues yet. I've just received a new PC (Win 7 pro / 64-bit) for the house which I'm in the process of building and installing the family on... (sloow). hope that gives a bit of help and background. |
|
@PhilipOakley Thank you. :) |
|
@YueLinHo - I've done my testing on the 'bash' window, rather than using the cmd.exe version (i.e. when I install, I say I'll keep them 'separate'), so there may be something there that needs fixed. |
|
Yeah, I would definitely first try with Git Bash. In general, I am not sure whether it is worth fixing the build in cmd if that is broken. |
|
@dscho the first step is to prove/disprove that the cmd.ex (and the bash) properly run the I'll keep chipping away at the issue. Thanks everyone for the feedback and support. |
| @@ -40,6 +41,7 @@ sub showUsage | |||
| # Parse command-line options | |||
| while (@ARGV) { | |||
| my $arg = shift @ARGV; | |||
| #print "Arg: $arg \n"; | |||
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
So I spent some hours trying to get this to work with Visual Studio 2013. Here is my report before I call it a night: quite a couple of compile errors. I was able to put a couple of very dirty work arounds in, along with some legitimate changes: dscho@821cdad |
| # if cl.exe does not exist, populate vsvars with the most recent Visual Studio path | ||
| type cl.exe 2> /dev/null || | ||
| vsvars="$(ls -t \ | ||
| "$PROGRAMFILES/Microsoft Visual Studio"*/Common7/Tools/vsvars32.bat | |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
FYI I am preparing the release right now, and as this issue is not really crucial from a user's point of view, I will take it out of the milestone. |
dscho
removed this from the First release of Git for Windows generated using the new SDK milestone
'make clean', or a 'git clean -dfx' will delete the PM stamp file, so it cannot be a direct target in such clean conditions, resulting in an error. Normally the PM.stamp is recreated by the git/Makefile, except when a dry-run is requested, for example, as used in the msysgit msvc-build script which implements the compat/vcbuild/README using contrib/buildsystems. The script msvc-build is introduced later in this series. Protect the PM.stamp target when the PM.stamp file does not exist, allowing a Git 'Makefile -n' to succeed on a clean repo. Signed-off-by: Philip Oakley <[email protected]> --- This is development of the original "[PATCH 4/17] Makefile: a dry-run can error out if no perl. Document the issue" 2015-06-25, (http://marc.info/?l=git&m=143519054716960&w=2), which simply documented the issue and then used NO_PERL to avoid the problem. See follow on email thread for some discussion.
The change to the quoting of the -I and -L options in the config.mak.uname has worked for the dry-run output, and partially works for the vcproj generate step. I included adding a space after the option and before the dq quotation mark for the space-in-filename-paths. The -L options in the generated vcproj are still wrong (lacking the quotes), so that's next. |
|
I'm getting suspicious that the -L flags such as -L"$(MSVC_SDK10)/Include/10.0.10240.0/ucrt" should be assigned into the LDFLAGS, not the CFLAGS. e.g. https://github.com/dscho/git/blob/master/Makefile#L1016 Though it's more of a guess based on local code convention. ... Yay, appears to get rid of the errors I was seeing in the 'additional options' of the command line properties. Now (tomorrow) to sort the 'struct timespec' ifndef definition |
|
-L specifies a linker library path, it us useful on CFLAGS only when cl.exe |
|
Yes, traditionally a "-L" arg should be in LDFLAGS. The script in "compat/vcbuild/scripts/clink.pl" just gets the full Unix cc/gcc command line and transforms it into a cl.exe or link.exe command line (after all the shell variables have been expanded). So when building with the Makefile it doesn't matter. If this is causing an issue for the generator, you could change it. |
Thanks. I have now done the change (locally) and it now works as per the conversion script. It's useful to have that subtle difference confirmed. |
|
Just a thought. If you're building with vs2008, you won't have the Windows Kits directories -- I think they are new with vs2015. You might open vs2008 developer command prompt and print the environment variables that the vcvars*.bat script sets up and compare that with the ones I put in the config.mak.uname. (I had to set all of them explicitly so that the Makefile would have them.) |
|
While -L should be a library path, the example from a few messages ago is not actually a library path, it is an include file path: "$(MSVC_SDK10)/Include/10.0.10240.0/ucrt" Therefore it should be going into a -I argument in CFLAGS. The Windows 10 SDK directory (default location, c:\Program Files (x86)\Windows Kits\10) contains Include and Lib directories. Those then contain variously numbered version directories Mine contains these three: 10.0.10150.0 However only the 14393 directory on my machine has the full set of header files. It's all quite messy, but if we can make git build either with the most recent Windows 10 SDK (currently 14393) or with all versions, that would be best. |
|
@PhilipOakley As of v2.11.0.windows.1, the Visual Studio 2015 support is included, and one of my continuous build jobs publishes You will notice that both @jeffhostetler and I relied heavily on your work to get there, thank you so much! Now, unfortunately there were a couple of incompatible changes, in particular with our move toward using |
|
Hi @dscho, I had made some progress, but as usual was lagging behind the pack.
I have a commit for fetching the nuget packages in the V2 format
I found the version of .Net that allowed using nuget on XP SP3.
I fixed one of @jeffhostetler bits of good work that needed the variables defined at the start of a block (c89 style)
I'd made some progress on the dropping of repo.or.cz in the msvc-build scripts but it was incomplete. I think I may pivot on your script to pick its solutions for the various library replacements - I just need that little bit of clear time ;-)
I hope to rebase my old PR (you had already taken some of the commits...) in the next few days.
I have just seen that you say 8b018c7 that you have recreated some of the compatibility code for XP (..SP3), which is great news for me. I had seen a note that the GetTickCount64() was in XP SP3, but that may have been a dev who had something extra installed that brought it in!
That's my status
Thank you for all your good work.
Philip
…----- Original Message -----
From: Johannes Schindelin
To: git-for-windows/git
Cc: Philip Oakley ; Mention
Sent: Thursday, December 22, 2016 9:45 AM
Subject: Re: [git-for-windows/git] Fix the msvc-build scripts (specifically those creating VS projects) for g4w-sdk / g4w (#256)
@PhilipOakley As of v2.11.0.windows.1, the Visual Studio 2015 support is included, and one of my continuous build jobs publishes vs/master that is generated whenever master changes.
You will notice that both @jeffhostetler and I relied heavily on your work to get there, thank you so much!
Now, unfortunately there were a couple of incompatible changes, in particular with our move toward using nuget.exe to retrieve the dependencies instead of relying on the (outdated) binaries on repo.or.cz. Therefore, it will require a little work to get this here PR rebased on top of the current master. Do you want me to give you a hand, or do you think you got everything you need?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
@PhilipOakley awesome! FWIW I obtained access to a Windows-7 VM with VS2008 and came up with a couple of patches myself: http://github.com/dscho/git/compare/git-for-windows:master...vs2008 It is far from complete, of course, I basically only got libgit to build (but nothing was linked yet), but that branch may have a couple of gold nuggets for you. Sorry for the latest commit (a "Work In Progress" one, smooshing together all kinds of changes I had not yet committed, to be split and weeded out), I just ran out of time before the holidays.
Right. I think I recreated those: 39f9089 and 35ebda5, right? Compiling with DEVELOPER=1 under GCC did not catch those issues, of course, as that code does not get compiled with GCC ;-)
I look forward to that!
Good. I hoped it would make your life easier.
The same goes for you! Thank you for keeping the ball rolling, and also for keeping an eye out for issues on the Git mailing list as well as helping to address the ever-increasing flood of user requests and tickets. Happy holidays and a happy new year! |
|
Hi @dscho, I have rebased my dscho-msvc branch on top of your VS2008. The one big problem for compilation on XP that I've see in wrting up one of the TODO commits is that the necessary NuGet version (2.7.4) that support the 'restore' command has been removed from the NuGet.org downloads, so it is no linger possible for folks on XP to bridge that gap (a .Net version, a NuGet version, and 'restore'). I have a copy of NuGet V2.7.4.. For the XP/VS2008 version I still have to sort out the correct install of the WDK that holds 'NtQueryObject' symbol - i.e find the location so that it is automatically picked up by VS2008, rather than it being a manual user setup. Also I've yet to work through your vcxproj.pm code that does the right things with libiconv, so I can get that into the old vcproj.pm Overall the project now builds with only the "fatal error LNK1181: cannot open input file 'libiconv.lib'", though the is obviously an imprortant warning ;-) |
|
That is great progress! Just so I understand: newer As to And yes, |
|
From: Johannes Schindelin ***@***.***
Sent: 27 December 2016 20:35
Sorry for the delay. I've had man-flu (head cold).
Just so I understand: newer nuget.exes do not support XP, or is there a different problem with them?
Correct. The last version that supported XP, and supported the 'restore' command, was 2.7.4. And it has disappeared from the NuGet repository, making life difficult for any others wanting to follow along.
As to NtQueryObject(), I was under the impression that this function had to be accessed manually on XP, i.e. via DECLARE_PROC_ADDR/INIT_PROC_ADDR.
I had it in my head that this was a hidden capability that required the WDK (Windows Driver Kit?) to be installed, so that the symbol became accessible. Though I may be thinking of the wrong function.
It certainly now appears that the manual linkage (i.e. the user is to add it in the VS2008 project after it has been generated by the code) is the way to go.
And yes, libiconv.lib is rather important. The real question is whether the NuGet package contains a version that is compatible with VS2008...
I agree about the importance. The two issues I was concerned about were:
Ensuring that the script converted the lib name correctly when processing the dry-run output - you have some code in the VS2015 script that does some fancy stuff.
Second, in my install, the nugget download is named iconv (rather than libiconv), so I wasn't sure if I had done stuff right. At this stage, I haven't pointed the project solution at the download (in build-extras), so it won't be seen anyway.
…--
Philip
|
Don't worry. Was down with a serious cold myself.
Hrm. But we actually do not need curl -O https://api.nuget.org/packages/openssl.v120.windesktop.msvcstl.dyn.rt-dyn.1.0.2.1.nupkgJust one quirk there: the The thing to keep in mind after that is that I just hacked up a script that should work (only very lightly tested here): #!/bin/sh
die () {
echo "$*" >&2
exit 1
}
for package in $(sed -n 's/.*id="\([^"]*\)" version="\([^"]*\).*/\1.\2/p' \
< compat/vcbuild/packages.config)
do
dir=compat/vcbuild/GEN.PKGS/$package
nupkg=$dir/$package.nupkg
test $nupkg ||
curl -o $nupkg https://api.nuget.org/packages/$package.nupkg ||
curl -o $nupkg https://api.nuget.org/packages/${package%.0}.nupkg ||
die "Could not download $nupkg"
test -f $dir/_rels/.rels ||
(cd $dir && unzip $package.nupkg) ||
die "Could not unpack $nupkg"
doneIf you run that in the top-level directory, it should populate the
Yep. But you can copy that stuff back from
I do see an |
|
@dscho, That info about the Nuget pacakges and the workaround on how to decode them is very useful. It means I can stop worrying about that part. For my own download of the NuGet pakages (back in late October!) I had used the V2 api portal (covered by a small patch). In terms of the I saw in my However, after your note, a bit more looking showed (in the GEN.PKGS) that I have: And the same for the redistributables So I do have the libiconv - I just wasn't looking in the right place, and haven't yet setup the .sln to link to them. Shaking of the head/chest cold has taken longer than I'd hoped - the next step (wark aside) is to repivot your code to get the linkages automatically correct, and then fingers crossed... As an aside, excusing my ignorance, what is the difference between V110 and V100 ? |
|
v100 and v110 refer to the version of Visual a Studio. The following link is unrelated to the discussion here, but talks a little about the versions: |
Ouch. That may very well mean that there is no support for VS2008 in any of the NuGet packages? |
I was going to guess that the difference (beyond the VS versions) was the difference in the 'C' Run Time (CRT) that has been mentioned else where, and which one (of the CRT's) that each VS version auto linked to. Otherwise, why do the different VS versions need different libraries - I would have thought that the library format was version independent.. |
|
You can mix libraries from different VS versions. This is what I remember, might be incomplete:
General these things must be the same setting accross all libs and own code to link successfully
|
|
Is support for VS 2008 really desirable? That's a really old compiler,
supporting it means foregoing lots of modern C++ features. Given that the
community version of VS 2015 is available for free is there really a
compelling reason to support VS 2008? VS 2017 will be coming out this year
- that's the fifth major release since VS 2008.
I also wonder about why so much effort is being spent on supporting Windows
XP. This is an unsupported and unpatched OS. Are there really a lot of
developers who want to run the latest version of git on such an old version
of Windows? Setting the minimum supported OS to Windows 7 simplifies many
things and shouldn't lock out many developers, I wouldn't think.
You can mix libraries from different VS versions.
Static libraries (.lib files) built with different VS versions cannot be
mixed, at least not without extreme care. This is explicitly checked for
and prohibited in recent versions of VS. The CRT structures change between
versions so if you wanted to try working around these prohibitions you
would need to make sure that your library used none or at least very few
CRT functions. I can track down references for my claim if desired, but
I've certainly hit issues with mixing VS versions enough times to be sure.
…On Wed, Jan 4, 2017 at 6:02 AM, Mike Friedrich ***@***.***> wrote:
You can mix libraries from different VS versions.
Appartements from higher memory footprint, you will have to make sure a
few things.
This is what I remember, might be incomplete:
- new and delete on an object is performed by the same runtime
- avoid STL use in shared headers
General these things must be the same setting accross all libs and own
code to link successfully
- character encoding setting (utf-16 vs ANSI vs none) and macro
- static/dynamic crt linkage
- debug/release settings and macros
- 32/64 bitness obviously
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#256 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AKTLqYixZtw_gFmLhByHDDQ7ax12dRa4ks5rO6aKgaJpZM4FfouR>
.
--
Bruce Dawson
|
|
Windows Embedded Standard 2009 still gets patched (end of license is 2024), that is still an XP. |
|
Hi Bruce,
The (my) XP / VS2008 thing is part of the joys of open source..;-)
I have a little XP netbook, and access to (full dev) VS2008 (My work does 30 year support stuff, so 2008 is, for some projects, quite new).
I agree that there is lots of newer stuff and that the main GfW project needs to support that, but underneath, we still use GCC anyway as the reference compiler for the C code (i.e. no fancy C++ stuff), with a -C89 coding standard as the guidance (VS2008 is C89), so there are lots of tradeoffs.
The original code that I'm modifying and "maintaining" was developed for VS2008, so as long as there are no true show stoppers, I'm still plodding on with seeing if I can get it going.
One other part *you could have a look at* is making sure that the VS2015 build scripts @dscho created actually work and install as you would expect. #256 (comment) and master...dscho:msvc
Philip
[Occassionaly I get access to the W7 m/c my wife uses, and my work m/c is W7, but in a secured environment, so no open source dev there, hence the XP fall back!]
----- Original Message -----
From: Bruce Dawson
To: git-for-windows/git
Cc: Philip Oakley ; Mention
Sent: Wednesday, January 04, 2017 5:05 PM
Subject: Re: [git-for-windows/git] Fix the msvc-build scripts (specifically those creating VS projects) for g4w-sdk / g4w (#256)
Is support for VS 2008 really desirable? That's a really old compiler,
supporting it means foregoing lots of modern C++ features. Given that the
community version of VS 2015 is available for free is there really a
compelling reason to support VS 2008? VS 2017 will be coming out this year
- that's the fifth major release since VS 2008.
I also wonder about why so much effort is being spent on supporting Windows
XP. This is an unsupported and unpatched OS. Are there really a lot of
developers who want to run the latest version of git on such an old version
of Windows? Setting the minimum supported OS to Windows 7 simplifies many
things and shouldn't lock out many developers, I wouldn't think.
You can mix libraries from different VS versions.
Static libraries (.lib files) built with different VS versions cannot be
mixed, at least not without extreme care. This is explicitly checked for
and prohibited in recent versions of VS. The CRT structures change between
versions so if you wanted to try working around these prohibitions you
would need to make sure that your library used none or at least very few
CRT functions. I can track down references for my claim if desired, but
I've certainly hit issues with mixing VS versions enough times to be sure.
On Wed, Jan 4, 2017 at 6:02 AM, Mike Friedrich <[email protected]>
wrote:
You can mix libraries from different VS versions.
Appartements from higher memory footprint, you will have to make sure a
few things.
This is what I remember, might be incomplete:
- new and delete on an object is performed by the same runtime
- avoid STL use in shared headers
General these things must be the same setting accross all libs and own
code to link successfully
- character encoding setting (utf-16 vs ANSI vs none) and macro
- static/dynamic crt linkage
- debug/release settings and macros
- 32/64 bitness obviously
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#256 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AKTLqYixZtw_gFmLhByHDDQ7ax12dRa4ks5rO6aKgaJpZM4FfouR>
.
…--
Bruce Dawson
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
Hi @randomascii ! Thank you for sharing your thoughts with us. I am sure you have thought things through and decided for good reasons that you do not want to work on VS 2008 support. Others, such as @PhilipOakley have equally good reasons to do work on VS 2008/XP support. I welcome all contributions, whether they target obsolete or not so obsolete platforms, as long as they do not increase my maintenance load (which they don't, I simply do not work too much on VS 2008 and focus exclusively on VS 2015 myself). Hopefully that clarifies the situation? |
|
@PhilipOakley how about a force-push with your current state? |
|
@PhilipOakley how about a force-push with your current state?
I'll see what I can do. I had a bit of an overload moment back when I was trying to revise the script, so put it to one side.
I'll add the split files and a big commit note as to where I was, but it may be a day or so to get some clear space.
Philip
|
No worries! It will be good to have the work-in-progress state in the open, though, so that other developers can possibly step in and try to come up with improvements. |
|
I have force pushed my status. I included a top WIP commit with a summary. The branch was rebased on to of @dscho 's repo. There are two independant things I was hooked up on.
point 2 is not essential for VS2008, it only matters for those on older platforms that can't handle the new VS versions. |
|
@PhilipOakley do you want to keep this ticket open? |
|
Let's close it. I know where it is. I can always re-open it if I manage to get the time to address this, (and I'm still on the same hardware ;-) |
This patch series fixes the msvc-build script and associated files in the compat and contrib directories to allow us Windows folks to use that most popular of IDEs, Visual Studio (despite it being from Microsoft...). The Git upstream (Junio) has asked that it be well tested here before it he could take it into his tree as he has no way of testing.
Reports are welcomed. Testing of the SAFESEH (top patch) option would be useful as I have no test system (at the moment) that brings out the conditions that need that case.