DNX451 support

[adamsitnik]

Introduction to DNX

I start with some short introduction since the idea of Cross-Platform .NET is quite fresh.

DNX stands for Dot Net eXecution environment. It lets us restore Nuget packages, compile the code and execute it with native CLR Host. Some good introduction can be found here. All that aims at cross-platform .NET. A very modularized .NET to be honest!

DNX451 is a DNX SDK running on .Net 4.5.1 (Desktop CLR / Full BCL and FCL). So it is not Cross-Platform yet, which NETCORE aka DNXCORE50 is.

To me DNX451 support is just a huge step forward for NETCORE support, which it the cool thing we aim for ;)

Changes in .NET applications development:

• We need DNX toolchain to be installed link. It contains of . NET Version Manager (DNVM) and DNX. Once installed, we can get our hands on latest runtime with simple „dnvm upgrade” command. „dnvm update-self” is also worth to mention.

•  We need .NET Core command-line (CLI) [link](https://github.com/dotnet/cli) to be installed.
  

• .csproj and package.config files have been replaced with .xproj and project.json files. .xproj automatically references all .cs files so you don’t have to update it with every new class/interface/enum added (number of git conflicts has just dropped) Project.json allows us to target multiple frameworks with one file and manage all dependencies in single place. Simplicity over complexity! Project.json.lock tells the compiler exactly where to look for our dependencies. You can produce it with „dnu dotnet restore”. Sometimes VS will do this for you (currently VS is still using dnu), sometimes you will have to do this on your own. Dnu is just an alias for a part of Dnx.
• Forget about .exe files, now you get .dll files with entrypoints that can be nonstatic and async methods! Moreover VS does not produce output files by default, it preffers to do this in memory. But of course there is an option in VS to get these files. Alt + F4 -> Build tab -> „Produce output on build” checkbox
• To start program you need to start it with „dnx dotnet run”. Run is just a command in your project.json file.
• Program’s startup seems to be longer to me now. If you set DNX_TRACE to 1 then you get detailed status updates from DNX. That explains a lot! (I created dedicated VS profile for BDN for this to make troubleshooting easier)
• All applications put nuget packages to the same folder now. It is „%USERPROFILE%/.dnx/packages”. MS plans to replace „dnx” with „nuget” in this path
• Assembly.Location is just an empty string now. That is because the way of loading assemblies has changed.
• In NECORE some of the APIs has changed. To make System.Object independent from System.Reflection, Object.GetType() returns a simple object that have only ONE field: Name. If you want to get more, you need to call other method: Object.GetType().GetTypeInfo()
• & more. Yet to discover, at least for me.

Failed PoC:

• I tried to use Nuget.Client v3 to get the dnu restore working, but it did not create project.json file I needed.
• I tried to use Microsoft.Dnx.Compilation to compile, but I have failed to make it work. Dnx tool itself is using this, but it does so much configuration that I was unable to reproduce it.
• I tried to use „dnu publish” to produce the dll first, and then run it with dnx run but it had one huge disadvantage: it was generating a lot of nested folders, which names I could not control, which caused a lot of „path too long exceptions”. I had to keep my benchmarks very close to drive’s root, like C:\bdn.

• I succeded with dnx toolchain and it got retired by Microsoft. I had to switch to dotnet cli which also reduced the number of tricky things!
  

• I got FSharp samples running with dnx, but it stopped working after moving to dotnet cli. It is just not fully supported yet. 
  

• I tried to read process output in async way but it did not worked for diagnosers (process needs to be alive to attach to it)
  

How it works

I have built a new toolchain that relies on the classic one. In order to reuse as much code as possible, most of the new classes just inherit from the existing ones.

• CHANGE: Instead of generating .csproj file with references to .dlls, new project.json file that defines the dependencies is being generated.
• NEW: To get all required dependencies dnu dotnet restore command is being executed. It is just a new „nuget restore”. It produces project.lock.json file.

•  CHANGE: Instead of direct call to msbuild, dotnet build command is being executed.
  

• CHANGE: Instead of compiling code and emitting assembly and then running it, a dnx run command is being executed. It does all that with single command and in memory (no .dll file). The disadvantage of doing it in memory is that we have to compile it with each iteration.
• CHANGE: I have changed the way how console output is being read. Instead of while(output.ReadLine() != null) I used events. I have also added reading of error output. This might give us more detailed informations on errors.

•   CHANGE: To run "classic" tests build the solution and run runClassicTests.cmd in the root directory. I had to make it this way because currently VS/xUnit does not support running dnx and non-dnx tests
  

Tricky things

• To get „dnu dotnet restore” working when executing in VS/xUnit runner I had to put the auto-generated project.json file closer to VS solution folder root (DnxGenerator. GetDirectoryPath).
• The other thing to get it running was to switch between dependency to BenchmarkDotNet project and package. When developing, we want our auto-generated program to get code from project, but when we are at production, then we want to it from the official nuget( BenchmarkDnxGenerator.GetTargetType()). I have relied it on the existing convention of AssemblyTitleAttribute ending with „-Dev”.

•   I had also to increment the package version, otherwise I could not test the production part, because the nuget was restoring existing version.
  

• To get „dnx run” working I had to define the compiler in project.json in explicit way (roslyn). When it was using the default compiler (DesignTime) it had differnt configuration and was unable to resolve dependencies.
• To get the x86/x64 working I had to tell dnvm to use exact processor architecture (BenchmarkDnxExecutor. BuildParameters)
• To ensure the high priority and right affinity in scenario where instead of starting single process I start cmd.exe that starts dnx.exedotnet.exe that starts the program I set these at Program.Main of the autogenerated program. This is because HIGH_PRIORITY is not inherited by child processes in Windows link

How to test

Testing from VS/xUnit runner requires no configuration.
Testing as a standalone application requires more effort:

• Build solution in Release

•  Go to Visual Studio -> Options -> Nuget Package Manager -> Package Sources, add a new source: "artifacts\bin\BenchmarkDotNet\Release" folder
  

• Add this folder as a nuget’s local feed link
• Create new VS solution with new project „Console Application (package)” (or add it to your existing solution)
• Remove NETCORE target from it’s project.json
• Install BenchmarkDotNet.0.9.2 package from your local feed
• Write some benchmarks like you used to do before.
• Go to the project’s root folder and run "dotnet restore" and "dotnet build"
• Execute „dotnet run” and ENJOY

It all seems crazy, but faking the nuget repo needs to be done only now, for testing the solution before official nuget package release. And the whole dnu/dxn/dotnet thing is just the new way of executing .NET apps so the rest will be no suprise to the users.

How to debug:

• DNX451: simply press F5
• NET40: Configure VS to start executable on debugging, for our purposes I have already done this for Samples project
  

Please start testing, and do not hesitate to ask. I think that if you accept this this PR I will remove existing .csprojs and old solution. I can also squash the commits if you want. I left it this way just to keep the history.

[adamsitnik]

@AndreyAkinshin Sorry for this! I can see that dotnet cli has changed the paths. I will push a fix asap

[AndreyAkinshin]

@adamsitnik, don't worry, it is fine. After this fix, I will merge it and publish v0.9.2 (no new features, only DNX support).

[adamsitnik]

@AndreyAkinshin Thanks! But I can imagine that it was not the best impression if first sample has failed to execute ;)

There were few changes in dotnet cli that caused troubles:

• path extended with runtimeId, now I just set the output path manually to avoid such problems in future
• warnings caused the dotnet cli to have exit code != 0. If this happens, I check whether the build succeded by checking if the executable exists

[AndreyAkinshin]

I successfully run it a few weeks ago, so, don't worry. =)

However, we still have some troubles. Try to do the following things:

1. Build the solution
2. Copy BenchmarkDotNet.0.9.2.nupkg to the folder with your local NuGet packages.
3. Create new DNX project, remove dnxcore50 dependency, install BenchmarkDotNet from the local NuGet feed.
4. Use the Md5VsSha256 example from ReadMe and try to run it.

It doesn't work on my machine. =(

[adamsitnik]

@AndreyAkinshin what error are you getting? Does it fail to restore nuget package, to build the app or to run it?

I just tried to reproduce, and it worked. But I have faced some weird issues with caching of the old local nuget packages.

What I did:

1. Remove artifacts from solution's root (it seems like nukpg was not replaced if I did not change the version)
2. Build Solution in Release
3. Remove %USERPROFILE%.nuget\packages\BenchmarkDotNet\0.9.2 with old dll installed if exists.
4. Remove old files from local feed, Copy the new nupkg to Nuget local feed
5. Install BDN 0.9.2 from local feed in new project targeting dnx451
6. It worked in both ways (dotnet restore dotnet run and also dnu restore dnx run)

[AndreyAkinshin]

Here is my log:

// ***** BenchmarkRunner: Start   *****
// Found benchmarks:
//   Md5VsSha256_Md5
//   Md5VsSha256_Sha256

// **************************
// Benchmark: Md5VsSha256_Md5
// *** Generate ***
// Result = Success
// DirectoryPath = c:\users\jetbrains\documents\visual studio 2015\Projects\ConsoleApp2\src\ConsoleApp2\..\Md5VsSha256_Md5

// *** Build ***
c:\users\jetbrains\documents\visual studio 2015\Projects\ConsoleApp2\src\Md5VsSha256_Md5\Program.cs(47,31): warning CS0169: The field 'Program.value' is never used
// Result = Success

// *** Execute ***
// Launch: 1
// Launch: 2
System.InvalidOperationException: StatSummary: Sequence contains no elements
   at BenchmarkDotNet.Mathematics.Statistics..ctor(IEnumerable`1 values)
   at BenchmarkDotNet.Running.BenchmarkRunner.Execute(ILogger logger, Benchmark benchmark, IToolchain toolchain, BuildResult buildResult, IConfig config)
   at BenchmarkDotNet.Running.BenchmarkRunner.Run(Benchmark benchmark, ILogger logger, IConfig config)
   at BenchmarkDotNet.Running.BenchmarkRunner.Run(IList`1 benchmarks, ILogger logger, String title, IConfig config)
   at BenchmarkDotNet.Running.BenchmarkRunner.Run(IList`1 benchmarks, IConfig config)
   at BenchmarkDotNet.Running.BenchmarkRunner.Run[T](IConfig config)
   at ConsoleApp2.Program.Main(String[] args) in c:\users\jetbrains\documents\visual studio 2015\Projects\ConsoleApp2\src\ConsoleApp2\Program.cs:line 42
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Dnx.Runtime.Common.EntryPointExecutor.Execute(Assembly assembly, String[] args, IServiceProvider serviceProvider)
   at Microsoft.Dnx.ApplicationHost.Program.<>c__DisplayClass3_0.<ExecuteMain>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
Press any key to continue . . .

I looks that generate and build stages work fine: I have an executable Md5VsSha256_Md5.exe and it works. Maybe you can print some additional diagnostic information for DNX toolchain (e.g. full path for target exe file).

[adamsitnik]

Thanks for the log. I will implement some kind of debug/trace mode for dnx toolchain. We will probably need it for user bug reports.

Just one more question: is the executable located in "ConsoleApp2..\Md5VsSha256_Md5\binaries" folder?

[AndreyAkinshin]

Full path: C:\Users\jetbrains\Documents\Visual Studio 2015\Projects\ConsoleApp2\src\Md5VsSha256_Md5\bin\RELEASE\dnx451\win7-x64\Md5VsSha256_Md5.exe
Full path for master project: C:\Users\jetbrains\Documents\Visual Studio 2015\Projects\ConsoleApp2\src\ConsoleApp2\project.json

[adamsitnik]

With my latest changes the folder structure should be like this:

C:.
|   app.config
|   Program.cs
|   project.json
|   project.lock.json
+---bin
|   \---RELEASE
|       \---dnx451
|               Algo_Md5VsSha256_Md5.exe
|               Algo_Md5VsSha256_Md5.pdb
|               
+---binaries
|       Algo_Md5VsSha256_Md5.exe
|       Algo_Md5VsSha256_Md5.pdb
|       BenchmarkDotNet.dll
|       BenchmarkDotNet.pdb
|       BenchmarkDotNet.Samples.exe
|       BenchmarkDotNet.Samples.pdb
|       
\---obj
    \---RELEASE
        \---dnx451
                dotnet-compile-csc.rsp
                dotnet-compile.assemblyinfo.cs
                dotnet-compile.rsp

The exe from dnx451 folder should fail. The executable from binaries folder is the only one runnable. Do you have this folder? If not, then could you try to repeat the nuget-cleanup step from my previous post?

[AndreyAkinshin]

Here is my tree:

├───ConsoleApp2
│   │   ConsoleApp2.xproj
│   │   ConsoleApp2.xproj.user
│   │   Md5VsSha256.log
│   │   Program.cs
│   │   project.json
│   │   project.lock.json
│   │
│   ├───bin
│   │   ├───Debug
│   │   │   └───dnx451
│   │   │           ConsoleApp2.exe
│   │   │           ConsoleApp2.pdb
│   │   │
│   │   └───RELEASE
│   │       └───dnx451
│   │               ConsoleApp2.exe
│   │               ConsoleApp2.pdb
│   │
│   ├───obj
│   │   ├───Debug
│   │   │   └───dnx451
│   │   │           dotnet-compile-csc.rsp
│   │   │           dotnet-compile.assemblyinfo.cs
│   │   │           dotnet-compile.rsp
│   │   │
│   │   └───RELEASE
│   │       └───dnx451
│   │               dotnet-compile-csc.rsp
│   │               dotnet-compile.assemblyinfo.cs
│   │               dotnet-compile.rsp
│   │
│   └───Properties
│           AssemblyInfo.cs
│
└───Md5VsSha256_Md5
    │   app.config
    │   Program.cs
    │   project.json
    │   project.lock.json
    │
    ├───bin
    │   └───RELEASE
    │       └───dnx451
    │           │   Md5VsSha256_Md5.exe
    │           │   Md5VsSha256_Md5.pdb
    │           │
    │           └───win7-x64
    │                   BenchmarkDotNet.dll
    │                   ConsoleApp2.exe
    │                   ConsoleApp2.pdb
    │                   Md5VsSha256_Md5.exe
    │                   Md5VsSha256_Md5.exe.config
    │                   Md5VsSha256_Md5.pdb
    │
    └───obj
        └───RELEASE
            └───dnx451
                    dotnet-compile-csc.rsp
                    dotnet-compile.assemblyinfo.cs
                    dotnet-compile.rsp

I tried to remove BDN from %USERPROFILE%.nuget\packages\ but it didn't help.

[adamsitnik]

Thanks for the detailed info. Now I am sure, that 0.9.2 is somewhere cashed on your machine without my recent fixes, which affect output folder structure (new folder called binaries should appear).

The easiest test would be to change the version number locally, let's say 0.9.3, deploy it to local nuget feed and change your test app to use 0.9.3.

[AndreyAkinshin]

Yes, you are right! I changed package version and now everything works great. It is very interesting, where NuGet keeps local cache.

I want to play a little bit more with DNX version of BDN. If everything is fine, I will merge your branch tomorrow.

[adamsitnik]

Great! I am very happy to hear that!

Please let me know if there are any more bugs or unclear things.

[AndreyAkinshin]

Finally merged with master!
@adamsitnik, I updated settings, now you can push directly to the main repo without PRs. Can you merge master into develop?

[adamsitnik]

@AndreyAkinshin Awesome! Thank you for your trust in me! I have just merged master into develop and added unit test for GCDiagnosers to make sure it works with both DNX and CLASSIC.

[AndreyAkinshin]

Cool, thanks!
If you find any minor bugs in DNX support, feel free to push fixes directly in the develop branch.
How it's going with CoreCLR support?

[adamsitnik]

Ok, great!

As for the CoreCLR, I need 1-2 days for development. The only problem I have is that I am a total newbie with Unix based OS, so I am not sure how long testing will take for me.

[AndreyAkinshin]

My second OS is Linux, so, I can help you with testing. =) You can create a new coreclr branch directly in the main repo.

[adamsitnik]

Cool! I will do that. Do you @AndreyAkinshin or @mattwarren plan to do some huge refactoring in the upcoming week? My Core changes will affect mostly all calls to Reflection methods and project.json files.

[AndreyAkinshin]

I can wait.

[mattwarren]

same here

Or at least I'll make sure I only check-in code under the Diagnostics project, so it won't be a problem.