Intermittent "cannot access the file '...\project.csprojAssemblyReference.cache'" build errors

[jkotas]

Packaging All Configurations leg in CI started often failing with errors like:

D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Microsoft.Common.CurrentVersion.targets(2106,5): warning MSB3088: Could not read state file "obj\Debug\netstandard2.0\project.csprojAssemblyReference.cache". The process cannot access the file 'D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Numerics.Vectors\netstandard2.0\obj\Debug\netstandard2.0\project.csprojAssemblyReference.cache' because it is being used by another process. [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Numerics.Vectors\netstandard2.0\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: The "ResolvePackageAssets" task failed unexpectedly. [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: System.IO.IOException: The process cannot access the file 'D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\obj\Debug\netstandard1.7\project.assets.cache' because it is being used by another process. [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle) [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options) [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at System.IO.File.Open(String path, FileMode mode, FileAccess access, FileShare share) [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at Microsoft.NET.Build.Tasks.ResolvePackageAssets.CacheWriter..ctor(ResolvePackageAssets task) [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at Microsoft.NET.Build.Tasks.ResolvePackageAssets.CacheReader..ctor(ResolvePackageAssets task) [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at Microsoft.NET.Build.Tasks.ResolvePackageAssets.ReadItemGroups() [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at Microsoft.NET.Build.Tasks.ResolvePackageAssets.ExecuteCore() [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at Microsoft.NET.Build.Tasks.TaskBase.Execute() [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\tools\sdk\2.1.300-preview2-008530\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [D:\j\workspace\windows-TGrou---0d2c9ac4\bin\testPkg\projects\System.Composition.TypedParts\netstandard1.7\project.csproj] [D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj]
D:\j\workspace\windows-TGrou---0d2c9ac4\pkg\test\testPackages.proj(177,5): error MSB3073: The command "D:\j\workspace\windows-TGrou---0d2c9ac4\bin/testPkg/tools//dotnet msbuild /t:Test "D:\j\workspace\windows-TGrou---0d2c9ac4\bin/testPkg/test.msbuild" /p:TestPackages=System.Composition.TypedParts" exited with code 1.
    1 Warning(s)
    15 Error(s)

[jkotas]

dotnet/corefx#29337 : https://ci3.dot.net/job/dotnet_corefx/job/master/job/windows-TGroup_all+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/11584/consoleText

[jkotas]

dotnet/corefx#29319 : https://ci3.dot.net/job/dotnet_corefx/job/master/job/windows-TGroup_all+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/11591/consoleText

[jkotas]

@ericstj Can this be related to your recent build tools changes?

[ericstj]

Not my buildtools change but could be race in packaging tests. I will look into it

[ericstj]

There are a combination of issues here. Some have always been present some were brought in with the new CLI: dotnet/corefx@bf1b082.

I'll try to address all of them.