Re-enable parallel platform execution in CI build (#231)

In a previous commit (b4439f4), the three platforms in the CI pipeline (Windows, Linux, and Mac) were modified to execute serially since they were all operating against the same shared test account, which caused the Pester tests to operate unstabily.

I've now created multiple test accounts so that that each platform can operate in isolation of each other.  I've updated the UT template to accept parameter input, and modified the CI (and release) pipeline to reference a different AccessToken/OwnerName/OrganizationName based on which platform is being targeted.

Additionally:

 * Had to fix some tests to keep everything passing as well;
    * Two Contents tests started failing again because GitHub once again changed the HTML output for the standard generated README.md file.  This time around, I've attempted to make the tests more robust to future changes.
   * Two RepositoryForks tests were unreliable in the way that they were written, so they were updated as well to be more robust when being executed in a parallel UT environment (since we can't control when a fork for this repository is being created/removed).
 * Updates the pipelines to always use the most recent OS images.
 * Fixes some invalid links on some of the Azure DevOps badges in the README

Author: HowardWolosky

README.md

@@ -3,14 +3,14 @@
 [![[GitHub version]](https://badge.fury.io/gh/microsoft%2FPowerShellForGitHub.svg)](https://github.com/microsoft/PowerShellForGitHub/releases)
 [![powershellgallery](https://img.shields.io/powershellgallery/v/PowerShellForGitHub)](https://www.powershellgallery.com/packages/PowerShellForGitHub)
 [![downloads](https://img.shields.io/powershellgallery/dt/PowerShellForGitHub.svg?label=downloads)](https://www.powershellgallery.com/packages/PowerShellForGitHub)
-![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/microsoft/PowerShellForGitHub)
+[![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/microsoft/PowerShellForGitHub)](https://github.com/microsoft/PowerShellForGitHub)
 [![downloads](https://img.shields.io/badge/license-MIT-green)](https://github.com/HowardWolosky/PowerShellForGitHub/blob/master/LICENSE)
 [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3990/badge)](https://bestpractices.coreinfrastructure.org/projects/3990)
 [![tweet](https://img.shields.io/twitter/url?url=https%3A%2F%2Ftwitter.com%2FQuackFu)](https://twitter.com/intent/tweet?text=%23PowerShellForGitHub%20%40QuackFu%20&original_referer=https://github.com/microsoft/PowerShellForGitHub)
 <br />
-[![Build status](https://dev.azure.com/ms/PowerShellForGitHub/_apis/build/status/PowerShellForGitHub-CI?branchName=master)](https://dev.azure.com/ms/PowerShellForGitHub/_build/latest?definitionId=109&branchName=master)
-[![Azure DevOps tests](https://img.shields.io/azure-devops/tests/ms/PowerShellForGitHub/109/master)](https://dev.azure.com/ms/PowerShellForGitHub/_build/latest?definitionId=109&branchName=master)
-[![Azure DevOps coverage](https://img.shields.io/azure-devops/coverage/ms/PowerShellForGitHub/109/master)](https://dev.azure.com/ms/PowerShellForGitHub/_build/latest?definitionId=109&branchName=master)
+[![Build status](https://dev.azure.com/ms/PowerShellForGitHub/_apis/build/status/PowerShellForGitHub-CI?branchName=master)](https://dev.azure.com/ms/PowerShellForGitHub/_build?definitionId=109&_a=summary&repositoryFilter=63&branchFilter=2197)
+[![Azure DevOps tests](https://img.shields.io/azure-devops/tests/ms/PowerShellForGitHub/109/master)](https://dev.azure.com/ms/PowerShellForGitHub/_build?definitionId=109&_a=summary&repositoryFilter=63&branchFilter=2197)
+[![Azure DevOps coverage](https://img.shields.io/azure-devops/coverage/ms/PowerShellForGitHub/109/master)](https://dev.azure.com/ms/PowerShellForGitHub/_build?definitionId=109&_a=summary&repositoryFilter=63&branchFilter=2197)
 <br />
 [![Help Wanted Issues](https://img.shields.io/github/issues/microsoft/PowerShellForGitHub/help%20wanted)](https://github.com/microsoft/PowerShellForGitHub/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22)
 [![GitHub last commit](https://img.shields.io/github/last-commit/microsoft/PowerShellForGitHub)](https://github.com/HowardWolosky/PowerShellForGitHub/commits/master)

Tests/GitHubContents.tests.ps1

@@ -22,7 +22,7 @@ try
 
     # Need two separate blocks to set constants because we need to reference a constant from the first block in this block.
     @{
-        htmlOutput = "<div id=`"file`" class=`"md`" data-path=`"README.md`"><article class=`"markdown-body entry-content container-lg`" itemprop=`"text`"><h1><a id=`"user-content-$($repoGuid.ToLower())`" class=`"anchor`" aria-hidden=`"true`" href=`"#$($repoGuid.ToLower())`"><svg class=`"octicon octicon-link`" viewBox=`"0 0 16 16`" version=`"1.1`" width=`"16`" height=`"16`" aria-hidden=`"true`"><path fill-rule=`"evenodd`" clip-rule=`"evenodd`" d=`"M7.775 3.275C7.64252 3.41717 7.57039 3.60522 7.57382 3.79952C7.57725 3.99382 7.65596 4.1792 7.79337 4.31662C7.93079 4.45403 8.11617 4.53274 8.31047 4.53617C8.50477 4.5396 8.69282 4.46748 8.835 4.335L10.085 3.085C10.2708 2.89918 10.4914 2.75177 10.7342 2.65121C10.977 2.55064 11.2372 2.49888 11.5 2.49888C11.7628 2.49888 12.023 2.55064 12.2658 2.65121C12.5086 2.75177 12.7292 2.89918 12.915 3.085C13.1008 3.27082 13.2482 3.49142 13.3488 3.7342C13.4493 3.97699 13.5011 4.23721 13.5011 4.5C13.5011 4.76279 13.4493 5.023 13.3488 5.26579C13.2482 5.50857 13.1008 5.72917 12.915 5.915L10.415 8.415C10.2292 8.60095 10.0087 8.74847 9.76588 8.84911C9.52308 8.94976 9.26283 9.00157 9 9.00157C8.73716 9.00157 8.47691 8.94976 8.23411 8.84911C7.99132 8.74847 7.77074 8.60095 7.585 8.415C7.44282 8.28252 7.25477 8.21039 7.06047 8.21382C6.86617 8.21725 6.68079 8.29596 6.54337 8.43337C6.40596 8.57079 6.32725 8.75617 6.32382 8.95047C6.32039 9.14477 6.39252 9.33282 6.525 9.475C6.85001 9.80004 7.23586 10.0579 7.66052 10.2338C8.08518 10.4097 8.54034 10.5002 9 10.5002C9.45965 10.5002 9.91481 10.4097 10.3395 10.2338C10.7641 10.0579 11.15 9.80004 11.475 9.475L13.975 6.975C14.6314 6.31858 15.0002 5.4283 15.0002 4.5C15.0002 3.57169 14.6314 2.68141 13.975 2.025C13.3186 1.36858 12.4283 0.999817 11.5 0.999817C10.5717 0.999817 9.68141 1.36858 9.02499 2.025L7.775 3.275ZM3.085 12.915C2.89904 12.7292 2.75152 12.5087 2.65088 12.2659C2.55023 12.0231 2.49842 11.7628 2.49842 11.5C2.49842 11.2372 2.55023 10.9769 2.65088 10.7341C2.75152 10.4913 2.89904 10.2707 3.085 10.085L5.585 7.585C5.77074 7.39904 5.99132 7.25152 6.23411 7.15088C6.47691 7.05023 6.73716 6.99842 7 6.99842C7.26283 6.99842 7.52308 7.05023 7.76588 7.15088C8.00867 7.25152 8.22925 7.39904 8.415 7.585C8.55717 7.71748 8.74522 7.7896 8.93952 7.78617C9.13382 7.78274 9.3192 7.70403 9.45662 7.56662C9.59403 7.4292 9.67274 7.24382 9.67617 7.04952C9.6796 6.85522 9.60748 6.66717 9.475 6.525C9.14999 6.19995 8.76413 5.94211 8.33947 5.7662C7.91481 5.59029 7.45965 5.49974 7 5.49974C6.54034 5.49974 6.08518 5.59029 5.66052 5.7662C5.23586 5.94211 4.85001 6.19995 4.525 6.525L2.025 9.02499C1.36858 9.68141 0.999817 10.5717 0.999817 11.5C0.999817 12.4283 1.36858 13.3186 2.025 13.975C2.68141 14.6314 3.57169 15.0002 4.5 15.0002C5.4283 15.0002 6.31858 14.6314 6.975 13.975L8.225 12.725C8.35748 12.5828 8.4296 12.3948 8.42617 12.2005C8.42274 12.0062 8.34403 11.8208 8.20662 11.6834C8.0692 11.546 7.88382 11.4672 7.68952 11.4638C7.49522 11.4604 7.30717 11.5325 7.165 11.665L5.915 12.915C5.72925 13.1009 5.50867 13.2485 5.26588 13.3491C5.02308 13.4498 4.76283 13.5016 4.5 13.5016C4.23716 13.5016 3.97691 13.4498 3.73411 13.3491C3.49132 13.2485 3.27074 13.1009 3.085 12.915Z`"></path></svg></a>$repoGuid</h1></article></div>"
+        htmlOutputStart = '<div id="file" class="md" data-path="README.md">'
         rawOutput = "# $repoGuid"
     }.GetEnumerator() | ForEach-Object {
         Set-Variable -Force -Scope Script -Option ReadOnly -Visibility Private -Name $_.Key -Value $_.Value
@@ -100,19 +100,27 @@ try
             $readmeFileBytes = Get-GitHubContent -OwnerName $script:ownerName -RepositoryName $repo.name -Path $readmeFileName -MediaType Html
             $readmeFileString = [System.Text.Encoding]::UTF8.GetString($readmeFileBytes)
 
+            # Replace newlines with empty for comparison purposes
+            $readmeNoBreaks = $readmeFileString.Replace("`n", "").Replace("`r", "")
             It "Should have the expected content" {
-                # Replace newlines with empty for comparison
-                $readmeFileString.Replace("`n", "").Replace("`r", "") | Should be $htmlOutput
+                # GitHub changes the syntax for this file too frequently, so we'll just do some
+                # partial matches to make sure we're getting HTML output for the right repo.
+                $readmeNoBreaks.StartsWith($htmlOutputStart) | Should -BeTrue
+                $readmeNoBreaks.IndexOf($repoGuid) | Should -BeGreaterOrEqual 0
             }
         }
 
         Context 'For getting html (string) file contents' {
 
             $readmeFileString = Get-GitHubContent -OwnerName $script:ownerName -RepositoryName $repo.name -Path $readmeFileName -MediaType Html -ResultAsString
 
+            # Replace newlines with empty for comparison purposes
+            $readmeNoBreaks = $readmeFileString.Replace("`n", "").Replace("`r", "")
             It "Should have the expected content" {
-                # Replace newlines with empty for comparison
-                $readmeFileString.Replace("`n", "").Replace("`r", "") | Should be $htmlOutput
+                # GitHub changes the syntax for this file too frequently, so we'll just do some
+                # partial matches to make sure we're getting HTML output for the right repo.
+                $readmeNoBreaks.StartsWith($htmlOutputStart) | Should -BeTrue
+                $readmeNoBreaks.IndexOf($repoGuid) | Should -BeGreaterOrEqual 0
             }
         }
 

Tests/GitHubRepositoryForks.tests.ps1

@@ -6,47 +6,64 @@
    Tests for GitHubRepositoryForks.ps1 module
 #>
 
+[CmdletBinding()]
+[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', '',
+    Justification='Suppress false positives in Pester code blocks')]
+param()
+
 # This is common test code setup logic for all Pester test files
 $moduleRootPath = Split-Path -Path $PSScriptRoot -Parent
 . (Join-Path -Path $moduleRootPath -ChildPath 'Tests\Common.ps1')
 
 try
 {
-    Describe 'Creating a new fork for user' {
-        $originalForks = @(Get-GitHubRepositoryFork -OwnerName Microsoft -RepositoryName PowerShellForGitHub)
+    # Define Script-scoped, readonly, hidden variables.
+    @{
+        upstreamOwnerName = 'microsoft'
+        upstreamRepositoryName = 'PowerShellForGitHub'
+    }.GetEnumerator() | ForEach-Object {
+        Set-Variable -Force -Scope Script -Option ReadOnly -Visibility Private -Name $_.Key -Value $_.Value
+    }
 
+    Describe 'Creating a new fork for user' {
         Context 'When a new fork is created' {
-            $repo = New-GitHubRepositoryFork -OwnerName Microsoft -RepositoryName PowerShellForGitHub
-            $newForks = @(Get-GitHubRepositoryFork -OwnerName Microsoft -RepositoryName PowerShellForGitHub -Sort Newest)
-
-            It 'Should have one more fork than before' {
-                ($newForks.Count - $originalForks.Count) | Should be 1
+            BeforeAll {
+                $repo = New-GitHubRepositoryFork -OwnerName $script:upstreamOwnerName -RepositoryName $script:upstreamRepositoryName
             }
 
-            It 'Should be the latest fork in the list' {
-                $newForks[0].full_name | Should be "$($script:ownerName)/PowerShellForGitHub"
+            AfterAll {
+                Remove-GitHubRepository -Uri $repo.svn_url -Confirm:$false
             }
 
-            Remove-GitHubRepository -Uri $repo.svn_url -Confirm:$false
+            $newForks = @(Get-GitHubRepositoryFork -OwnerName $script:upstreamOwnerName -RepositoryName $script:upstreamRepositoryName -Sort Newest)
+            $ourFork = $newForks | Where-Object { $_.owner.login -eq $script:ownerName }
+
+            It 'Should be in the list' {
+                # Doing this syntax, because due to odd timing with GitHub, it's possible it may
+                # think that there's an existing clone out there and so may name this one "...-1"
+                $ourFork.full_name.StartsWith("$($script:ownerName)/$script:upstreamRepositoryName") | Should -BeTrue
+            }
         }
     }
 
     Describe 'Creating a new fork for an org' {
-        $originalForks = @(Get-GitHubRepositoryFork -OwnerName Microsoft -RepositoryName PowerShellForGitHub)
-
         Context 'When a new fork is created' {
-            $repo = New-GitHubRepositoryFork -OwnerName Microsoft -RepositoryName PowerShellForGitHub -OrganizationName $script:organizationName
-            $newForks = @(Get-GitHubRepositoryFork -OwnerName Microsoft -RepositoryName PowerShellForGitHub -Sort Newest)
-
-            It 'Should have one more fork than before' {
-                ($newForks.Count - $originalForks.Count) | Should be 1
+            BeforeAll {
+                $repo = New-GitHubRepositoryFork -OwnerName $script:upstreamOwnerName -RepositoryName $script:upstreamRepositoryName -OrganizationName $script:organizationName
             }
 
-            It 'Should be the latest fork in the list' {
-                $newForks[0].full_name | Should be "$($script:organizationName)/PowerShellForGitHub"
+            AfterAll {
+                Remove-GitHubRepository -Uri $repo.svn_url -Confirm:$false
             }
 
-            Remove-GitHubRepository -Uri $repo.svn_url -Confirm:$false
+            $newForks = @(Get-GitHubRepositoryFork -OwnerName $script:upstreamOwnerName -RepositoryName $script:upstreamRepositoryName -Sort Newest)
+            $ourFork = $newForks | Where-Object { $_.owner.login -eq $script:organizationName }
+
+            It 'Should be in the list' {
+                # Doing this syntax, because due to odd timing with GitHub, it's possible it may
+                # think that there's an existing clone out there and so may name this one "...-1"
+                $ourFork.full_name.StartsWith("$($script:organizationName)/$script:upstreamRepositoryName") | Should -BeTrue
+            }
         }
     }
 }

build/pipelines/azure-pipelines.ci.yaml

@@ -21,26 +21,39 @@ trigger:
 jobs:
 - job: Windows
   pool:
-    vmImage: 'vs2017-win2016'
+    vmImage: 'windows-latest'
   steps:
   - template: ./templates/verify-testConfigSettingsHash.yaml
   - template: ./templates/run-staticAnalysis.yaml
   - template: ./templates/run-unitTests.yaml
+    parameters:
+      gitHubAccessToken: $(WindowsCIGitHubAccessToken)
+      gitHubOwnerName: $(WindowsCIGitHubOwnerName)
+      gitHubOrganizationName: $(WindowsCIGitHubOrganizationName)
+      platformName: 'Windows'
 
 - job: Linux
   pool:
-    vmImage: 'ubuntu-16.04'
-  dependsOn: Windows # Run in series instead of parallel because the UT's are modifying the same shared state
+    vmImage: 'ubuntu-latest'
   steps:
   - template: ./templates/verify-testConfigSettingsHash.yaml
   - template: ./templates/run-staticAnalysis.yaml
   - template: ./templates/run-unitTests.yaml
+    parameters:
+      gitHubAccessToken: $(LinuxCIGitHubAccessToken)
+      gitHubOwnerName: $(LinuxCIGitHubOwnerName)
+      gitHubOrganizationName: $(LinuxCIGitHubOrganizationName)
+      platformName: 'Linux'
 
 - job: macOS
   pool:
-    vmImage: 'macOS-10.14'
-  dependsOn: Linux # Run in series instead of parallel because the UT's are modifying the same shared state
+    vmImage: 'macOS-latest'
   steps:
   - template: ./templates/verify-testConfigSettingsHash.yaml
   - template: ./templates/run-staticAnalysis.yaml
   - template: ./templates/run-unitTests.yaml
+    parameters:
+      gitHubAccessToken: $(MacCIGitHubAccessToken)
+      gitHubOwnerName: $(MacCIGitHubOwnerName)
+      gitHubOrganizationName: $(MacCIGitHubOrganizationName)
+      platformName: 'macOS'

build/pipelines/azure-pipelines.release.yaml

@@ -27,10 +27,15 @@ variables:
 jobs:
 - job: Validate
   pool:
-    vmImage: 'vs2017-win2016'
+    vmImage: 'windows-latest'
   steps:
   - template: ./templates/run-staticAnalysis.yaml
   - template: ./templates/run-unitTests.yaml
+    parameters:
+      gitHubAccessToken: $(GitHubAccessToken)
+      gitHubOwnerName: $(GitHubOwnerName)
+      gitHubOrganizationName: $(GitHubOrganizationName)
+      platformName: 'Windows'
 
 - job: Release
   dependsOn: Validate

build/pipelines/templates/run-unitTests.yaml

@@ -13,6 +13,17 @@
 #  3. GitHubOrganizationName - The default "organization" that will be used for tests.
 #--------------------------------------------------------------------------------------------------
 
+parameters:
+- name: 'gitHubAccessToken'
+  type: string
+- name: 'gitHubOwnerName'
+  type: string
+- name: 'gitHubOrganizationName'
+  type: string
+- name: 'platformName'
+  default: 'Windows'
+  type: string
+
 steps:
   - powershell: |
       Install-Module -Name Pester -Repository PSGallery -Scope CurrentUser -AllowClobber -SkipPublisherCheck -RequiredVersion 4.10.1 -Force -Verbose
@@ -24,14 +35,14 @@ steps:
     workingDirectory: '$(System.DefaultWorkingDirectory)'
     displayName: 'Run Unit Tests via Pester'
     env:
-      ciAccessToken: $(GitHubAccessToken)
-      ciOwnerName: $(GitHubOwnerName)
-      ciOrganizationName: $(GitHubOrganizationName)
+      ciAccessToken: ${{ parameters.gitHubAccessToken }}
+      ciOwnerName: ${{ parameters.gitHubOwnerName }}
+      ciOrganizationName: ${{ parameters.gitHubOrganizationName }}
 
   - task: PublishTestResults@2
     displayName: 'Publish Test Results'
     inputs:
-      testRunTitle: 'Windows Test Results for Pester'
+      testRunTitle: '${{ parameters.platformName }} Test Results for Pester'
       buildPlatform: 'Windows'
       testRunner: NUnit
       testResultsFiles: '../Pester/test-results.xml'