Tests may be broken on Windows

[felicitymay]

When I run tests locally using npm test on master:

I get the following output:

$ npm test

> help.github.com@0.0.1 test C:\2_Semmle\Repositories\help-docs
> jest && standard && npm run check-deps

FAIL tests/unit/products.js (31.288 s)
  ● products module › every product is valid

    Custom message:
      {
      "product": {
        "id": "github",
        "name": "GitHub.com",
        "href": "/github",
        "dir": "content/github",
        "toc": "content\\github\\index.md",
        "wip": false,
        "hasEnterpriseUserVersions": true
      },
      "errors": [
        {
          "attribute": "pattern",
          "property": "toc",
          "expected": "^content/.*?index.md$",
          "actual": "content\\github\\index.md",
          "message": "invalid input"
        }
      ]
    }

    expect(received).toBe(expected) // Object.is equality

    Expected: true
    Received: false

      14 |       const { valid, errors } = revalidator.validate(product, schema)
      15 |       const expectation = JSON.stringify({ product, errors }, null, 2)
    > 16 |       expect(valid, expectation).toBe(true)
         |                                  ^
      17 |     })
      18 |   })
      19 | })

      at forEach (tests/unit/products.js:16:34)
          at Array.forEach (<anonymous>)
      at Object.<anonymous> (tests/unit/products.js:13:29)

PASS tests/routing/deprecated-enterprise-versions.js (33.404 s)
PASS tests/unit/toc-links.js (43.006 s)
FAIL tests/rendering/server.js (57.626 s)
  ● stylesheets › compiles and sets the right content-type header

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 301

      740 |   it('compiles and sets the right content-type header', async () => {
      741 |     const res = await get('/dist/index.css')
    > 742 |     expect(res.statusCode).toBe(200)
          |                            ^
      743 |     expect(res.headers['content-type']).toBe('text/css; charset=UTF-8')
      744 |   })
      745 | })

      at Object.<anonymous> (tests/rendering/server.js:742:28)

  ● client-side JavaScript bundle › returns a 200 response

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 301

      753 |
      754 |   it('returns a 200 response', async () => {
    > 755 |     expect(res.statusCode).toBe(200)
          |                            ^
      756 |   })
      757 |
      758 |   it('sets the right content-type header', async () => {

      at Object.<anonymous> (tests/rendering/server.js:755:28)

  ● client-side JavaScript bundle › sets the right content-type header

    expect(received).toBe(expected) // Object.is equality

    Expected: "application/javascript; charset=UTF-8"
    Received: "text/plain; charset=utf-8"

      757 |
      758 |   it('sets the right content-type header', async () => {
    > 759 |     expect(res.headers['content-type']).toBe('application/javascript; charset=UTF-8')
          |                                         ^
      760 |   })
      761 |
      762 |   // TODO: configure webpack to create production bundle in the test env

      at Object.<anonymous> (tests/rendering/server.js:759:41)

PASS tests/rendering/header.js (28.09 s)
PASS tests/content/site-tree.js (29.112 s)
PASS tests/rendering/breadcrumbs.js (31.095 s)
FAIL tests/content/article-links.js (82.541 s)
  ● article links › in "content\actions\hosting-your-own-runners\monitoring-and-troubleshooting-self-hosted-runners.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [962]: Starting
      - [962]: Started
      - [962]: Started
      - [962]: √
      - [962]: 2020-02-11
      - [962]: 2020-02-11
      - [962]: 2020-02-11

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 7

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\enterprise\admin\enterprise-management\migrating-from-github-enterprise-1110x-to-2123.md" › URLs must not contain a hard-coded version number

    Custom message:
      Found article links with hard-coded version numbers:
      - [Upgrading to the latest release](/enterprise/11.10.340/admin/articles/upgrading-to-the-latest-release/)
      - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)
      - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 3

      180 |
      181 |         const errorMessage = formatLinkError(versionLinkErrorText, matches)
    > 182 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      183 |       })
      184 |
      185 |       test('URLs must not contain a hard-coded domain name', async () => {

      at Object.<anonymous> (tests/content/article-links.js:182:46)

  ● article links › in "content\enterprise\admin\user-management\configuring-email-for-notifications.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [13210]: connect
      - [13210]: 51DC9163323:
      - [13216]: 51DC9163323:
      - [17250]: 51DC9163323:
      - [13217]: 51DC9163323:
      - [17250]: 51DC9163323:
      - [13210]: disconnect

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 7

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\building-a-strong-community\editing-wiki-content.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [Link Text](full-URL-of-wiki-page)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\building-a-strong-community\manually-creating-a-single-issue-template-for-your-repository.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [DATE]: [FEATURE

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\enforcing-best-practices-with-github-policies\constraints.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [a-z]([a-z]|-)
      - [a-z]([a-z]|-)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 2

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\enforcing-best-practices-with-github-policies\overview.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [A-Z]([a-z]|-)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\finding-security-vulnerabilities-and-errors-in-your-code\sarif-support-for-code-scanning.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [here](1)
      - [here](2)
      - [ruleIndex](1)
      - [ruleID](2)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 4

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\rest\overview\libraries.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [pithub-github] ([CPAN][pithub-cpan])
      - [net-github-github] ([CPAN][net-github-cpan])

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 2

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "data\reusables\repositories\relative-links.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [Contribution guidelines for this project](docs/CONTRIBUTING.md)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

(node:7752) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
PASS tests/routing/top-developer-site-path-redirects.js (29.88 s)
PASS tests/routing/redirects.js (29.404 s)
PASS tests/content/site-data.js (33.601 s)
PASS tests/rendering/page-titles.js (30.475 s)
PASS tests/rendering/sidebar.js (31.603 s)
PASS tests/routing/developer-site-redirects.js (28.341 s)
PASS tests/rendering/footer.js (27.494 s)
PASS tests/content/graphql.js (29.668 s)
PASS tests/rendering/curated-homepage-links.js (27.603 s)
PASS tests/content/webhooks.js (26.777 s)
PASS tests/content/featured-links.js (27.765 s)
PASS tests/routing/language-code-redirects.js (32.3 s)
PASS tests/links-and-images/links-and-images.js (150.636 s)
PASS tests/rendering/head.js (27.813 s)
PASS tests/rendering/rest.js (26.164 s)
PASS tests/unit/pages.js (18.399 s)
PASS tests/content/glossary.js (11.873 s)
PASS tests/content/site-data-references.js (14.73 s)
PASS tests/content/crowdin-config.js (10.643 s)
PASS tests/unit/liquid.js
PASS tests/unit/page.js
PASS tests/content/liquid-line-breaks.js (24.988 s)
PASS tests/unit/find-page.js
PASS tests/rendering/robots-txt.js
PASS tests/unit/algolia/parse-page-sections-into-records.js
PASS tests/content/remove-liquid-statements.js
PASS tests/routing/middleware/redirects/help-to-docs.js
PASS tests/unit/enterprise-versions.js
PASS tests/unit/permalink.js
PASS tests/content/gitignore.js
PASS tests/meta/orphan-tests.js
PASS tests/unit/languages.js
PASS tests/unit/product-names.js
PASS tests/content/algolia-search.js
FAIL tests/content/category-pages.js (6.286 s)
  ● category pages › category index "github\getting-started-with-github\index.md" › contains all expected articles

    Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.

      at mapper (node_modules/jest-jasmine2/build/queueRunner.js:29:45)

  ● category pages › category index "github\getting-started-with-github\index.md" › does not any unexpected articles

    Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.

      at mapper (node_modules/jest-jasmine2/build/queueRunner.js:29:45)

PASS tests/unit/liquid-helpers.js (10.572 s)
PASS tests/links-and-images/developer-links-and-images.js (736.398 s)
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.

Summary of all failing tests
FAIL tests/unit/products.js (31.288 s)
  ● products module › every product is valid

    Custom message:
      {
      "product": {
        "id": "github",
        "name": "GitHub.com",
        "href": "/github",
        "dir": "content/github",
        "toc": "content\\github\\index.md",
        "wip": false,
        "hasEnterpriseUserVersions": true
      },
      "errors": [
        {
          "attribute": "pattern",
          "property": "toc",
          "expected": "^content/.*?index.md$",
          "actual": "content\\github\\index.md",
          "message": "invalid input"
        }
      ]
    }

    expect(received).toBe(expected) // Object.is equality

    Expected: true
    Received: false

      14 |       const { valid, errors } = revalidator.validate(product, schema)
      15 |       const expectation = JSON.stringify({ product, errors }, null, 2)
    > 16 |       expect(valid, expectation).toBe(true)
         |                                  ^
      17 |     })
      18 |   })
      19 | })

      at forEach (tests/unit/products.js:16:34)
          at Array.forEach (<anonymous>)
      at Object.<anonymous> (tests/unit/products.js:13:29)

FAIL tests/rendering/server.js (57.626 s)
  ● stylesheets › compiles and sets the right content-type header

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 301

      740 |   it('compiles and sets the right content-type header', async () => {
      741 |     const res = await get('/dist/index.css')
    > 742 |     expect(res.statusCode).toBe(200)
          |                            ^
      743 |     expect(res.headers['content-type']).toBe('text/css; charset=UTF-8')
      744 |   })
      745 | })

      at Object.<anonymous> (tests/rendering/server.js:742:28)

  ● client-side JavaScript bundle › returns a 200 response

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 301

      753 |
      754 |   it('returns a 200 response', async () => {
    > 755 |     expect(res.statusCode).toBe(200)
          |                            ^
      756 |   })
      757 |
      758 |   it('sets the right content-type header', async () => {

      at Object.<anonymous> (tests/rendering/server.js:755:28)

  ● client-side JavaScript bundle › sets the right content-type header

    expect(received).toBe(expected) // Object.is equality

    Expected: "application/javascript; charset=UTF-8"
    Received: "text/plain; charset=utf-8"

      757 |
      758 |   it('sets the right content-type header', async () => {
    > 759 |     expect(res.headers['content-type']).toBe('application/javascript; charset=UTF-8')
          |                                         ^
      760 |   })
      761 |
      762 |   // TODO: configure webpack to create production bundle in the test env

      at Object.<anonymous> (tests/rendering/server.js:759:41)

FAIL tests/content/article-links.js (82.541 s)
  ● article links › in "content\actions\hosting-your-own-runners\monitoring-and-troubleshooting-self-hosted-runners.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [962]: Starting
      - [962]: Started
      - [962]: Started
      - [962]: √
      - [962]: 2020-02-11
      - [962]: 2020-02-11
      - [962]: 2020-02-11

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 7

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\enterprise\admin\enterprise-management\migrating-from-github-enterprise-1110x-to-2123.md" › URLs must not contain a hard-coded version number

    Custom message:
      Found article links with hard-coded version numbers:
      - [Upgrading to the latest release](/enterprise/11.10.340/admin/articles/upgrading-to-the-latest-release/)
      - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)
      - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 3

      180 |
      181 |         const errorMessage = formatLinkError(versionLinkErrorText, matches)
    > 182 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      183 |       })
      184 |
      185 |       test('URLs must not contain a hard-coded domain name', async () => {

      at Object.<anonymous> (tests/content/article-links.js:182:46)

  ● article links › in "content\enterprise\admin\user-management\configuring-email-for-notifications.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [13210]: connect
      - [13210]: 51DC9163323:
      - [13216]: 51DC9163323:
      - [17250]: 51DC9163323:
      - [13217]: 51DC9163323:
      - [17250]: 51DC9163323:
      - [13210]: disconnect

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 7

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\building-a-strong-community\editing-wiki-content.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [Link Text](full-URL-of-wiki-page)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\building-a-strong-community\manually-creating-a-single-issue-template-for-your-repository.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [DATE]: [FEATURE

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\enforcing-best-practices-with-github-policies\constraints.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [a-z]([a-z]|-)
      - [a-z]([a-z]|-)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 2

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\enforcing-best-practices-with-github-policies\overview.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [A-Z]([a-z]|-)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\github\finding-security-vulnerabilities-and-errors-in-your-code\sarif-support-for-code-scanning.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [here](1)
      - [here](2)
      - [ruleIndex](1)
      - [ruleID](2)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 4

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "content\rest\overview\libraries.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [pithub-github] ([CPAN][pithub-cpan])
      - [net-github-github] ([CPAN][net-github-cpan])

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 2

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

  ● article links › in "data\reusables\repositories\relative-links.md" › relative URLs must start with "/"

    Custom message:
      Found unexpected relative article links:
      - [Contribution guidelines for this project](docs/CONTRIBUTING.md)

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

      159 |
      160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
    > 161 |         expect(matches.length, errorMessage).toBe(0)
          |                                              ^
      162 |       })
      163 |
      164 |       test('URLs must not contain a hard-coded language code', async () => {

      at Object.<anonymous> (tests/content/article-links.js:161:46)

FAIL tests/content/category-pages.js (6.286 s)
  ● category pages › category index "github\getting-started-with-github\index.md" › contains all expected articles

    Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.

      at mapper (node_modules/jest-jasmine2/build/queueRunner.js:29:45)

  ● category pages › category index "github\getting-started-with-github\index.md" › does not any unexpected articles

    Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.

      at mapper (node_modules/jest-jasmine2/build/queueRunner.js:29:45)


Test Suites: 4 failed, 41 passed, 45 total
Tests:       16 failed, 7 skipped, 2 todo, 14879 passed, 14904 total
Snapshots:   0 total
Time:        741.238 s, estimated 781 s
Ran all test suites.
npm ERR! Test failed.  See above for more details.

[heiskr]

Hi 👋🏼 When you run node --version, which version do you have installed?

[felicitymay]

6.10.2 - sorry if I should have tried updating npm instead of assuming that the tests were broken on Windows again

[heiskr]

Thanks for the information :) I would try updating to Node version 12.18.3 first. Many of these errors look like features that came in later Node versions.

[felicitymay]

Oh wow - it sounds as if I'm a long way out of date 😞

[felicitymay]

Ah - somehow I typed the wrong thing - I should have waited until the morning to answer.

I'm not so far out of date when I try the right command: v12.8.1
Will try updating to 12.18.3 and see if that helps.

[felicitymay]

Having upgraded, it looks as if it's better but I still see errors and the tests appear to get stuck after links-and-images.js

The following output is for the same 3add879269e SHA, which was master (and therefore, I assume should have passing tests):

``` $ npm test

help.github.com@0.0.1 test C:\2_Semmle\Repositories\help-docs
jest && standard && npm run check-deps

PASS tests/content/remove-liquid-statements.js
PASS tests/unit/page.js (7.745 s)
PASS tests/routing/redirects.js (46.417 s)
PASS tests/routing/deprecated-enterprise-versions.js (41.978 s)
PASS tests/rendering/breadcrumbs.js (45.279 s)
PASS tests/unit/liquid-helpers.js (13.896 s)
PASS tests/routing/developer-site-redirects.js (63.723 s)
PASS tests/content/category-pages.js
PASS tests/unit/pages.js (17.772 s)
PASS tests/rendering/robots-txt.js
FAIL tests/rendering/server.js (75.094 s)
● stylesheets › compiles and sets the right content-type header

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 301

  740 |   it('compiles and sets the right content-type header', async () => {
  741 |     const res = await get('/dist/index.css')
> 742 |     expect(res.statusCode).toBe(200)
      |                            ^
  743 |     expect(res.headers['content-type']).toBe('text/css; charset=UTF-8')
  744 |   })
  745 | })

  at Object.<anonymous> (tests/rendering/server.js:742:28)
      at runMicrotasks (<anonymous>)

● client-side JavaScript bundle › returns a 200 response

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 301

  753 |
  754 |   it('returns a 200 response', async () => {
> 755 |     expect(res.statusCode).toBe(200)
      |                            ^
  756 |   })
  757 |
  758 |   it('sets the right content-type header', async () => {

  at Object.<anonymous> (tests/rendering/server.js:755:28)
      at runMicrotasks (<anonymous>)

● client-side JavaScript bundle › sets the right content-type header

expect(received).toBe(expected) // Object.is equality

Expected: "application/javascript; charset=UTF-8"
Received: "text/plain; charset=utf-8"

  757 |
  758 |   it('sets the right content-type header', async () => {
> 759 |     expect(res.headers['content-type']).toBe('application/javascript; charset=UTF-8')
      |                                         ^
  760 |   })
  761 |
  762 |   // TODO: configure webpack to create production bundle in the test env

  at Object.<anonymous> (tests/rendering/server.js:759:41)
      at runMicrotasks (<anonymous>)

PASS tests/rendering/header.js (34.49 s)
PASS tests/content/site-data.js (34.854 s)
PASS tests/content/graphql.js (31.457 s)
PASS tests/content/liquid-line-breaks.js (31.078 s)
PASS tests/routing/middleware/redirects/help-to-docs.js
FAIL tests/content/article-links.js (107.833 s)
● article links › in "content\actions\hosting-your-own-runners\monitoring-and-troubleshooting-self-hosted-runners.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [962]: Starting
  - [962]: Started
  - [962]: Started
  - [962]: √
  - [962]: 2020-02-11
  - [962]: 2020-02-11
  - [962]: 2020-02-11

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 7

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\enterprise\admin\enterprise-management\migrating-from-github-enterprise-1110x-to-2123.md" › URLs must not contain a hard-coded version number

Custom message:
  Found article links with hard-coded version numbers:
  - [Upgrading to the latest release](/enterprise/11.10.340/admin/articles/upgrading-to-the-latest-release/)
  - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)
  - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 3

  180 |
  181 |         const errorMessage = formatLinkError(versionLinkErrorText, matches)
> 182 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  183 |       })
  184 |
  185 |       test('URLs must not contain a hard-coded domain name', async () => {

  at Object.<anonymous> (tests/content/article-links.js:182:46)

● article links › in "content\enterprise\admin\user-management\configuring-email-for-notifications.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [13210]: connect
  - [13210]: 51DC9163323:
  - [13216]: 51DC9163323:
  - [17250]: 51DC9163323:
  - [13217]: 51DC9163323:
  - [17250]: 51DC9163323:
  - [13210]: disconnect

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 7

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\building-a-strong-community\editing-wiki-content.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [Link Text](full-URL-of-wiki-page)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\building-a-strong-community\manually-creating-a-single-issue-template-for-your-repository.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [DATE]: [FEATURE

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\enforcing-best-practices-with-github-policies\constraints.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [a-z]([a-z]|-)
  - [a-z]([a-z]|-)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 2

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\enforcing-best-practices-with-github-policies\overview.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [A-Z]([a-z]|-)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\finding-security-vulnerabilities-and-errors-in-your-code\sarif-support-for-code-scanning.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [here](1)
  - [here](2)
  - [ruleIndex](1)
  - [ruleID](2)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 4

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\rest\overview\libraries.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [pithub-github] ([CPAN][pithub-cpan])
  - [net-github-github] ([CPAN][net-github-cpan])

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 2

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "data\reusables\repositories\relative-links.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [Contribution guidelines for this project](docs/CONTRIBUTING.md)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

PASS tests/content/site-data-references.js (17.319 s)
PASS tests/content/site-tree.js (36.4 s)
PASS tests/content/featured-links.js (34.831 s)
PASS tests/unit/algolia/parse-page-sections-into-records.js
PASS tests/content/webhooks.js (33.161 s)
PASS tests/unit/permalink.js
PASS tests/rendering/page-titles.js (40.634 s)
PASS tests/content/glossary.js (13.029 s)
PASS tests/rendering/rest.js (33.872 s)
FAIL tests/unit/products.js (35.441 s)
● products module › every product is valid

Custom message:
  {
  "product": {
    "id": "github",
    "name": "GitHub.com",
    "href": "/github",
    "dir": "content/github",
    "toc": "content\\github\\index.md",
    "wip": false,
    "hasEnterpriseUserVersions": true
  },
  "errors": [
    {
      "attribute": "pattern",
      "property": "toc",
      "expected": "^content/.*?index.md$",
      "actual": "content\\github\\index.md",
      "message": "invalid input"
    }
  ]
}

expect(received).toBe(expected) // Object.is equality

Expected: true
Received: false

  14 |       const { valid, errors } = revalidator.validate(product, schema)
  15 |       const expectation = JSON.stringify({ product, errors }, null, 2)
> 16 |       expect(valid, expectation).toBe(true)
     |                                  ^
  17 |     })
  18 |   })
  19 | })

  at forEach (tests/unit/products.js:16:34)
      at Array.forEach (<anonymous>)
  at Object.<anonymous> (tests/unit/products.js:13:29)

PASS tests/unit/liquid.js
PASS tests/content/crowdin-config.js (13.508 s)
PASS tests/unit/enterprise-versions.js
PASS tests/unit/find-page.js
PASS tests/rendering/curated-homepage-links.js (35.968 s)
PASS tests/content/algolia-search.js
PASS tests/rendering/sidebar.js (35.207 s)
PASS tests/rendering/head.js (34.608 s)
PASS tests/meta/orphan-tests.js
PASS tests/unit/languages.js
PASS tests/unit/product-names.js
PASS tests/content/gitignore.js
(node:15772) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
PASS tests/routing/top-developer-site-path-redirects.js (37.896 s)
PASS tests/unit/toc-links.js (48.047 s)
PASS tests/routing/language-code-redirects.js (29.58 s)
PASS tests/rendering/footer.js (33.798 s)
PASS tests/links-and-images/links-and-images.js (292.192 s)

Running it on the current `main` (`677d9cb59ef`), it looks as if I should have left the first test for longer before killing it. I get the following:

help.github.com@0.0.1 test C:\2_Semmle\Repositories\help-docs
jest && standard && npm run check-deps

PASS tests/content/remove-liquid-statements.js
PASS tests/unit/page.js (6.385 s)
PASS tests/unit/liquid-helpers.js (13.877 s)
PASS tests/routing/redirects.js (40.58 s)
PASS tests/rendering/breadcrumbs.js (37.88 s)
PASS tests/routing/deprecated-enterprise-versions.js (36.677 s)
PASS tests/routing/developer-site-redirects.js (57.047 s)
PASS tests/unit/pages.js (17.949 s)
PASS tests/content/category-pages.js
PASS tests/rendering/robots-txt.js
FAIL tests/rendering/server.js (64.648 s)
● static assets › fonts

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 301

  626 | describe('static assets', () => {
  627 |   test('fonts', async () => {
> 628 |     expect((await get('/dist/fonts/Inter-Medium.woff')).statusCode).toBe(200)
      |                                                                     ^
  629 |     expect((await get('/dist/fonts/Inter-Regular.woff')).statusCode).toBe(200)
  630 |   })
  631 | })

  at Object.<anonymous> (tests/rendering/server.js:628:69)
      at runMicrotasks (<anonymous>)

● stylesheets › compiles and sets the right content-type header

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 301

  724 |   it('compiles and sets the right content-type header', async () => {
  725 |     const res = await get('/dist/index.css')
> 726 |     expect(res.statusCode).toBe(200)
      |                            ^
  727 |     expect(res.headers['content-type']).toBe('text/css; charset=UTF-8')
  728 |   })
  729 | })

  at Object.<anonymous> (tests/rendering/server.js:726:28)
      at runMicrotasks (<anonymous>)

● client-side JavaScript bundle › returns a 200 response

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 301

  737 |
  738 |   it('returns a 200 response', async () => {
> 739 |     expect(res.statusCode).toBe(200)
      |                            ^
  740 |   })
  741 |
  742 |   it('sets the right content-type header', async () => {

  at Object.<anonymous> (tests/rendering/server.js:739:28)
      at runMicrotasks (<anonymous>)

● client-side JavaScript bundle › sets the right content-type header

expect(received).toBe(expected) // Object.is equality

Expected: "application/javascript; charset=UTF-8"
Received: "text/plain; charset=utf-8"

  741 |
  742 |   it('sets the right content-type header', async () => {
> 743 |     expect(res.headers['content-type']).toBe('application/javascript; charset=UTF-8')
      |                                         ^
  744 |   })
  745 |
  746 |   // TODO: configure webpack to create production bundle in the test env

  at Object.<anonymous> (tests/rendering/server.js:743:41)
      at runMicrotasks (<anonymous>)

PASS tests/rendering/header.js (34.021 s)
PASS tests/content/graphql.js (31.44 s)
PASS tests/content/site-data.js (35.615 s)
PASS tests/content/liquid-line-breaks.js (33.801 s)
PASS tests/rendering/events.js
● Console

console.log
  data should have required property 'type'

  at middleware/events.js:20:56

console.log
  data should have required property 'url'

  at middleware/events.js:20:56

console.log
  data.url should match format "uri"

  at middleware/events.js:20:56

console.log
  data should have required property 'vote'

  at middleware/events.js:20:56

console.log
  data.vote should be equal to one of the allowed values

  at middleware/events.js:20:56

console.log
  data.email should match format "email"

  at middleware/events.js:20:56

console.log
  data.comment should be string

  at middleware/events.js:20:56

console.log
  data.category should be equal to one of the allowed values

  at middleware/events.js:20:56

PASS tests/routing/middleware/redirects/help-to-docs.js
FAIL tests/content/article-links.js (106.856 s)
● article links › in "content\actions\hosting-your-own-runners\monitoring-and-troubleshooting-self-hosted-runners.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [962]: Starting
  - [962]: Started
  - [962]: Started
  - [962]: √
  - [962]: 2020-02-11
  - [962]: 2020-02-11
  - [962]: 2020-02-11

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 7

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\enterprise\admin\enterprise-management\migrating-from-github-enterprise-1110x-to-2123.md" › URLs must not contain a hard-coded version number

Custom message:
  Found article links with hard-coded version numbers:
  - [Upgrading to the latest release](/enterprise/11.10.340/admin/articles/upgrading-to-the-latest-release/)
  - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)
  - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 3

  180 |
  181 |         const errorMessage = formatLinkError(versionLinkErrorText, matches)
> 182 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  183 |       })
  184 |
  185 |       test('URLs must not contain a hard-coded domain name', async () => {

  at Object.<anonymous> (tests/content/article-links.js:182:46)

● article links › in "content\enterprise\admin\user-management\configuring-email-for-notifications.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [13210]: connect
  - [13210]: 51DC9163323:
  - [13216]: 51DC9163323:
  - [17250]: 51DC9163323:
  - [13217]: 51DC9163323:
  - [17250]: 51DC9163323:
  - [13210]: disconnect

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 7

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\building-a-strong-community\editing-wiki-content.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [Link Text](full-URL-of-wiki-page)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\building-a-strong-community\manually-creating-a-single-issue-template-for-your-repository.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [DATE]: [FEATURE

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\enforcing-best-practices-with-github-policies\constraints.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [a-z]([a-z]|-)
  - [a-z]([a-z]|-)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 2

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\enforcing-best-practices-with-github-policies\overview.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [A-Z]([a-z]|-)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\finding-security-vulnerabilities-and-errors-in-your-code\sarif-support-for-code-scanning.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [here](1)
  - [here](2)
  - [ruleIndex](1)
  - [ruleID](2)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 4

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\rest\overview\libraries.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [pithub-github] ([CPAN][pithub-cpan])
  - [net-github-github] ([CPAN][net-github-cpan])

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 2

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "data\reusables\repositories\relative-links.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [Contribution guidelines for this project](docs/CONTRIBUTING.md)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

PASS tests/content/site-data-references.js (17.258 s)
PASS tests/content/site-tree.js (35.689 s)
PASS tests/content/featured-links.js (34.507 s)
PASS tests/unit/algolia/parse-page-sections-into-records.js
PASS tests/content/webhooks.js (32.953 s)
PASS tests/unit/permalink.js
PASS tests/content/glossary.js (12.952 s)
PASS tests/rendering/octicon.js
PASS tests/rendering/rest.js (34.013 s)
PASS tests/rendering/page-titles.js (40.096 s)
FAIL tests/unit/products.js (35.836 s)
● products module › every product is valid

Custom message:
  {
  "product": {
    "id": "github",
    "name": "GitHub.com",
    "href": "/github",
    "dir": "content/github",
    "toc": "content\\github\\index.md",
    "wip": false,
    "hasEnterpriseUserVersions": true
  },
  "errors": [
    {
      "attribute": "pattern",
      "property": "toc",
      "expected": "^content/.*?index.md$",
      "actual": "content\\github\\index.md",
      "message": "invalid input"
    }
  ]
}

expect(received).toBe(expected) // Object.is equality

Expected: true
Received: false

  14 |       const { valid, errors } = revalidator.validate(product, schema)
  15 |       const expectation = JSON.stringify({ product, errors }, null, 2)
> 16 |       expect(valid, expectation).toBe(true)
     |                                  ^
  17 |     })
  18 |   })
  19 | })

  at forEach (tests/unit/products.js:16:34)
      at Array.forEach (<anonymous>)
  at Object.<anonymous> (tests/unit/products.js:13:29)

PASS tests/unit/liquid.js
PASS tests/content/crowdin-config.js (12.993 s)
PASS tests/unit/enterprise-versions.js
PASS tests/unit/find-page.js
PASS tests/unit/failbot.js
PASS tests/rendering/curated-homepage-links.js (34.048 s)
PASS tests/content/algolia-search.js
PASS tests/rendering/sidebar.js (33.698 s)
PASS tests/rendering/head.js (33.909 s)
PASS tests/meta/orphan-tests.js
PASS tests/unit/languages.js
PASS tests/unit/product-names.js
PASS tests/content/gitignore.js
(node:2368) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
PASS tests/routing/top-developer-site-path-redirects.js (36.152 s)
PASS tests/routing/language-code-redirects.js (29.76 s)
PASS tests/unit/toc-links.js (50.083 s)
PASS tests/rendering/footer.js (31.287 s)
PASS tests/links-and-images/links-and-images.js (204.24 s)
FAIL tests/links-and-images/developer-links-and-images.js (1025.675 s)
● page rendering › every page has image references that can be resolved

Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.

  at mapper (node_modules/jest-jasmine2/build/queueRunner.js:27:45)

● page rendering › every page has links with anchors that can be resolved

Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.

  at mapper (node_modules/jest-jasmine2/build/queueRunner.js:27:45)

● page rendering › every page has links that can be resolved

Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.

  at mapper (node_modules/jest-jasmine2/build/queueRunner.js:27:45)

A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.
Summary of all failing tests
FAIL tests/rendering/server.js (64.648 s)
● static assets › fonts

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 301

  626 | describe('static assets', () => {
  627 |   test('fonts', async () => {
> 628 |     expect((await get('/dist/fonts/Inter-Medium.woff')).statusCode).toBe(200)
      |                                                                     ^
  629 |     expect((await get('/dist/fonts/Inter-Regular.woff')).statusCode).toBe(200)
  630 |   })
  631 | })

  at Object.<anonymous> (tests/rendering/server.js:628:69)
      at runMicrotasks (<anonymous>)

● stylesheets › compiles and sets the right content-type header

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 301

  724 |   it('compiles and sets the right content-type header', async () => {
  725 |     const res = await get('/dist/index.css')
> 726 |     expect(res.statusCode).toBe(200)
      |                            ^
  727 |     expect(res.headers['content-type']).toBe('text/css; charset=UTF-8')
  728 |   })
  729 | })

  at Object.<anonymous> (tests/rendering/server.js:726:28)
      at runMicrotasks (<anonymous>)

● client-side JavaScript bundle › returns a 200 response

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 301

  737 |
  738 |   it('returns a 200 response', async () => {
> 739 |     expect(res.statusCode).toBe(200)
      |                            ^
  740 |   })
  741 |
  742 |   it('sets the right content-type header', async () => {

  at Object.<anonymous> (tests/rendering/server.js:739:28)
      at runMicrotasks (<anonymous>)

● client-side JavaScript bundle › sets the right content-type header

expect(received).toBe(expected) // Object.is equality

Expected: "application/javascript; charset=UTF-8"
Received: "text/plain; charset=utf-8"

  741 |
  742 |   it('sets the right content-type header', async () => {
> 743 |     expect(res.headers['content-type']).toBe('application/javascript; charset=UTF-8')
      |                                         ^
  744 |   })
  745 |
  746 |   // TODO: configure webpack to create production bundle in the test env

  at Object.<anonymous> (tests/rendering/server.js:743:41)
      at runMicrotasks (<anonymous>)

FAIL tests/content/article-links.js (106.856 s)
● article links › in "content\actions\hosting-your-own-runners\monitoring-and-troubleshooting-self-hosted-runners.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [962]: Starting
  - [962]: Started
  - [962]: Started
  - [962]: √
  - [962]: 2020-02-11
  - [962]: 2020-02-11
  - [962]: 2020-02-11

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 7

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\enterprise\admin\enterprise-management\migrating-from-github-enterprise-1110x-to-2123.md" › URLs must not contain a hard-coded version number

Custom message:
  Found article links with hard-coded version numbers:
  - [Upgrading to the latest release](/enterprise/11.10.340/admin/articles/upgrading-to-the-latest-release/)
  - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)
  - [Provisioning and Installation](/enterprise/2.1/admin/guides/installation/provisioning-and-installation/)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 3

  180 |
  181 |         const errorMessage = formatLinkError(versionLinkErrorText, matches)
> 182 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  183 |       })
  184 |
  185 |       test('URLs must not contain a hard-coded domain name', async () => {

  at Object.<anonymous> (tests/content/article-links.js:182:46)

● article links › in "content\enterprise\admin\user-management\configuring-email-for-notifications.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [13210]: connect
  - [13210]: 51DC9163323:
  - [13216]: 51DC9163323:
  - [17250]: 51DC9163323:
  - [13217]: 51DC9163323:
  - [17250]: 51DC9163323:
  - [13210]: disconnect

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 7

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\building-a-strong-community\editing-wiki-content.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [Link Text](full-URL-of-wiki-page)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\building-a-strong-community\manually-creating-a-single-issue-template-for-your-repository.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [DATE]: [FEATURE

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\enforcing-best-practices-with-github-policies\constraints.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [a-z]([a-z]|-)
  - [a-z]([a-z]|-)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 2

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\enforcing-best-practices-with-github-policies\overview.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [A-Z]([a-z]|-)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\github\finding-security-vulnerabilities-and-errors-in-your-code\sarif-support-for-code-scanning.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [here](1)
  - [here](2)
  - [ruleIndex](1)
  - [ruleID](2)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 4

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "content\rest\overview\libraries.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [pithub-github] ([CPAN][pithub-cpan])
  - [net-github-github] ([CPAN][net-github-cpan])

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 2

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

● article links › in "data\reusables\repositories\relative-links.md" › relative URLs must start with "/"

Custom message:
  Found unexpected relative article links:
  - [Contribution guidelines for this project](docs/CONTRIBUTING.md)

expect(received).toBe(expected) // Object.is equality

Expected: 0
Received: 1

  159 |
  160 |         const errorMessage = formatLinkError(relativeArticleLinkErrorText, matches)
> 161 |         expect(matches.length, errorMessage).toBe(0)
      |                                              ^
  162 |       })
  163 |
  164 |       test('URLs must not contain a hard-coded language code', async () => {

  at Object.<anonymous> (tests/content/article-links.js:161:46)

FAIL tests/unit/products.js (35.836 s)
● products module › every product is valid

Custom message:
  {
  "product": {
    "id": "github",
    "name": "GitHub.com",
    "href": "/github",
    "dir": "content/github",
    "toc": "content\\github\\index.md",
    "wip": false,
    "hasEnterpriseUserVersions": true
  },
  "errors": [
    {
      "attribute": "pattern",
      "property": "toc",
      "expected": "^content/.*?index.md$",
      "actual": "content\\github\\index.md",
      "message": "invalid input"
    }
  ]
}

expect(received).toBe(expected) // Object.is equality

Expected: true
Received: false

  14 |       const { valid, errors } = revalidator.validate(product, schema)
  15 |       const expectation = JSON.stringify({ product, errors }, null, 2)
> 16 |       expect(valid, expectation).toBe(true)
     |                                  ^
  17 |     })
  18 |   })
  19 | })

  at forEach (tests/unit/products.js:16:34)
      at Array.forEach (<anonymous>)
  at Object.<anonymous> (tests/unit/products.js:13:29)

FAIL tests/links-and-images/developer-links-and-images.js (1025.675 s)
● page rendering › every page has image references that can be resolved

Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.

  at mapper (node_modules/jest-jasmine2/build/queueRunner.js:27:45)

● page rendering › every page has links with anchors that can be resolved

Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.

  at mapper (node_modules/jest-jasmine2/build/queueRunner.js:27:45)

● page rendering › every page has links that can be resolved

Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 1000000 ms timeout specified by jest.setTimeout.

  at mapper (node_modules/jest-jasmine2/build/queueRunner.js:27:45)

Test Suites: 4 failed, 44 passed, 48 total
Tests: 17 failed, 7 skipped, 2 todo, 14914 passed, 14940 total
Snapshots: 0 total
Time: 1031.863 s
Ran all test suites.
npm ERR! Test failed. See above for more details.

</details>

[felicitymay]

I'm working on the assumption that, since we ensure tests are green before we merge PRs, I shouldn't be seeing so many test failures on the default branch.

[chiedo]

You can see my output here

> help.github.com@0.0.1 test /Users/chiedo/github.com/github/docs-internal
> jest && standard && npm run check-deps

 PASS  tests/content/remove-liquid-statements.js
 PASS  tests/content/category-pages.js
 PASS  tests/unit/page.js
 PASS  tests/rendering/robots-txt.js
 PASS  tests/unit/liquid-helpers.js (18.562 s)
 PASS  tests/unit/pages.js (23.373 s)
 PASS  tests/content/liquid-line-breaks.js (20.234 s)
 PASS  tests/rendering/events.js
  ● Console

    console.log
      data should have required property 'type'

      at middleware/events.js:20:56

    console.log
      data should have required property 'url'

      at middleware/events.js:20:56

    console.log
      data.url should match format "uri"

      at middleware/events.js:20:56

    console.log
      data should have required property 'vote'

      at middleware/events.js:20:56

    console.log
      data.vote should be equal to one of the allowed values

      at middleware/events.js:20:56

    console.log
      data.email should match format "email"

      at middleware/events.js:20:56

    console.log
      data.comment should be string

      at middleware/events.js:20:56

    console.log
      data.category should be equal to one of the allowed values

      at middleware/events.js:20:56

 PASS  tests/routing/middleware/redirects/help-to-docs.js
 PASS  tests/content/graphql.js (26.05 s)
 PASS  tests/content/site-data.js (29.524 s)
 PASS  tests/rendering/header.js (35.524 s)
 PASS  tests/content/featured-links.js (34.106 s)
 PASS  tests/unit/algolia/parse-page-sections-into-records.js
 PASS  tests/rendering/breadcrumbs.js (37.474 s)
 PASS  tests/unit/permalink.js
 PASS  tests/routing/redirects.js (39.21 s)
 PASS  tests/routing/deprecated-enterprise-versions.js (39.579 s)
 PASS  tests/rendering/octicon.js
 PASS  tests/content/algolia-search.js
 PASS  tests/content/site-data-references.js (22.721 s)
 PASS  tests/content/glossary.js (10.205 s)
 PASS  tests/unit/liquid.js
 PASS  tests/routing/developer-site-redirects.js (50.53 s)
 PASS  tests/unit/enterprise-versions.js
 PASS  tests/unit/failbot.js
 PASS  tests/unit/find-page.js
 PASS  tests/content/site-tree.js (39.102 s)
 PASS  tests/content/crowdin-config.js (20.07 s)
 FAIL  tests/rendering/server.js (61.003 s)
  ● static assets › fonts

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 404

      627 | describe('static assets', () => {
      628 |   test('fonts', async () => {
    > 629 |     expect((await get('/dist/fonts/Inter-Medium.woff')).statusCode).toBe(200)
          |                                                                     ^
      630 |     expect((await get('/dist/fonts/Inter-Regular.woff')).statusCode).toBe(200)
      631 |   })
      632 | })

      at Object.<anonymous> (tests/rendering/server.js:629:69)
          at runMicrotasks (<anonymous>)

  ● stylesheets › compiles and sets the right content-type header

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 404

      725 |   it('compiles and sets the right content-type header', async () => {
      726 |     const res = await get('/dist/index.css')
    > 727 |     expect(res.statusCode).toBe(200)
          |                            ^
      728 |     expect(res.headers['content-type']).toBe('text/css; charset=UTF-8')
      729 |   })
      730 | })

      at Object.<anonymous> (tests/rendering/server.js:727:28)
          at runMicrotasks (<anonymous>)

  ● client-side JavaScript bundle › returns a 200 response

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 404

      738 | 
      739 |   it('returns a 200 response', async () => {
    > 740 |     expect(res.statusCode).toBe(200)
          |                            ^
      741 |   })
      742 | 
      743 |   it('sets the right content-type header', async () => {

      at Object.<anonymous> (tests/rendering/server.js:740:28)
          at runMicrotasks (<anonymous>)

  ● client-side JavaScript bundle › sets the right content-type header

    expect(received).toBe(expected) // Object.is equality

    Expected: "application/javascript; charset=UTF-8"
    Received: "text/html; charset=utf-8"

      742 | 
      743 |   it('sets the right content-type header', async () => {
    > 744 |     expect(res.headers['content-type']).toBe('application/javascript; charset=UTF-8')
          |                                         ^
      745 |   })
      746 | 
      747 |   // TODO: configure webpack to create production bundle in the test env

      at Object.<anonymous> (tests/rendering/server.js:744:41)
          at runMicrotasks (<anonymous>)

 PASS  tests/meta/orphan-tests.js
 PASS  tests/unit/languages.js
 PASS  tests/unit/algolia/rank.js
 PASS  tests/unit/product-names.js
 PASS  tests/content/gitignore.js
 PASS  tests/content/webhooks.js (35.523 s)
 PASS  tests/rendering/rest.js (37.34 s)
 PASS  tests/rendering/page-titles.js (39.461 s)
 PASS  tests/unit/products.js (36.661 s)
 PASS  tests/rendering/curated-homepage-links.js (35.93 s)
 PASS  tests/rendering/sidebar.js (35.062 s)
 PASS  tests/rendering/head.js (33.517 s)
(node:73877) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
 PASS  tests/routing/language-code-redirects.js (28.374 s)
 PASS  tests/unit/toc-links.js (38.5 s)
 PASS  tests/content/lint-files.js (86.962 s)
 PASS  tests/rendering/footer.js (27.285 s)
 PASS  tests/routing/top-developer-site-path-redirects.js (37.331 s)
 PASS  tests/links-and-images/links-and-images.js (112.676 s)
 PASS  tests/links-and-images/developer-links-and-images.js (259.921 s)

Summary of all failing tests
 FAIL  tests/rendering/server.js (61.003 s)
  ● static assets › fonts

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 404

      627 | describe('static assets', () => {
      628 |   test('fonts', async () => {
    > 629 |     expect((await get('/dist/fonts/Inter-Medium.woff')).statusCode).toBe(200)
          |                                                                     ^
      630 |     expect((await get('/dist/fonts/Inter-Regular.woff')).statusCode).toBe(200)
      631 |   })
      632 | })

      at Object.<anonymous> (tests/rendering/server.js:629:69)
          at runMicrotasks (<anonymous>)

  ● stylesheets › compiles and sets the right content-type header

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 404

      725 |   it('compiles and sets the right content-type header', async () => {
      726 |     const res = await get('/dist/index.css')
    > 727 |     expect(res.statusCode).toBe(200)
          |                            ^
      728 |     expect(res.headers['content-type']).toBe('text/css; charset=UTF-8')
      729 |   })
      730 | })

      at Object.<anonymous> (tests/rendering/server.js:727:28)
          at runMicrotasks (<anonymous>)

  ● client-side JavaScript bundle › returns a 200 response

    expect(received).toBe(expected) // Object.is equality

    Expected: 200
    Received: 404

      738 | 
      739 |   it('returns a 200 response', async () => {
    > 740 |     expect(res.statusCode).toBe(200)
          |                            ^
      741 |   })
      742 | 
      743 |   it('sets the right content-type header', async () => {

      at Object.<anonymous> (tests/rendering/server.js:740:28)
          at runMicrotasks (<anonymous>)

  ● client-side JavaScript bundle › sets the right content-type header

    expect(received).toBe(expected) // Object.is equality

    Expected: "application/javascript; charset=UTF-8"
    Received: "text/html; charset=utf-8"

      742 | 
      743 |   it('sets the right content-type header', async () => {
    > 744 |     expect(res.headers['content-type']).toBe('application/javascript; charset=UTF-8')
          |                                         ^
      745 |   })
      746 | 
      747 |   // TODO: configure webpack to create production bundle in the test env

      at Object.<anonymous> (tests/rendering/server.js:744:41)
          at runMicrotasks (<anonymous>)


Test Suites: 1 failed, 48 passed, 49 total
Tests:       4 failed, 9 skipped, 2 todo, 18211 passed, 18226 total
Snapshots:   0 total
Time:        260.692 s
Ran all test suites.
npm ERR! Test failed.  See above for more details.

But some of the tests do appear to be broken on main for me also. With that being said, I'm not sure when we can invest in making sure the tests work on Windows. Is it possible for you to rely on GitHub Actions to run your tests instead until this is resolved?

Also I've investigated using Docker as a way to make sure you have an identical development experience to the experience Mac and Linux users have. ... it's ready to use but rough around the edges.

More details coming soon.

[chiedo]

Connected with @jeffmcaffer. He's going to run point on this and I'm going to pair with him as needed to provide some additional engineering support!