Run test suites in azure ci #1401

jneira · 2019-10-03T08:20:41Z

fendor · 2019-10-03T08:55:26Z

Some of these issues should be fixed by #1126, so maybe we can try again to enable azure-ci testing, once that pr has landed.

jneira · 2019-10-03T11:30:54Z

I am just running the pr in azure with tests enabled: https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=212

jneira · 2019-10-07T23:03:36Z

Hi, after applying the workaround to avoid the errors compiling cabal-helper at runtime the number of failed tests has decreased sensibly:

master: https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=219
- Quite builds have only errors related with liquid haskell and even a linux build succeded: https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=219&jobId=86dfe75d-acf3-5a7d-391c-10fa0ecb27bb

The functional tests suite with the hie-bios branch hasmore fails: https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=221. Most of them are like:

haskell-ide-engine>   1) Completion.completions works
haskell-ide-engine>        uncaught exception: ConduitParserException
haskell-ide-engine>        NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "Logging notification" (BothFailed (Unexpected "ConduitParser.empty") (NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "Logging notification" (BothFailed (Unexpected "ConduitParser.empty") (NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "NotificationMessage ServerMethod PublishDiagnosticsParams" (NamedParserException "Logging notification" (BothFailed (Unexpected "ConduitParser.empty") (Unexpected "ConduitParser.empty")))))))))))))))
haskell-ide-engine> 
haskell-ide-engine>   To rerun use: --match "/Completion/completions/works/"

jneira · 2019-12-26T10:16:57Z

I've reran tests in azure with actual master: https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=334&view=results

The unit-test suite passes
The dispatcher and plugin-dispatcher suites pass
The functional-test fails due to different behaviour of received diagnostics (logs: https://gist.github.com/jneira/ae1968cc67b28d3f222ed3001cbbc052)
The wrapper-test fails due to different output of the stack command

fendor · 2019-12-26T15:30:13Z

This would be great! maybe we should start poking members of the haskell organizations to enable azure for this project?

jneira · 2019-12-27T15:45:48Z

Absolutely, do we know who has the permissions to do it, @alanz, @bubba?

alanz · 2019-12-27T17:54:39Z

I think the haskell committee has control of this. Not sure who specifically.

jneira · 2019-12-27T20:34:22Z

Ok, maybe we can wait to have the azure setup complete, including green test suites and deploy.

EDIT: i get linux and macos to run the entire test suite succesfully: https://dev.azure.com/jneira/0fa2df97-1d6d-4a00-935a-03a77e7170b8/_build/results?buildId=344&view=logs&j=d2c7b64e-54d7-53d5-7f80-77915c458472

jneira · 2020-01-10T08:12:16Z

Update:

Windows wrapper-test errors:

haskell-ide-engine>   test\wrapper\HieWrapper.hs:18:9: 
haskell-ide-engine>   1) version checking picks up a stack.yaml with 8.2.1
haskell-ide-engine>        expected: "8.2.1"
haskell-ide-engine>         but got: ""
haskell-ide-engine> 
haskell-ide-engine>   To rerun use: --match "/version checking/picks up a stack.yaml with 8.2.1/"
haskell-ide-engine> 
haskell-ide-engine>   test\wrapper\HieWrapper.hs:23:9: 
haskell-ide-engine>   2) version checking picks up a stack.yaml with 8.2.2
haskell-ide-engine>        expected: "8.2.2"
haskell-ide-engine>         but got: "8.6.5"
haskell-ide-engine> 
haskell-ide-engine>   To rerun use: --match "/version checking/picks up a stack.yaml with 8.2.2/"

windows dispatcher-test errors:

Failures:

  test\dispatcher\Main.hs:157:5: 
  1) functional dispatch defers responses until module is loaded
       uncaught exception: ErrorCall
       Prelude.head: empty list

  To rerun use: --match "/functional dispatch/defers responses until module is loaded/"

  test\dispatcher\Main.hs:205:7: 
  2) functional dispatch instantly responds to deferred requests if cache is available
       expected: ("references",Just [DocumentHighlight {_range = Range {_start = Position {_line = 7, _character = 0}, _end = Position {_line = 7, _character = 2}}, _kind = Just HkWrite},DocumentHighlight {_range = Range {_start = Position {_line = 7, _character = 0}, _end = Position {_line = 7, _character = 2}}, _kind = Just HkWrite},DocumentHighlight {_range = Range {_start = Position {_line = 5, _character = 6}, _end = Position {_line = 5, _character = 8}}, _kind = Just HkRead},DocumentHighlight {_range = Range {_start = Position {_line = 7, _character = 0}, _end = Position {_line = 7, _character = 2}}, _kind = Just HkWrite},DocumentHighlight {_range = Range {_start = Position {_line = 7, _character = 0}, _end = Position {_line = 7, _character = 2}}, _kind = Just HkWrite},DocumentHighlight {_range = Range {_start = Position {_line = 5, _character = 6}, _end = Position {_line = 5, _character = 8}}, _kind = Just HkRead}])
        but got: ("req2",Nothing)

  To rerun use: --match "/functional dispatch/instantly responds to deferred requests if cache is available/"

  test\dispatcher\Main.hs:257:7: 
  3) functional dispatch returns hints as diagnostics
       expected: ("r5",Just (PublishDiagnosticsParams {_uri = Uri {getUri = "file:///D:/a/1/s/test/testdata/FuncTest.hs"}, _diagnostics = List [Diagnostic {_range = Range {_start = Position {_line = 9, _character = 6}, _end = Position {_line = 10, _character = 18}}, _severity = Just DsInfo, _code = Just (StringValue "Redundant do"), _source = Just "hlint", _message = "Redundant do\nFound:\n  do putStrLn \"hello\"\nWhy not:\n  putStrLn \"hello\"\n", _relatedInformation = Nothing}]}))
        but got: ("references",Nothing)

  To rerun use: --match "/functional dispatch/returns hints as diagnostics/"

  test\dispatcher\Main.hs:297:7: 
  4) functional dispatch instantly responds to failed modules with no cache with the default
       expected: ("req7",Just [])
        but got: ("r5",Nothing)

  To rerun use: --match "/functional dispatch/instantly responds to failed modules with no cache with the default/"

windows func-test error for ghc-8.6.4:

haskell-ide-engine>   1) Hover.hover works
haskell-ide-engine>        predicate failed on: HoverContents (MarkupContent {_kind = MkMarkdown, _value = "\n```haskell\nsum :: [Int] -> Int\n```\n```haskell\nsum :: forall (t :: * -> *) a. (Foldable t, Num a) => t a -> a\n```\n\n\n\nbase Data.Foldable"})
haskell-ide-engine> 
haskell-ide-engine>   To rerun use: --match "/Hover/hover/works/"

windows func-test error with cabal anf ghc-8.4.4

Failures:

  src\Language\Haskell\LSP\Test\Decoding.hs:121:32: 
  1) Completion.completions works
       uncaught exception: ErrorCall
       Couldn't decode response for the request type: Shutdown
       "{\"result\":{\"capabilities\":{\"typeDefinitionProvider\":true,\"foldingRangeProvider\":false,\"textDocumentSync\":{\"openClose\":true,\"change\":2,\"willSave\":false,\"willSaveWaitUntil\":false,\"save\":{\"includeText\":false}},\"workspace\":{},\"implementationProvider\":true,\"documentRangeFormattingProvider\":true,\"documentHighlightProvider\":true,\"executeCommandProvider\":{\"commands\":[\"1904:applyrefact:applyOne\",\"1904:applyrefact:applyAll\",\"1904:generic:type\",\"1904:ghcmod:check\",\"1904:ghcmod:type\",\"1904:ghcmod:casesplit\",\"1904:hsimport:import\",\"1904:package:add\",\"1904:pragmas:addPragma\"]},\"renameProvider\":false,\"colorProvider\":false,\"definitionProvider\":true,\"hoverProvider\":true,\"codeActionProvider\":true,\"completionProvider\":{\"triggerCharacters\":[\".\"],\"resolveProvider\":true},\"documentSymbolProvider\":true,\"documentFormattingProvider\":true,\"referencesProvider\":true}},\"jsonrpc\":\"2.0\",\"id\":0}"
       CallStack (from HasCallStack):
         error, called at src\Language\Haskell\LSP\Test\Decoding.hs:121:32 in lsp-test-0.10.0.0-2370d2b00c7c9eae10f8c415728453dde8ccc33a:Language.Haskell.LSP.Test.Decoding

  To rerun use: --match "/Completion/completions/works/"

Commenting the previous test triggers another functional errors, so i've deactivated func-test for win+cabal+ghc-8.4.4

macos error in dispatcher-test for ghc-8.4

haskell-ide-engine> dispatcher-test: 
haskell-ide-engine> lookupSymbol failed in relocateSection (RELOC_GOT)
haskell-ide-engine> /Users/runner/runners/2.163.1/work/1/s/.stack/programs/x86_64-osx/ghc-8.4.3/lib/ghc-8.4.3/integer-gmp-1.0.2.0/HSinteger-gmp-1.0.2.0.o: unknown symbol `___gmp_rands'

Errors in unis-test involving cabal helper reported here: DanielG/cabal-helper#91

* Enabled test-suites: * Linux, with stack or cabal, all test suites are enabled * Macos: * ghc-8.6.*: all suites enabled * ghc-8.4.*: func-test and dispatcher-test disabled * Windows * unit-test enabled * Except CabalHelper ones for ghc-8.6.4 * func-test enabled with almost all tests * cabal job: * Aadded one for linux Make progress towards #1401 Closes #1431

jneira · 2020-01-24T11:50:43Z

Intermitent errors:
- stack windows jobs fail with C:\path\to\ghc\strip.exe: unable to rename 'D:\sr\snapshots\add677e6\bin\happy.exe'; reason: File exists. I've hit it in local, and you have to delete the snapshot
- ~~stack windows job fail installing the specific ghc for liquidhaskell: error decompressing tar~~
  - resolved installing ghc's in the same step, at the beginning

jneira · 2020-02-03T22:46:02Z

After #1601 there are a new failing unit test in windows:

All windows jobs using ghc-8.6 (stack and cabal)

Failures:

  test/unit\\ApplyRefactPluginSpec.hs:42:5: 
  1) ApplyRefactPlugin, apply-refact plugin, apply-refact plugin commands, applies one hint only
       uncaught exception: GhcException
       unit-test.exe: could not detect mingw toolchain

  To rerun use: --match "/ApplyRefactPlugin/apply-refact plugin/apply-refact plugin commands/applies one hint only/"

All windows ghc-8.4 jobs

Failures:

  test/unit\\ApplyRefactPluginSpec.hs:134:5: 
  1) ApplyRefactPlugin, apply-refact plugin, apply-refact plugin commands, respects hlint config files in project root dir
       uncaught exception: IOException of type NoSuchThing
       .\settings: openFile: does not exist (No such file or directory)

  To rerun use: --match "/ApplyRefactPlugin/apply-refact plugin/apply-refact plugin commands/respects hlint config files in project root dir/"

I cant reproduce locally 😟
The first error is thrown by ghc here: https://gitlab.haskell.org/ghc/ghc/blob/4898df1cc25132dc9e2599d4fa4e1bbc9423cda5/compiler/main/SysTools/BaseDir.hs

jneira self-assigned this Jan 3, 2020

jneira mentioned this issue Jan 12, 2020

Enable all working test suites and add linux-cabal job in Azure #1571

Merged

jneira added build Continuous integration and building testing labels Jan 14, 2020

jneira removed their assignment Sep 14, 2020

Tarmean mentioned this issue Oct 6, 2020

Retrie fails on Windows with could not detect mingw toolchain haskell/haskell-language-server#471

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Run test suites in azure ci #1401

Run test suites in azure ci #1401

jneira commented Oct 3, 2019 •

edited

Loading

fendor commented Oct 3, 2019

Uh oh!

jneira commented Oct 3, 2019

Uh oh!

jneira commented Oct 7, 2019 •

edited

Loading

Uh oh!

jneira commented Dec 26, 2019 •

edited

Loading

Uh oh!

fendor commented Dec 26, 2019

Uh oh!

jneira commented Dec 27, 2019 •

edited

Loading

Uh oh!

alanz commented Dec 27, 2019

Uh oh!

jneira commented Dec 27, 2019 •

edited

Loading

Uh oh!

jneira commented Jan 10, 2020 •

edited

Loading

Uh oh!

jneira commented Jan 24, 2020 •

edited

Loading

Uh oh!

jneira commented Feb 3, 2020 •

edited

Loading

Uh oh!

Run test suites in azure ci #1401

Run test suites in azure ci #1401

Comments

jneira commented Oct 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

fendor commented Oct 3, 2019

Uh oh!

jneira commented Oct 3, 2019

Uh oh!

jneira commented Oct 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jneira commented Dec 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fendor commented Dec 26, 2019

Uh oh!

jneira commented Dec 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alanz commented Dec 27, 2019

Uh oh!

jneira commented Dec 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jneira commented Jan 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jneira commented Jan 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jneira commented Feb 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jneira commented Oct 3, 2019 •

edited

Loading

jneira commented Oct 7, 2019 •

edited

Loading

jneira commented Dec 26, 2019 •

edited

Loading

jneira commented Dec 27, 2019 •

edited

Loading

jneira commented Dec 27, 2019 •

edited

Loading

jneira commented Jan 10, 2020 •

edited

Loading

jneira commented Jan 24, 2020 •

edited

Loading

jneira commented Feb 3, 2020 •

edited

Loading