VS Python analysis engine integration (#1231)

* Undo changes

* Test fixes

* Increase timeout

* Remove double event listening

* Remove test

* Revert "Remove test"

This reverts commit e240c3f.

* Revert "Remove double event listening"

This reverts commit af573be.

* Undo changes

* Test fixes

* .NET Core check

* Better find dotnet

* Fix pip test

* Linting tests

* Undo accidental changes

* Add clone and build PTVS

* Appveyor PTVS build

* Fix slashes

* Enable build

* Try absolute path

* Fix xcopy switch

* Activate Analysis Engine test on Appveyor

* Temporary only run new tests

* Disable PEP hint tests

* Test fix

* Disable appveyor build and tests for PTVS for now

* Remove analysis engine test from the set

* Remove VS image for now

* Build/sign VSXI project

* Run vsce from cmd

* Rename

* Abs path vsce

* Path

* Move project

* Ignore publishing project

* Try csproj

* Add framework

* Ignore build output folder

* Package before build

* Try batch instead of PS

* Fix path quotes

* #1096 The if statement is automatically formatted incorrectly

* Merge fix

* Add more tests

* More tests

* Typo

* Test

* Also better handle multiline arguments

* Changes lost on squash

* More lost changes

* Restore Jedi/PTVS setting

* Update tests to new PTVS

* Signature tests

* Add PTVS tests task

* Analysis Engine contribution

* Add Mac/Linux info

* Disable csproj build

* Add unzip to dependencies

* Minor fixes to doc

* Change setting type to bool

* Report progress on status bar

* Simplify

* CR feedback

* Fix launching fx-independent code on Mac/Linux

* Add title

Author: Mikhail Arkhipov

.gitignore

@@ -16,3 +16,5 @@ coverage/
 pythonFiles/experimental/ptvsd/**
 debug_coverage*/**
 analysis/**
+bin/**
+obj/**

.vscode/launch.json

@@ -87,6 +87,26 @@
             ],
             "preLaunchTask": "Compile"
         },
+        {
+            "name": "Launch Analysis Engine Tests",
+            "type": "extensionHost",
+            "request": "launch",
+            "runtimeExecutable": "${execPath}",
+            "args": [
+                "${workspaceFolder}/src/test",
+                "--extensionDevelopmentPath=${workspaceFolder}",
+                "--extensionTestsPath=${workspaceFolder}/out/test"
+            ],
+            "stopOnEntry": false,
+            "sourceMaps": true,
+            "outFiles": [
+                "${workspaceFolder}/out/**/*.js"
+            ],
+            "preLaunchTask": "Compile",
+            "env": {
+                "VSC_PYTHON_ANALYSIS": "1"
+            }
+        },
         {
             "name": "Launch Tests (with code coverage)",
             "type": "extensionHost",
@@ -114,4 +134,4 @@
             ]
         }
     ]
-}
+}

.vscodeignore

@@ -37,3 +37,11 @@ CODING_STANDARDS.md
 CONTRIBUTING.md
 news/**
 debug_coverage*/**
+analysis/publish*.*
+vscode-python-signing.*
+packageExtension.cmd
+bin/**
+obj/**
+BuildOutput/**
+
+

CONTRIBUTING - PYTHON_ANALYSIS.md

@@ -0,0 +1,61 @@
+# Contributing to Microsoft Python Analysis Engine
+[![Contributing to Python Tools for Visual Studio](https://github.com/Microsoft/PTVS/blob/master/CONTRIBUTING.md)]
+
+[![Build Status (Travis)](https://travis-ci.org/Microsoft/vscode-python.svg?branch=master)](https://travis-ci.org/Microsoft/vscode-python) [![Build status (AppVeyor)](https://ci.appveyor.com/api/projects/status/s0pt8d79gqw222j7?svg=true)](https://ci.appveyor.com/project/DonJayamanne/vscode-python-v3vd6) [![codecov](https://codecov.io/gh/Microsoft/vscode-python/branch/master/graph/badge.svg)](https://codecov.io/gh/Microsoft/vscode-python)
+
+
+## Contributing a pull request
+
+### Prerequisites
+
+1. .NET Core 2.0+ SDK
+   - [Windows](https://www.microsoft.com/net/learn/get-started/windows)
+   - [Mac OS](https://www.microsoft.com/net/learn/get-started/macos)
+   - [Linux](https://www.microsoft.com/net/learn/get-started/linux/rhel)
+2. C# Extension to VS Code (all platforms)
+3. Python 2.7
+4. Python 3.6
+
+*Alternative:* [Visual Studio 2017](https://www.visualstudio.com/downloads/) (Windows only) with .NET Core and C# Workloads. Community Edition is free and is fully functional.
+
+### Setup
+
+```shell
+git clone https://github.com/microsoft/ptvs
+cd Python/Product/VsCode/AnalysisVsc
+dotnet build
+```
+
+Visual Studio 2017:
+1. Open solution in Python/Product/VsCode
+2. Build AnalysisVsc project
+3. Binaries arrive in *Python/BuildOutput/VsCode/raw*
+4. Delete contents of the *analysis* folder in the Python Extension folder
+5. Copy *.dll, *.pdb, *.json fron *Python/BuildOutput/VsCode/raw* to *analysis*
+
+### Debugging code in Python Extension to VS Code
+Folow regular TypeScript debugging steps
+
+### Debugging C# code in Python Analysis Engine
+1. Launch another instance of VS Code
+2. Open Python/Product/VsCode/AnalysisVsc folder
+3. Python Analysis Engine code is in *Python/Product/VsCode/Analysis*
+4. Run extension from VS Code
+5. In the instance with C# code select Dotnet Attach launch task.
+6. Attach to *dotnet* process running *Microsoft.PythonTools.VsCode.dll*
+
+On Windows you can also attach from Visual Studio 2017.
+
+### Validate your changes
+
+1. Build C# code
+2. Copy binaries to *analysis* folder
+3. Use the `Launch Extension` launch option.
+
+### Unit Tests
+1. Run the Unit Tests via the `Launch Analysis Engine Tests`.
+2. On Windows you can also open complete PTVS solution in Visual Studio and run its tests (or at least the Analysis section).
+
+
+### Coding Standards
+See [![Contributing to Python Tools for Visual Studio](https://github.com/Microsoft/PTVS/blob/master/CONTRIBUTING.md)]

CONTRIBUTING.md

@@ -2,7 +2,8 @@
 
 [![Build Status (Travis)](https://travis-ci.org/Microsoft/vscode-python.svg?branch=master)](https://travis-ci.org/Microsoft/vscode-python) [![Build status (AppVeyor)](https://ci.appveyor.com/api/projects/status/s0pt8d79gqw222j7?svg=true)](https://ci.appveyor.com/project/DonJayamanne/vscode-python-v3vd6) [![codecov](https://codecov.io/gh/Microsoft/vscode-python/branch/master/graph/badge.svg)](https://codecov.io/gh/Microsoft/vscode-python)
 
-
+# Contributing to Microsoft Python Analysis Engine
+[![Contributing to Python Analysis Engine](https://github.com/Microsoft/vscode-python/blob/master/CONTRIBUTING - PYTHON_ANALYSIS.md)]
 
 ## Contributing a pull request
 

appveyor.yml

@@ -1,3 +1,6 @@
+#image: Visual Studio 2017
+#shallow_clone: true
+
 environment:
   matrix:
     - PYTHON: "C:\\Python36"
@@ -20,11 +23,23 @@ install:
   - python -m easy_install -U setuptools
   - "%PYTHON%/Scripts/pip.exe install --upgrade -r requirements.txt"
 
+build: off
+# build_script:
+#   - git clone https://github.com/MikhailArkhipov/PTVS.git c:/projects/PTVS
+#   - "cd c:\\projects\\PTVS"
+#   - git checkout origin/vsc
+#   - "cd Python\\Product\\VSCode\\AnalysisVsc"
+#   - "dotnet --info"
+#   - "dotnet build"
+#   - "cd c:\\projects\\vscode-python"
+#   - "xcopy /S /I c:\\projects\\PTVS\\BuildOutput\\VsCode\\raw analysis"
+
 test_script:
   - yarn run clean
   - yarn run vscode:prepublish
   - yarn run testDebugger --silent
   - yarn run testSingleWorkspace --silent
   - yarn run testMultiWorkspace --silent
+  # - yarn run testAnalysisEngine --silent
+
 
-build: off

package.json

@@ -1052,6 +1052,12 @@
                     "default": "${workspaceFolder}/.env",
                     "scope": "resource"
                 },
+                "python.jediEnabled": {
+                    "type": "boolean",
+                    "default": true,
+                    "description": "Enables Jedi as IntelliSense engine instead of Microsoft Python Analysis Engine.",
+                    "scope": "resource"
+                },
                 "python.jediPath": {
                     "type": "string",
                     "default": "",
@@ -1727,6 +1733,7 @@
         "testDebugger": "node ./out/test/debuggerTest.js",
         "testSingleWorkspace": "node ./out/test/standardTest.js",
         "testMultiWorkspace": "node ./out/test/multiRootTest.js",
+        "testAnalysisEngine": "node ./out/test/analysisEngineTest.js",
         "precommit": "node gulpfile.js",
         "lint-staged": "node gulpfile.js",
         "lint": "tslint src/**/*.ts -t verbose",
@@ -1750,6 +1757,7 @@
         "opn": "^5.1.0",
         "pidusage": "^1.2.0",
         "reflect-metadata": "^0.1.12",
+        "request-progress": "^3.0.0",
         "rxjs": "^5.5.2",
         "semver": "^5.4.1",
         "sudo-prompt": "^8.0.0",
@@ -1759,6 +1767,7 @@
         "uint64be": "^1.0.1",
         "unicode": "^10.0.0",
         "untildify": "^3.0.2",
+        "unzip": "^0.1.11",
         "vscode-debugadapter": "^1.0.1",
         "vscode-debugprotocol": "^1.0.1",
         "vscode-extension-telemetry": "^0.0.14",

packageExtension.cmd

@@ -0,0 +1 @@
+%1\vsce package --out %2