docs: Fix up README, polish deduction and intro

sdboyer · sdboyer · commit fd6c461b9cc8 · 2018-01-16T23:47:42.000-08:00
diff --git a/README.md b/README.md
@@ -8,38 +8,15 @@
 
 ## Dep
 
-`dep` is a prototype dependency management tool for Go. It requires Go 1.8 or newer to compile.
+`dep` is a prototype dependency management tool for Go. It requires Go 1.8 or newer to compile. **`dep` is safe for production use.**
 
 `dep` is the official _experiment_, but not yet the official tool. Check out the [Roadmap](https://github.com/golang/dep/wiki/Roadmap) for more on what this means!
 
-## Current status
+For guides and reference materials about `dep`, see [the documentation](https://golang.github.io/dep).
 
-`dep` is safe for production use. That means two things:
+## Installation
 
-* Any valid metadata file (`Gopkg.toml` and `Gopkg.lock`) will be readable and considered valid by any future version of `dep`.
-* The CLI UI is mostly stable. `dep init` and `dep ensure` are mostly set; `dep status` is likely to change a fair bit, and `dep prune` is [going to be absorbed into `dep ensure`](https://github.com/golang/dep/issues/944).
-
-That said, keep in mind the following:
-
-* `dep init` on an existing project can be a rocky experience - we try to automatically convert from other tools' metadata files, and that process is often complex and murky. Once your project is converted and you're using `dep ensure`, its behavior is quite stable.
-* `dep` still has nasty bugs, but in general these are comparable to or fewer than other tools out there.
-* `dep` is [pretty slow right now](https://github.com/golang/dep/blob/master/docs/FAQ.md#why-is-dep-slow), especially on the first couple times you run it. Just know that there is a _lot_ of headroom for improvement, and work is actively underway.
-* `dep` is still changing rapidly. If you need stability (e.g. for CI), it's best to rely on a released version, not tip.
-* `dep`'s exported API interface will continue to change in unpredictable, backwards-incompatible ways until we tag a v1.0.0 release.
-
-## Context
-
-- [The Saga of Go Dependency Management](https://blog.gopheracademy.com/advent-2016/saga-go-dependency-management/)
-- Official Google Docs
-  - [Go Packaging Proposal Process](https://docs.google.com/document/d/18tNd8r5DV0yluCR7tPvkMTsWD_lYcRO7NhpNSDymRr8/edit)
-  - [User Stories](https://docs.google.com/document/d/1wT8e8wBHMrSRHY4UF_60GCgyWGqvYye4THvaDARPySs/edit)
-  - [Features](https://docs.google.com/document/d/1JNP6DgSK-c6KqveIhQk-n_HAw3hsZkL-okoleM43NgA/edit)
-  - [Design Space](https://docs.google.com/document/d/1TpQlQYovCoX9FkpgsoxzdvZplghudHAiQOame30A-v8/edit)
-- [Frequently Asked Questions](docs/FAQ.md)
-
-## Setup
-
-Grab the latest binary from the [releases](https://github.com/golang/dep/releases) page.
+Unless you are hacking on dep, it is strongly recommended that you use a released version. You can grab the latest binary from the [releases](https://github.com/golang/dep/releases) page.
 
 On macOS you can install or upgrade to the latest released version with Homebrew:
 
@@ -54,217 +31,12 @@ If you're interested in hacking on `dep`, you can install via `go get`:
 go get -u github.com/golang/dep/cmd/dep
 ```
 
-To start managing dependencies using dep, run the following from your project's root directory:
-
-```sh
-$ dep init
-```
-
-This does the following:
-
-1. Look for [existing dependency management files](docs/FAQ.md#what-external-tools-are-supported) to convert
-1. Check if your dependencies use dep
-1. Identify your dependencies
-1. Back up your existing `vendor/` directory (if you have one) to
-`_vendor-TIMESTAMP/`
-1. Pick the highest compatible version for each dependency
-1. Generate [`Gopkg.toml`](docs/Gopkg.toml.md) ("manifest") and `Gopkg.lock` files
-1. Install the dependencies in `vendor/`
-
-## Usage
-
-There is one main subcommand you will use: `dep ensure`. `ensure` first checks that `Gopkg.lock` is consistent with `Gopkg.toml` and the `import`s in your code. If any
-changes are detected, `dep`'s solver works out a new `Gopkg.lock`. Then, `dep` checks if the contents of `vendor/` are what `Gopkg.lock` (the new one if applicable, else the existing one) says it should be, and rewrites `vendor/` as needed to bring it into line.
-
-In essence, `dep ensure` [works in two phases to keep four buckets of state in sync](https://youtu.be/5LtMb090AZI?t=20m4s):
-
-<img width="463" alt="states-flow" src="https://user-images.githubusercontent.com/21599/29223886-22dd2578-7e96-11e7-8b51-3637b9ddc715.png">
-
-
-_Note: until we ship [vendor verification](https://github.com/golang/dep/issues/121), we can't efficiently perform the `Gopkg.lock` <-> `vendor/` comparison, so `dep ensure` unconditionally regenerates all of `vendor/` to be safe._
-
-`dep ensure` is safe to run early and often. See the help text for more detailed
-usage instructions.
-
-```sh
-$ dep help ensure
-```
-
-### Installing dependencies
-
-(if your `vendor/` directory isn't [checked in with your code](docs/FAQ.md#should-i-commit-my-vendor-directory))
-
-<!-- may change with https://github.com/golang/dep/pull/489 -->
-
-```sh
-$ dep ensure
-```
-
-If a dependency already exists in your `vendor/` folder, dep will ensure it
-matches the constraints from the manifest. If the dependency is missing from
-`vendor/`, the latest version allowed by your manifest will be installed.
-
-### Adding a dependency
-
-```sh
-$ dep ensure -add github.com/foo/bar
-```
-
-This adds a version constraint to your `Gopkg.toml`, and updates `Gopkg.lock` and `vendor/`. Now, import and use the package in your code! ✨
-
-`dep ensure -add` has some subtle behavior variations depending on the project or package named, and the state of your tree. See `dep ensure -examples` for more information.
-
-### Changing dependencies
-
-If you want to:
-
-* Change the allowed `version`/`branch`/`revision`
-* Switch to using a fork
-
-for one or more dependencies, do the following:
-
-1. Manually edit your `Gopkg.toml`.
-1. Run
-
-    ```sh
-    $ dep ensure
-    ```
-
-### Checking the status of dependencies
-
-Run `dep status` to see the current status of all your dependencies.
-
-```sh
-$ dep status
-PROJECT                             CONSTRAINT     VERSION        REVISION  LATEST
-github.com/Masterminds/semver       branch 2.x     branch 2.x     139cc09   c2e7f6c
-github.com/Masterminds/vcs          ^1.11.0        v1.11.1        3084677   3084677
-github.com/armon/go-radix           *              branch master  4239b77   4239b77
-```
-
-On top of that, if you have added new imports to your project or modified `Gopkg.toml` without running `dep ensure` again, `dep status` will tell you there is a mismatch between `Gopkg.lock` and the current status of the project.
-
-```sh
-$ dep status
-Lock inputs-digest mismatch due to the following packages missing from the lock:
-
-PROJECT                         MISSING PACKAGES
-github.com/Masterminds/goutils  [github.com/Masterminds/goutils]
-
-This happens when a new import is added. Run `dep ensure` to install the missing packages.
-```
-
-As `dep status` suggests, run `dep ensure` to update your lockfile. Then run `dep status` again, and the lock mismatch should go away.
-
-### Visualizing dependencies
-
-Generate a visual representation of the dependency tree by piping the output of `dep status -dot` to [graphviz](http://www.graphviz.org/).
-#### Linux
-```
-$ sudo apt-get install graphviz
-$ dep status -dot | dot -T png | display
-```
-#### MacOS
-```
-$ brew install graphviz
-$ dep status -dot | dot -T png | open -f -a /Applications/Preview.app
-```
-#### Windows
-```
-> choco install graphviz.portable
-> dep status -dot | dot -T png -o status.png; start status.png
-```
-<p align="center"><img src="docs/img/StatusGraph.png"></p>
-
-### Updating dependencies
-
-Updating brings the version of a dependency in `Gopkg.lock` and `vendor/` to the latest version allowed by the constraints in `Gopkg.toml`.
-
-You can update just a targeted subset of dependencies (recommended):
-
-```sh
-$ dep ensure -update github.com/some/project github.com/other/project
-$ dep ensure -update github.com/another/project
-```
-
-Or you can update all your dependencies at once:
-
-```sh
-$ dep ensure -update
-```
-
-"Latest" means different things depending on the type of constraint in use. If you're depending on a `branch`, `dep` will update to the latest tip of that branch. If you're depending on a `version` using [a semver range](#semantic-versioning), it will update to the latest version in that range.
-
-### Removing dependencies
-
-1. Remove the `import`s and all usage from your code.
-1. Remove `[[constraint]]` rules from `Gopkg.toml` (if any).
-1. Run
-
-    ```sh
-    $ dep ensure
-    ```
-
-### Testing changes to a dependency
-
-Making changes in your `vendor/` directory directly is not recommended, as dep
-will overwrite any changes. Instead:
-
-1. Delete the dependency from the `vendor/` directory.
-
-    ```sh
-    rm -rf vendor/<dependency>
-    ```
-
-1. Add that dependency to your `GOPATH`, if it isn't already.
-
-    ```sh
-    $ go get <dependency>
-    ```
-
-1. Modify the dependency in `$GOPATH/src/<dependency>`.
-1. Test, build, etc.
-
-Don't run `dep ensure` until you're done. `dep ensure` will reinstall the
-dependency into `vendor/` based on your manifest, as if you were installing from
-scratch.
-
-This solution works for short-term use, but for something long-term, take a look
-at [virtualgo](https://github.com/GetStream/vg).
-
-To test out code that has been pushed as a new version, or to a branch or fork,
-see [changing dependencies](#changing-dependencies).
-
-## Semantic Versioning
-
-`dep ensure` uses an external [semver library](https://github.com/Masterminds/semver) to interpret the version constraints you specify in the manifest. The comparison operators are:
-
-* `=`: equal
-* `!=`: not equal
-* `>`: greater than
-* `<`: less than
-* `>=`: greater than or equal to
-* `<=`: less than or equal to
-* `-`: literal range. Eg: 1.2 - 1.4.5 is equivalent to >= 1.2, <= 1.4.5
-* `~`: minor range. Eg: ~1.2.3 is equivalent to >= 1.2.3, < 1.3.0
-* `^`: major range. Eg: ^1.2.3 is equivalent to >= 1.2.3, < 2.0.0
-* `[xX*]`: wildcard. Eg: 1.2.x is equivalent to >= 1.2.0, < 1.3.0
-
-You might, for example, include a constraint in your manifest that specifies `version = "=2.0.0"` to pin a dependency to version 2.0.0, or constrain to minor releases with: `version = "2.*"`. Refer to the [semver library](https://github.com/Masterminds/semver) documentation for more info.
-
-**Note**: When you specify a version *without an operator*, `dep` automatically uses the `^` operator by default. `dep ensure` will interpret the given version as the min-boundary of a range, for example:
-
-* `1.2.3` becomes the range `>=1.2.3, <2.0.0`
-* `0.2.3` becomes the range `>=0.2.3, <0.3.0`
-* `0.0.3` becomes the range `>=0.0.3, <0.1.0`
-
 ## Feedback
 
 Feedback is greatly appreciated.
 At this stage, the maintainers are most interested in feedback centered on the user experience (UX) of the tool.
 Do you have workflows that the tool supports well, or doesn't support at all?
 Do any of the commands have surprising effects, output, or results?
-Please check the existing issues and [FAQ](docs/FAQ.md) to see if your feedback has already been reported.
 If not, please file an issue, describing what you did or wanted to do, what you expected to happen, and what actually happened.
 
 ## Contributing
diff --git a/docs/Gopkg.toml.md b/docs/Gopkg.toml.md
@@ -120,21 +120,34 @@ system2-data = "value that is used by another system"
 
 
 
-----
+## Version rules
 
+Version rules can be used in either `[[constraint]]` or `[[override]]` stanzas. There are three types of version rules - `version`, `branch`, and `revision`. At most one of the three types can be specified.
 
+### `version`
 
-## Version rules
+`version` is a property of `constraint`s and `override`s. It is used to specify version constraint of a specific dependency. It can be used to target an arbitrary VCS tag, or a semantic version, or a range of semantic versions.
 
-Version rules can be used in either `[[constraint]]` or `[[override]]` stanzas. There are three types of version rules - `version`, `branch`, and `revision`. At most one of the three types can be specified.
+Specifying semantic version ranges can be done using the following operators:
 
-TODOOOOOOOOOOOOO
+* `=`: equal
+* `!=`: not equal
+* `>`: greater than
+* `<`: less than
+* `>=`: greater than or equal to
+* `<=`: less than or equal to
+* `-`: literal range. Eg: 1.2 - 1.4.5 is equivalent to >= 1.2, <= 1.4.5
+* `~`: minor range. Eg: ~1.2.3 is equivalent to >= 1.2.3, < 1.3.0
+* `^`: major range. Eg: ^1.2.3 is equivalent to >= 1.2.3, < 2.0.0
+* `[xX*]`: wildcard. Eg: 1.2.x is equivalent to >= 1.2.0, < 1.3.0
 
-### `version`
+You might, for example, include a rule that specifies `version = "=2.0.0"` to pin a dependency to version 2.0.0, or constrain to minor releases with: `version = "~2.1.0"`. Refer to the [semver library](https://github.com/Masterminds/semver) documentation for more info.
 
-`version` is a property of `constraint`s and `override`s. It is used to specify version constraint of a specific dependency.
+**Note**: When you specify a version *without an operator*, `dep` automatically uses the `^` operator by default. `dep ensure` will interpret the given version as the min-boundary of a range, for example:
 
-Internally, dep uses [Masterminds/semver](https://github.com/Masterminds/semver) to work with semver versioning.
+* `1.2.3` becomes the range `>=1.2.3, <2.0.0`
+* `0.2.3` becomes the range `>=0.2.3, <0.3.0`
+* `0.0.3` becomes the range `>=0.0.3, <0.1.0`
 
 `~` and `=` operators can be used with the versions. When a version is specified without any operator, `dep` automatically adds a caret operator, `^`. The caret operator pins the left-most non-zero digit in the version. For example:
 ```
@@ -150,11 +163,21 @@ To pin a version of direct dependency in manifest, prefix the version with `=`.
   version = "=0.8.0"
 ```
 
-[Why is dep ignoring a version constraint in the manifest?](FAQ.md#why-is-dep-ignoring-a-version-constraint-in-the-manifest)
+### `branch`
+
+Using a `branch` constraint will cause dep to use the named branch (e.g., `branch = "master"`) for a particular dependency. The revision at the tip of the branch will be recorded into `Gopkg.lock`, and almost always remain the same until a change is requested, via `dep ensure -update`.
+
+In general, you should prefer semantic versions to branches, when a project has made them available.
+
+### `revision`
+
+A `revision` is the underlying immutable identifier - like a git commit SHA1. While it is allowed to constrain to a `revision`, doing so is almost always an antipattern. 
+
+Usually, folks are inclined to pin to a revision because they feel it will somehow improve their project's reproducibility. That is not a good reason. `Gopkg.lock` provides reproducibility. Only use `revision` if you have a good reason to believe that _no_ other version of that dependency _could_ work.
 
 # Example
 
-Here's an example of a sample Gopkg.toml with most of the elements
+A sample  `Gopkg.toml` with most elements present:
 
 ```toml
 required = ["github.com/user/thing/cmd/thing"]
diff --git a/docs/deduction.md b/docs/deduction.md
@@ -7,37 +7,20 @@ Deduction is dep's algorithm for looking at an import path and determining the p
 - `github.com/golang/dep/gps` -> `github.com/golang/dep`
 - `bitbucket.org/foo/bar/baz` -> `bitbucket.org/foo/bar`
 
-The set of hosts supported by static deduction are the same as [those supported by `go get`]().
+The set of hosts supported by static deduction are the same as [those supported by `go get`](https://golang.org/cmd/go/#hdr-Remote_import_paths):
 
-If the static logic cannot identify the root for a given import path, the algorithm continues to a dynamic component, where it makes an HTTP(S) request to the import path, and a server is expected to return a response that indicates the root import path. This, again, is equivalent to the behavior of `go get`.
+* GitHub
+* Bitbucket
+* Launchpad
+* IBM DevOps Services
 
+In addition, dep also handles [gopkg.in](http://gopkg.in) directly with static deduction because, owing to internal implementation details, it is the easiest way of also attaching filters to adapt the versioning semantics of gopkg.in import paths into dep's versioning model. This turns out fine, as gopkg.in's rules mapping rules are themselves entirely static.
 
+If the static logic cannot identify the root for a given import path, the algorithm continues to a dynamic component: dep makes an HTTP(S) request to the import path, and a server is expected to send back the root import path embedded within the HTML response. Again, this directly emulates the behavior of `go get`.
 
 Import path deduction is applied to all of the following:
 
 * `import` statements found in all `.go` files
-* Import paths named in the [`required`](gopkg.toml.md#required) property in `Gopkg.toml`
-* `name` properties in both [`[[constraint]]`](Gopkg.toml.md#constraint) and [`[[override]]`](Gopkg.toml.md#override) stanzas in `Gopkg.toml`. This is solely for validation purposes, enforcing that these names correspond strictly to source roots.
+* Import paths in the [`required`](gopkg.toml.md#required) list in `Gopkg.toml`
+* `name` properties in both [`[[constraint]]`](Gopkg.toml.md#constraint) and [`[[override]]`](Gopkg.toml.md#override) stanzas in `Gopkg.toml`. This is solely for validation purposes, enforcing that these names correspond only to project/source roots.
 
-
-
-
-The results of import path deduction are, in practice, almost entirely fixed; it's easy to imagine why. In the public ecosystem, even dynamic deductions rarely change in practice, as it would either require:
-
-- a `go-get` metadata service to intentionally change its mappings, or
-- a `go-get` metadata service to disappear.
-
-`go get` itself is only partially resilient to these cases, but each is potentially catastrophic for a package's retrievability across the ecosystem, sooner rather than later. This steep and abrupt downside makes it nearly impossible for projects only accessible via an unreliable metadata service to ever become popular or widely used in the ecosystem. Thus, in the public ecosystem, we almost only ever see reliable, well-behaved services.
-
-
-
-
-
-
-
-
-
-
-Because deduction has a dynamic component, the deducibility of any given path necessarily cannot be fixed. However, because the 
-
-## Static deduction
diff --git a/docs/introduction.md b/docs/introduction.md
@@ -8,7 +8,4 @@ Welcome! This is documentation for dep, the "official experiment" dependency man
 
 This site has both guides and reference documents. The guides are practical explanations of how to actually do things with dep, whereas the reference material provides deeper dives on specific topics. Of particular note is the [glossary](#glossary.md) - if you're unfamiliar with terminology used in this documentation, make sure to check there!
 
-
-
-* link to guides
-* installing dep
+If you're trying dep for the first time, check out [Creating a New Project](new-project.md), or [Migrating to dep](migrating.md) if you have an existing Go project that you want to manage with dep.
