docs: Update docs with info about noverify

Author: sdboyer

docs/Gopkg.toml.md

@@ -8,6 +8,7 @@ The `Gopkg.toml` file is initially generated by `dep init`, and is primarily han
 * _Package graph rules:_ [`required`](#required) and [`ignored`](#ignored) allow the user to manipulate the import graph by including or excluding import paths, respectively.
 * [`metadata`](#metadata) are a user-defined maps of key-value pairs that dep will ignore. They provide a data sidecar for tools building on top of dep.
 * [`prune`](#prune) settings determine what files and directories can be deemed unnecessary, and thus automatically removed from `vendor/`.
+* [`noverify`](#noverify) is a list of project roots for which [vendor verification](glossary.md#vendor-verification) is skipped.
 
 Note that because TOML does not adhere to a tree structure, the `required` and `ignored` fields must be declared before any `[[constraint]]` or `[[override]]`.
 
@@ -228,6 +229,20 @@ Almost all projects will be fine without setting any project-specific rules, and
 
 It is usually safe to set `non-go = true`, as well. However, as dep only has a clear model for the role played by Go files, and non-Go files necessarily fall outside that model, there can be no comparable general definition of safety.
 
+## `noverify`
+
+The `noverify` field is a list of [project roots](glossary.md#project-root) to exclude from [vendor verification](glossary.md#vendor-verification).
+
+Dep uses per-project hash digests, computed after pruning and recorded in [Gopkg.lock](Gopkg.lock.md#digest), to determine if the contents of `vendor/` are as expected. If the recorded digest and the hash of the corresponding tree in `vendor/` differ, that project is considered to be out of sync:
+
+* `dep ensure` will regenerate it
+* `dep check` will complain of a hash mismatch and exit 1
+
+It is strongly preferable for almost all workflows that you leave `vendor/` unmodified, in whatever state dep puts it in. However, this isn't always an option. If you have no choice but to modify `vendor/` for a particular project, then add the project root for that project to `noverify`. This will have the following effects:
+
+* `dep ensure` will ignore hash mismatches for the project, and only regenerate it in `vendor/` if absolutely necessary (prune options change, package list changes, version changes)
+* `dep check` will continue to report hash mismatches (albeit with an annotation about `noverify`) for the project, but will no longer exit 1. 
+
 ## Scope
 
 `dep` evaluates
@@ -236,13 +251,7 @@ It is usually safe to set `non-go = true`, as well. However, as dep only has a c
 * `required`
 * `ignored`
 
-only in the root project, i.e. the project where `dep` runs. For example,
-
-You have a project: `github.com/urname/goproject`.
-
-`github.com/foo/bar` is a dependency for your project.
-
-Here `dep` evaluates the `Gopkg.toml` files of these packages as follows.
+only in the root project, i.e. the project where `dep` runs. For example, if you have a project: `github.com/urname/goproject`, and `github.com/foo/bar` is a dependency for your project, then dep will evaluate the `Gopkg.toml` files of these projects as follows:
 
 | github.com/urname/goproject | github.com/foo/bar |
 | --------------------------- | ------------------ |
@@ -251,8 +260,8 @@ Here `dep` evaluates the `Gopkg.toml` files of these packages as follows.
 | required ✔                  | required ✖         |
 | ignored ✔                   | ignored ✖          |
 
-️✔ : Evaluated
-✖ ️: Not evaluated
+✔ : Evaluated
+✖ : Not evaluated
 
 # Example
 
@@ -266,6 +275,8 @@ ignored = [
   "bitbucket.org/user/project/pkgA/pkgY"
 ]
 
+noverify = ["github.com/something/odd"]
+
 [metadata]
 codename = "foo"
 

docs/daily-dep.md

@@ -19,9 +19,7 @@ Dep's main command is `dep ensure`. The verb is "ensure" to imply that the actio
 
 > Hey dep, please make sure that [my project](glossary.md#current-project) is [in sync](glossary.md#sync): that [`Gopkg.lock`](Gopkg.lock.md) satisfies all the imports in my project, and all the rules in[ `Gopkg.toml`](Gopkg.toml.md), and that `vendor/` contains exactly what `Gopkg.lock` says it should."
 
-As the narrative indicates, `dep ensure` is a holistic operation; rather than offering a series of commands that you run in succession to incrementally achieve some final state, each run of `dep ensure` delivers a safe, complete, and reproducible set of dependencies with respect to the current state of your project. You might imagine repeated runs of `dep ensure` as being a bit like a frog, hopping from one lily pad to the next.
-
-`dep ensure` also guarantees that, barring `kill -9`, power failure, or a critical bug, its disk writes are all-or-nothing: on any given run, either nothing changes (and you get an error), or you're on the nearest safe lily pad. This makes `dep ensure` fine to run at most any time.
+As the narrative indicates, `dep ensure` is a holistic operation. Rather than offering a series of commands to run in succession that incrementally achieve some final state, each run of `dep ensure` delivers a safe, complete, and reproducible set of dependencies with respect to the current state of your project. You might imagine repeated runs of `dep ensure` as being a bit like a frog, hopping from one lily pad to the next.
 
 ## Using `dep ensure`
 
@@ -32,7 +30,7 @@ There are four times when you'll run `dep ensure`:
 * To catch up after importing a package for the first time in your project, or removing the last import of a package in your project
 * To catch up to a change to a rule in `Gopkg.toml`
 
-There's also an implicit fifth time: when you're not sure if one of the above has happened. Running `dep ensure` without any additional flags will get your project back in sync - a known good state. As such, it's generally safe to defensively run `dep ensure` as a way of simply making sure that your project is in that state.
+If you're not sure if there have been changes to imports or `Gopkg.toml` rules, run `dep check`. It will tell you what is out of sync in your project. If anything is out of sync, running `dep ensure` will bring it back into line.
 
 Let's explore each of these moments. To play along, you'll need to `cd` into a project that's already been set up by `dep init`. If you haven't done that yet, check out the guides for [new projects](new-project.md) and [migrations](migrating.md).
 
@@ -83,20 +81,16 @@ $ dep ensure -update
 
 ### Adding and removing `import` statements
 
-As noted in [the section on adding dependencies](#adding-a-new-dependency), dep relies on the `import` statements in your code to figure out which dependencies your project actually needs. Thus, when you add or remove import statements, dep might need to care about it.
+As noted in [the section on adding dependencies](#adding-a-new-dependency), dep relies on the `import` statements in your code to figure out which dependencies your project actually needs. Thus, when you add or remove import statements, dep often needs to care about it.
 
-It's only "might," though, because most of the time, adding or removing imports doesn't matter to dep. Only if one of the following has occurred will a `dep ensure` be necessary to bring the project back in sync:
+ Only if one of the following has occurred will a `dep ensure` be necessary to bring the project back in sync:
 
 1.  You've added the first `import` of a package, but already `import` other packages from that project.
 2.  You've removed the last `import` of a package, but still `import` other packages from that project.
 3.  You've added the first `import` of any package within a particular project. (Note: this is the [alternate adding approach](#adding-a-new-dependency))
 4.  You've removed the last `import` of a package from within a particular project.
 
-In short, dep is concerned with the set of unique import paths across your entire project, and only cares when you make a change that adds or removes an import path from that set.
-
-Of course, it can be tough to keep track of whether adding or removing (especially removing) a particular import statement actually does change the overall set—especially on large projects. Fortunately, you needn't keep close track, as you can run `dep ensure` and it will automatically pick up any additions or removals, and bring your project back in sync.
-
-Only if it is the first/last import of a project being added/removed - cases 3 and 4 - are additional steps needed: `Gopkg.toml` should be updated to add/remove the corresponding project's `[[constraint]]`.
+In short, dep is concerned with the set of unique import paths across your entire project, and only cares when you make a change that adds or removes an import path from that set. `dep check` will quickly report any such issues, which will be resolved by running `dep ensure`.
 
 ### Rule changes in `Gopkg.toml`
 
@@ -141,8 +135,9 @@ $ dep status -dot | dot -T png | open -f -a /Applications/Preview.app
 
 Here are the key takeaways from this guide:
 
+* `dep check` will quickly report any ways in which your project is out of [sync](glossary.md#sync).
 * `dep ensure -update` is the preferred way to update dependencies, though it's less effective for projects that don't publish semver releases.
-* `dep ensure -add` is usually the easiest way to introduce new dependencies, though it's not the only one. To add more than one at a time, you'll need to use multiple arguments, not multiple invocations - and make sure to add real `import` statements for the projects after the command completes!
+* `dep ensure -add` is usually the easiest way to introduce new dependencies, though you can also just add new `import` statements then run `dep ensure`.
 * If you ever make a manual change in `Gopkg.toml`, it's best to run `dep ensure` to make sure everything's in sync.
 * `dep ensure` is almost never the wrong thing to run; if you're not sure what's going on, running it will bring you back to safety ("the nearest lilypad"), or fail informatively.
 

docs/ensure-mechanics.md

@@ -67,7 +67,7 @@ Specifically, dep defines a number of invariants that must be met:
 
 If peeking ahead reveals that the sync invariants are already met, then the corresponding function needn't do any work; if they don't, then dep takes the resolution step. Either way, when `dep ensure` finishes, we can be sure that we're in the "known good state" of where all sync invariants are maintained.
 
-`dep check` will evaluate all of the above relations, and if any invariants do not hold, it will print a description of the desync and exit 1.
+`dep check` will evaluate all of the above relations, and if any invariants do not hold, it will print a description of the desync and exit 1. This behavior can be disabled on a per-project basis using the [`noverify` field in Gopkg.toml](Gopkg.toml.md#noverify).
 
 ## `dep ensure` flags and behavior variations
 

docs/env-vars.md

@@ -16,11 +16,17 @@ Environment variables are passed through to subcommands, and therefore can be us
 
 ### `DEPCACHEAGE`
 
-If set to a [duration](https://golang.org/pkg/time/#ParseDuration) (e.g. `24h`), it will enable caching of metadata from source repositories: lists of published versions, the contents of a project's `Gopkg.toml` file at a particular version, sets of packages and imports at a particular version.
+If set to a [duration](https://golang.org/pkg/time/#ParseDuration) (e.g. `24h`), it will enable caching of metadata from source repositories: 
 
-A duration must be set to enable caching (in future versions of dep, it will be on by default), but it is only used for mutable information, like version lists. Information associated with an immutable VCS revision (packages and imports; `Gopkg.toml` declarations) is cached indefinitely.
+* Lists of published versions
+* The contents of a project's `Gopkg.toml` file, at a particular version
+* A project's tree of packages and imports, at a particular version
 
-The cache lives in `$DEPCACHEDIR/bolt-v1.db`, where the version number is an internal number indicating associated with a particular data schema dep uses. The file can be removed safely; the database will be rebuilt as needed.
+A duration must be set to enable caching. (In future versions of dep, it will be on by default). The duration is used as a TTL, but only for mutable information, like version lists. Information associated with an immutable VCS revision (packages and imports; `Gopkg.toml` declarations) is cached indefinitely.
+
+The cache lives in `$DEPCACHEDIR/bolt-v1.db`, where the version number is an internal number associated with a particular data schema dep uses.
+
+The file can be removed safely; the database will be automatically rebuilt as needed.
 
 ### `DEPCACHEDIR`
 

docs/glossary.md

@@ -26,6 +26,7 @@ dep uses some specialized terminology. Learn about it here!
 * [Source Root](#source-root)
 * [Sync](#sync)
 * [Transitive Dependency](#transitive-dependency)
+* [Vendor Verification](#vendor-verification)
 
 ---
 
@@ -144,3 +145,9 @@ This concept is explored in detail in [ensure mechanics](ensure-mechanics.md#sta
 A project's transitive dependencies are those dependencies that it does not import itself, but are imported by one of its dependencies.
 
 If each letter in `A -> B -> C -> D` represents a distinct project containing only a single package, and `->` indicates an import statement, then `C` and `D` are `A`'s transitive dependencies, whereas `B` is a [direct dependency](#transitive-dependency) of `A`.
+
+### Vendor Verification
+
+Dep guarantees that `vendor/` contains exactly the expected code by hashing the contents of each project and storing the resulting [digest in Gopkg.lock](Gopkg.lock.md#digest). This digest is computed _after_ pruning rules are applied.
+
+The digest is used to determine if the contents of `vendor/` need to be regenerated during a `dep ensure` run, and `dep check` uses it to determine whether `Gopkg.lock` and `vendor/` are in [sync](#sync). The [`noverify`](Gopkg.toml.md#noverify) list in `Gopkg.toml` can be used to bypass most of these verification behaviors.

docs/the-solver.md

@@ -2,7 +2,7 @@
 title: The Solver
 ---
 
-At the heart of dep is a constraint solving engine - a [CDCL](<https://en.wikipedia.org/wiki/Conflict-Driven_Clause_Learning>)-style [SMT]() solver, tailored specifically to the domain of Go package management. It lives in the `github.com/golang/dep/gps` package, and is where the work of determining a valid, transitively complete dependency graph (aka, the contents of `Gopkg.lock`) is performed.
+At the heart of dep is a constraint solving engine - a [CDCL](https://en.wikipedia.org/wiki/Conflict-Driven_Clause_Learning)-style solver (albeit light on the "CL" part), tailored specifically to the domain of Go package management. It lives in the `github.com/golang/dep/gps` package, and is where the work of determining a valid, transitively complete dependency graph (aka, the contents of `Gopkg.lock`) is performed.
 
 This page will eventually detail the solver's mechanics, but in the meantime, there are [docs for an older version of the solver](https://github.com/sdboyer/gps/wiki/gps-for-Contributors) that are still accurate enough to provide a rough picture of its behavior.