[incremental] skip type-checking

[nikomatsakis]

This is a kind of meta-bug aimed at skipping type-checking, which is (to some extent) the "holy grail" of incremental compilation. This bug only contains a list of issues and a task breakdown. For background on the general approach we have in mind, see this gist.

Substeps:

• [incremental] introduce ensure() operation #45210: Introduce the ensure operation first described here and use it on the typeck_tables_of invocations. -- @theotherjimmy is on it
• [incremental] Isolate the "used trait imports" part of typeck tables into a distinct query #45214: Isolate the "used trait imports" part of typeck tables into a distinct query
• Enable serialization and deserialization of incremental results, at least for some types.
  • In particular, we will need to support serializing and re-loading Vec<DefId>. It wouldn't hurt to support () while we're at it.
  • This is likely best tackled by @michaelwoerister or @alexcrichton, but that may not be the case.

Ultimately, all of the the uses of typeck_tables_of must be analyzed and eliminated. These uses are listed here in order to mine for producing more subtasks. =)

• src/librustc_passes/consts.rs:142: self.tables = self.tcx.typeck_tables_of(item_def_id);
• src/librustc_typeck/check_unused.rs:69: let tables = tcx.typeck_tables_of(item_def_id);
• src/librustc_save_analysis/dump_visitor.rs:109: let tables = self.tcx.typeck_tables_of(item_def_id);
• src/librustc_typeck/collect.rs:1155: return tcx.typeck_tables_of(def_id).node_id_to_type(hir_id);
• src/librustc_typeck/collect.rs:1192: tcx.typeck_tables_of(owner).node_id_to_type(hir_id)
• src/librustc_typeck/collect.rs:1244: tcx.typeck_tables_of(def_id).closure_tys()[hir_id]
• src/librustc_privacy/lib.rs:503: replace(tables, tcx.typeck_tables_of(def_id))
• src/librustc_const_eval/pattern.rs:606: self.tables = self.tcx.typeck_tables_of(def_id);
• src/librustc_const_eval/eval.rs:381: tables: tcx.typeck_tables_of(def_id),
• src/librustc_const_eval/eval.rs:770: let tables = tcx.typeck_tables_of(def_id);
• src/librustc_borrowck/borrowck/mod.rs:100: let tables = tcx.typeck_tables_of(owner_def_id);
• src/librustc_borrowck/borrowck/mod.rs:198: let tables = tcx.typeck_tables_of(owner_def_id);
• src/librustc/cfg/construct.rs:57: let tables = tcx.typeck_tables_of(owner_def_id);
• src/librustc/ty/mod.rs:2108: self.typeck_tables_of(self.hir.body_owner_def_id(body))
  • not a direct user, see calls of body_tables() below
• src/librustc/middle/intrinsicck.rs:136: let tables = self.tcx.typeck_tables_of(owner_def_id);
• src/librustc_typeck/check/mod.rs:734: tcx.typeck_tables_of(body_owner_def_id);
• src/librustc_typeck/check/mod.rs:756: tcx.typeck_tables_of(def_id).generator_sigs()[hir_id].map(|s| ty::Binder(s))
• src/librustc_typeck/check/mod.rs:764: tcx.typeck_tables_of(def_id).closure_kinds()[hir_id].0
• src/librustc_typeck/check/mod.rs:857: return tcx.typeck_tables_of(outer_def_id);
• src/librustc_typeck/check/mod.rs:1152: tcx.typeck_tables_of(tcx.hir.local_def_id(it.id));
• src/librustc_typeck/check/mod.rs:1574: tcx.typeck_tables_of(tcx.hir.local_def_id(e.node_id));
• src/librustc_mir/transform/generator.rs:712: let interior = *tcx.typeck_tables_of(def_id).generator_interiors().get(hir_id).unwrap();
• src/librustc_mir/hair/cx/mod.rs:96: tables: tcx.typeck_tables_of(src_def_id),
• src/tools/clippy/clippy_lints/src/consts.rs:302: tables: self.tcx.typeck_tables_of(def_id),
• src/librustc_privacy/lib.rs:518: let orig_tables = replace(&mut self.tables, self.tcx.body_tables(body));
• src/librustc_privacy/lib.rs:672: let orig_tables = replace(&mut self.tables, self.tcx.body_tables(body));
• src/librustc_driver/pretty.rs:505: self.tables.set(self.tcx.body_tables(id));
• src/librustc_const_eval/check_match.rs:53: tables: self.tcx.body_tables(b),
• src/librustc/lint/context.rs:636: self.tables = self.tcx.body_tables(body);
• src/librustc/lint/context.rs:696: self.tables = self.tcx.body_tables(body_id);
• src/librustc/middle/liveness.rs:530: let tables = ir.tcx.body_tables(body);
• src/librustc/middle/reachable.rs:101: self.tables = self.tcx.body_tables(body);
• src/librustc/middle/dead.rs:218: self.tables = self.tcx.body_tables(body);
• src/librustc_mir/build/mod.rs:106: let gen_ty = tcx.body_tables(body_id).node_id_to_type(fn_hir_id);
• src/librustc_mir/build/mod.rs:247: let closure_ty = tcx.body_tables(body_id).node_id_to_type(closure_expr_hir_id);
• src/tools/clippy/clippy_lints/src/attrs.rs:172: is_relevant_expr(tcx, tcx.body_tables(eid), &tcx.hir.body(eid).value)
• src/tools/clippy/clippy_lints/src/attrs.rs:180: ImplItemKind::Method(_, eid) => is_relevant_expr(tcx, tcx.body_tables(eid), &tcx.hir.body(eid).value),
• src/tools/clippy/clippy_lints/src/attrs.rs:189: is_relevant_expr(tcx, tcx.body_tables(eid), &tcx.hir.body(eid).value)
• src/tools/clippy/clippy_lints/src/functions.rs:154: let tables = cx.tcx.body_tables(body.id());

[theotherjimmy]

I have implemented ensure on a branch. I'll make a PR

[nikomatsakis]

@theotherjimmy great, see #45210 too =)

[hcpl]

@nikomatsakis for markdown working under spoilers, please also use <p> tag:

<details><p>
# newline required after <p>
# ... checkboxes here ...
</p></details>

[nikomatsakis]

Added #45214, another step along the way.

[theotherjimmy]

The PR I promised is #45228

[theotherjimmy]

For my next trick: Isolate the "used trait imports" part of typeck tables into a distinct query

[theotherjimmy]

It looks like @cjkenn has #45214, "used trait imports". I'll take a a look at serialization of Vec<DefId> and ().

[cjgillot]

Status of the issue:

1. The "used trait imports" query exists and is used;
2. Saving Vecs seems to be possible, but is not used right now;
3. Some parts of the compiler use the query for only one field at a time (see below).

About usage of typeck_tables_of and body_tables:

• for ensuring it runs

librustc_typeck/check/mod.rs:1751:            tcx.typeck_tables_of(def_id);
librustc_typeck/check/mod.rs:1755:            tcx.typeck_tables_of(tcx.hir().local_def_id(it.hir_id));
librustc_typeck/check/mod.rs:2673:            tcx.typeck_tables_of(tcx.hir().local_def_id(e.hir_id));

• for closure_kind_origins

librustc_mir/borrow_check/diagnostics/conflict_errors.rs:194:                    let tables = self.infcx.tcx.typeck_tables_of(id.expect_local());
librustc_mir/borrow_check/diagnostics/mod.rs:104:                        self.infcx.tcx.typeck_tables_of(did).closure_kind_origins().get(hir_id)
librustc_mir/borrow_check/diagnostics/mod.rs:127:                    self.infcx.tcx.typeck_tables_of(did).closure_kind_origins().get(hir_id)

• for concrete_opaque_types

librustc_mir/borrow_check/type_check/mod.rs:1236:            &tcx.typeck_tables_of(anon_owner_def_id.expect_local()).concrete_opaque_types;
librustc_typeck/collect/type_of.rs:123:                            &tcx.typeck_tables_of(owner.expect_local()).concrete_opaque_types
librustc_typeck/collect/type_of.rs:431:            if !self.tcx.typeck_tables_of(def_id).concrete_opaque_types.contains_key(&self.def_id) {

• for liberated_fn_sigs

librustc_typeck/collect.rs:1478:                    let fn_sig = tcx.typeck_tables_of(def_id).liberated_fn_sigs()[hir_id];

• for tainted_by_errors

librustc_mir/const_eval/eval_queries.rs:294:            if let Some(error_reported) = tcx.typeck_tables_of(def_id).tainted_by_errors {
librustc_mir/interpret/eval_context.rs:407:                if let Some(error_reported) = self.tcx.typeck_tables_of(did).tainted_by_errors {
librustc_typeck/collect/type_of.rs:141:                                tcx.typeck_tables_of(owner.expect_local()).tainted_by_errors

• for upvars_list

librustc_mir/interpret/validity.rs:203:                    let tables = self.ecx.tcx.typeck_tables_of(def_id);

• for user_provided_sigs

librustc_mir/borrow_check/type_check/input_output.rs:39:            let typeck_tables = self.tcx().typeck_tables_of(self.mir_def_id.expect_local());

• for node_type and node_type_opt

librustc_mir/borrow_check/diagnostics/conflict_errors.rs:883:                                    .typeck_tables_of(def_id)
librustc_mir/borrow_check/diagnostics/mutability_errors.rs:510:            let tables = self.infcx.tcx.typeck_tables_of(def_id);
librustc_mir/borrow_check/universal_regions.rs:501:                    let tables = tcx.typeck_tables_of(self.mir_def_id.expect_local());
librustc_typeck/collect/type_of.rs:606:    let ty = tcx.diagnostic_only_typeck_tables_of(def_id).node_type(body_id.hir_id);
librustc_mir_build/build/mod.rs:89:                    let gen_ty = tcx.body_tables(body_id).node_type(id);
librustc_mir_build/build/mod.rs:144:                let gen_ty = tcx.body_tables(body_id).node_type(id);
librustc_mir_build/build/mod.rs:211:    let closure_ty = tcx.body_tables(body_id).node_type(closure_expr_id);

• for general usage

librustc_mir/borrow_check/mod.rs:140:    let tables = tcx.typeck_tables_of(def_id);
librustc_mir_build/hair/cx/mod.rs:59:        let tables = tcx.typeck_tables_of(src_def_id);
librustc_passes/intrinsicck.rs:135:        let tables = self.tcx.typeck_tables_of(owner_def_id);
librustc_passes/liveness.rs:685:        let tables = ir.tcx.typeck_tables_of(def_id);
librustc_privacy/lib.rs:357:    if tcx.has_typeck_tables(def_id) { tcx.typeck_tables_of(def_id) } else { empty_tables }
librustc_save_analysis/dump_visitor.rs:113:            self.tcx.typeck_tables_of(item_def_id)
librustc_trait_selection/traits/error_reporting/suggestions.rs:1246:                query_tables = self.tcx.typeck_tables_of(generator_did.expect_local());
librustc_driver/pretty.rs:333:            self.tables.set(self.tcx.body_tables(id));
librustc_lint/late.rs:108:        self.context.tables = self.context.tcx.body_tables(body);
librustc_lint/late.rs:184:        self.context.tables = self.context.tcx.body_tables(body_id);
librustc_mir_build/hair/pattern/check_match.rs:30:        MatchVisitor { tcx, tables: tcx.body_tables(body_id), param_env: tcx.param_env(def_id) };
librustc_passes/dead.rs:223:        self.tables = self.tcx.body_tables(body);
librustc_passes/reachable.rs:85:        self.tables = self.tcx.body_tables(body);
librustc_privacy/lib.rs:1092:        let orig_tables = mem::replace(&mut self.tables, self.tcx.body_tables(body));
librustc_privacy/lib.rs:1234:        let orig_tables = mem::replace(&mut self.tables, self.tcx.body_tables(body));

I think making closure_kind_origins, concrete_opaque_types and tainted_by_errors standalone queries may be worth it. liberated_fn_sigs, upvars_list and user_provided_sigs are only used once. Removing the body_tables alias may be worth it too.

[nikomatsakis]

I .. think this is fixed?

[nikomatsakis]

But maybe there's room to do better

[nikomatsakis]

@cjgillot it seems like it'd be good to have some concrete examples of cases where we don't get re-use, but should/could? (Not sure if you had some you were using for analysis.)