Controlling scope of build options

[JukkaL]

Currently the --implicit-any mode flag is global, as it uniformly affects how mypy processes every file in a mypy run. Other global flags are being considered or implemented, including --strict and the "weak mode". Similarly, we could have options to disable/enable individual errors/warnings. These kinds of mode flags have some inconvenient properties, and I elaborate on this below.

A. Combining type checked programs is hard

It's hard to combine two independently developed codebases into a single type checked program, as it may be necessary to use the least common denominator options to type check the combined program without spurious warnings. Global flags effectively introduce multiple mypy dialects that aren't quite compatible. Using the least common denominator options is clearly not desirable as programmers likely chose to enable some options for their code for a good reason.

Integrating multiple pieces of code without loosening type checking may involve a lot of refactoring or adding # type: ignore comments.

B. Migrating to stricter type checking mode is hard

Migrating from the default mode to --strict, for example, would have to be performed for the whole code base to avoid generating warnings. If there is, say, millions of lines code, this can be a huge undertaking, and may require the co-operation of owners of every part of the codebase. It would be much nicer to migrate to stricter options gradually, similar to how mypy supports migration from dynamic to static typing in small increments. This would also give more flexibility to owners of particular modules.

C. Options aren't self-documenting

When just looking at a source file, it's not obvious under which options it's supposed to type checked. This is less of a problem within a single, uniform codebase, but it becomes more important when combining code from multiple sources. It's easy to "forget" that some files should be checked in the strict mode when copying or moving source code, for example, since the knowledge lives in external configuration such as build scripts instead of being close to the physical code.

---

Here are some ways to overcome some of these problems:

1. We type check code bases with different global options using separate mypy runs, stubbing out dependencies. So if codebases A and B have different global options, and A depends on B, we first type check B using the desired options and generate a set of stub files. We then separately type check A using a different set of options, and using stubs for B. This increases complexity and requires additional tooling to automatically generate stubs. Additionally, this doesn't significantly simplify the migration to stricter options within a single codebase.
2. Provide fine grained configuration outside source files. For example, perhaps mypy will accept options such as --strict=package1,package2 that will enforce some options for particular packages only. Alternatively, we could have a configuration file with rules such as strict=package/**,module, or perhaps even a hierarchy of such configuration files. This provides more flexibility but it makes it easy to forget to modify the configuration when adding new files or moving files around.
3. Provide per-file option flags. We always default to some well-defined mode, and we support comments such as # mypy: strict near the top of the file that change mode options for that file only. This is very flexible and it makes files self-contained, but it's easy to forget to add the desired options to new files. We could also have a tool that enforces that certain options are always used in certain packages, similar to (2) above, in addition to per-file comments.
4. Don't provide any options. Make the default mode "good enough" for most people.

My proposal is to choose some variant of option 3.

---

Note that I don't consider --silent-import a global option because it only affects files not being processed, so it's not really problematic in the same sense as the other options I discussed above.

[gvanrossum]

I like the per-file flags in the source code. Maybe we could also have command-line flags to override these or perhaps to change the default? It might make sense to try a code base with a more strict flag to see how it responds. A less strict mode doesn't make as much sense.

…

--Guido (mobile)

[ddfisher]

I agree: per-file flags in the source code seems like the clear winner here. The downside of implementing them sooner rather than later is that it locks us into the names we choose much more than command-line flags do, in my opinion, and I don't feel like we have our naming scheme sorted out yet. Have we been getting feature requests for this yet?

That said, I actually don't find problems A or C particularly convincing.
A: Merging two independently developed codebases into a single codebase is 1) rare and 2) difficult for many other reasons (build system, naming conventions, duplication of utility functionality, etc). I'm don't think we need to be particularly concerned about optimizing for this use case. Having one codebase used as the library for another is more common, but I think mypy handles that situation well.
C: I'm not sure that per-file flags actually fix this. Files are large, and the common case is that people jump to the middle of them, not start reading from the top. Sure, per-file flags make it easier to check the flags for a particular file (by just looking at the top), but you already have to be thinking about doing so, and it's not that much harder to track down the build script. In fact, this creates the additional problem that e.g. moving functions between files in the same codebase can change their typing mode, which isn't the case if each codebase is homogeneously typed.

I think continuing to have command-line flags is important. Here are some benefits that I see:

• They're easier to get started with than per-file flags.
• It makes it easy for users to try out various features and understand mypy's capabilities.
• It allows someone to use a stricter-than-default typing mode for most of their codebase without having to remember to put an identical header on all their files.
• The help text serves a simple, local documentation to the per-file flags.

I think Guido brought up an interesting question: should command-line flags override per-file flags, or just set the default? Each supports a different use case:
override: I have a files (or a bunch of files) that all have per-file flags that say they should be type checked in a certain way, but I want to experimentally see what happens if they're type checked in a different way. (If command-line flags just set the default, I'd have to modify all those files.)
default: I want most of the files in my codebase to be type checked a certain way, but some files may need to be type checked more or less strictly. (If command-line flags overrode per-file flags, I'd have to add a per-file to header to all my files, which might accidentally get left out of new files being created, etc.)

Both of these seem common enough to me that I think we should consider eventually supporting both. By default, command-line flags would just set the default option, but there should be some -f equivalent which causes them to override the per-file flags.

Finally, I disagree about --silent-imports not being a global option in the same way. Toggling silent imports can affect whether or not a file will type check, causing problems A, B, and C above.

[gvanrossum]

Maybe we can close this now we have a mypy.ini file with per-file options?