Usage of leading underscores

[rimey]

Normal Python usage, of course, uses leading underscores to indicate that a name is intended to be private. My understanding is that this library deviates from this convention by instead using leading underscores to limit what is included in the Sphinx-generated documentation. Names with leading underscores defined at module level are often used in other modules, and names with leading underscores defined at class level are often used outside the class, and even from other modules. In effect, such names are treated as private to the package. Is this understanding correct?

I would suggest looking into controlling visibility in the Sphinx-generated documentation separately and gradually migrating to normal Python usage.

[dhermes]

My understanding is that this library deviates from this convention by instead using leading underscores to limit what is included in the Sphinx-generated documentation

Nope, we also use them to indicate the names are not intended as public.

Any specific examples you have in mind?

[tseaver]

We talk about / think about private attributes / members at the level of library users, not necessarily within the library itself:

• We consider private attributes / data members as being freely available to unit tests (some of them really have no other purpose, even).
• Because they are about the user-facing API exposed by the library, we normally avoid using them from system tests, or exposing them in example snippets.
• On a case-by-case basis, we have allowed use from within another module in the same package, in effect treating the using-module as a "friend" in the C++ / Java sense.
• At least one of our sub-packages (gcloud.streaming) should really be private in terms of library users, but carries a public-looking name due to a quirk/bug in nose (it ignores packages whose names begin with _). Nevertheless, it is heavily used by gcloud.storage.blob.

[rimey]

@tseaver Another big category is classes and module-level functions in "helper" modules.

This notion of things being private "at the level of library users" isn't really normal Python usage.

It's not just about generated docs?

[dhermes]

Most of these helper modules are intentionally non-public, if that's what you mean.

[rimey]

@dhermes No, I mean that names defined at module level in helper modules are prefixed with leading underscores whether or not they are actually intended to be private to that helper module.

[dhermes]

Yeah names in _helper modules could certainly all be public, since the module itself is not. That may have just been overly defensive programming. But the _helper modules themselves will never be picked up by Sphinx, so that isn't the cause of those names.

[tseaver]

I don't see a bug here: the helpers are not intended to be public, which means that renaming them makes them seem as though they might be public (e.g., when found from a module which imports them by name).

[dhermes]

I don't care very much, though @jonparrott has said compelling things to me along these lines. As for public names from a private module, aren't imported members not "considered" part of the public API.