Wanted: a good way to expose the intended user interface

[timholy]

As Julia nears perfection :half-serious-wink:, I have the sense that discoverability remains one of the laggards. This is basically a cross-post from https://discourse.julialang.org/t/exploring-package-contents/48937. A brief summary of the highlights:

• names(SomePkg) works pretty well when the entire intended interface is exported
• names(CSV) is basically useless for a package like CSV where the "public" API requires module scoping. (For the record, I applaud that decision by the CSV developers.) names(CSV, all=true) is less than ideal because there is no mechanism to distinguish internal methods from ones intended for user consumption, and also delivers types, constants, and other things that may not be of interest
• methods(CSV) yields nothing
• apropos(CSV) is quite useful, if the module name appears in the docstring. Conversely, if the module name appears in docstrings not intended for public usage (see next item) you can get false-positives.
• reaching in to Base.Docs.meta(CSV) is also revealing. But many of us add docstrings to internal methods to help ourselves and other developers understand the internal logic of a complicated package. So just having a docstring is not quite the same thing as being part of the user interface. This also fails for packages that mostly re-export other packages. Compare ?Images with Base.Docs.meta(Images) to see the dramatic difference this can make.

One thought might be to add

module CSV

export PooledString

export scoped=true File, Rows, read, write

...

end

It's not really exported, but it is marked as "elevated" and perhaps that marker could be used by other tools.

[JeffBezanson]

Related: #30204

[timholy]

Ah, good. I was looking for something like that, but it's amazing how minor the differences in the search string cause a miss.

[LilithHafner]

I like this because

• It is simple for package authors to support
• It plays nicely with names
• It covers the majority of the "what is the API of this package" question

This does not provide a full formal answer to the "what is the API of this package" question and that is okay. API includes function semantics and a system for formally defining function semantics is a programming language—I prefer to keep that informal. What we would have is "the package API is a subset of the constructs in names(MyPackage), all of which may be used publically in some way but some of which may have restrictions, see their respective docstrings" which is IMO a much better answer than the current situation.

Questions about backward compatibility

• Can we get away with names(Package) returning things that are scoped exported by default or must we preserve getglobal.((@__MODULE__,), names(CSV))?
• For modules that have not been updated to use export scoped=true, should names(MyModule, include_scoped_exports=true) return only explicit exports, underrepresenting CSV.jl's API, or return all symbols, overrepresenting CSV.jl's. API (I think the former is clearly the right answer: private by default)

[c42f]

Awesome! I didn't see this issue when I made the comment in #49973 (comment) proposing almost exactly the same thing Tim already proposed here. It's encouraging that we reached the same conclusions (and almost the same syntax haha!)

Anyway, I agree @LilithHafner: I think this very simple addition is all we need for now. Having API defined in terms of which symbols are accessible is clearly "in the spirit" of what exists already in Julia; we can just do this, it'll be consistent with what we have already and it'll improve the situation a lot.

• Can we get away with names(Package) returning things that are scoped exported by default or must we preserve getglobal.((@__MODULE__,), names(CSV))?

Probably we can get away with it? Seems worth trying and seeing how it feels.

• For modules that have not been updated to use export scoped=true, should names(MyModule, include_scoped_exports=true) return only explicit exports, underrepresenting CSV.jl's API, or return all symbols, overrepresenting CSV.jl's. API (I think the former is clearly the right answer: private by default)

Private by default seems clearly right. How would we determine which modules have "not been updated" anyway? Some modules might not want to use any scoped exports.

[jakobnissen]

Closed by #50105