formatting conventions for function documentation

[stevengj]

Right now, the @doc help suggests;

  @doc """
    # The Foo Function
    `foo(x)`: Foo the living hell out of `x`.
  """ ->
  function foo() ...

In method documentation, wouldn't it be much more satisfactory () for the help command to automatically prepend the signature for foo before the documentation output as needed?

In generic-function documentation (e.g. documentation for sum in all its forms), obviously this is not possible (since there will in general be multiple possible signatures). However, we should establish some convention for what the documentation should look like.

[johnmyleswhite]

I've wondered if we'd need automatic stubs for documentation. One of the things I most like about R's documentation system is that individual-level documentation of every single parameter of a method. I'd really like to see that kind of detail in most documentation for Julia as it matures.

[ViralBShah]

Yes, having the signature automatically is certainly nice. Also, some structure and style that is common across Base and eventually across packages is also good to target for. Should we try to pick a few functions and establish what documentation should like ideally for those, and then bake that in?

[MikeInnes]

Automatically displaying the signature would be nice in some cases, but not all; e.g. if you define your user-level function as f(args...) and call a lower level routine, or Python, or whatever, the actual method signature isn't meaningful. Same for abbreviated names etc.

One thing I'd definitely like to see is some form of one-line summary convention. We can do all sorts of nice things with that, like intellisense-style autocomplete in IPython/Juno and even live, smart code hints, which I have a very rough prototype of:

That said, I'm also leaning towards letting standards evolve naturally rather than enforcing something early on. The result is likely to be better and using markdown means it's pretty trivial to pull out structured data even if the conventions are fairly loose.

[StefanKarpinski]

One convention I've thought about from time to time is to use long descriptive argument names. We do have a tendency to abbriviate especially on some additional method definitions, however, which is convenient; maybe we could have some system where foobarbaz could be abbreviated to fbb – i.e. by looking for long names on the first method defintion (or in the docstring itself?) and associating abbreviated argument names with the long names that they are subsequences of.

[hayd]

Perhaps see numpy for inspiration https://github.com/numpy/numpy/blob/master/doc/example.py https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt.

Numpy basically uses markdown but doesn't do anything special to extract sections:

@doc """
Overview of `foo` function.

Parameters
----------
`x`
    thing to be fooed.
""" ->
function foo() ...

perhaps this could be lowered/expanded to something like (although these probably need to be ordered):

@doc MetaDoc[:docstring => md"Overview of `foo` function",
             :parameters => [:x => md"thing to be fooed."]] ->
function foo() ...

In the syntax of #8588 (comment).

I think having this making actual metadata rather than just a convention will be useful... this way one could, say, extract all the references in a module, or highlight the doc of the current parameter as you type, or toggle through just code examples.

[ViralBShah]

These are the sections @milktrader suggested. These seem like a great start.

Description 
...
Usage
...
Arguments
...
Details
...
References
...
See Also
...

[johnmyleswhite]

In addition to Arguments, we need Returns as well.

[stevengj]

Compare to the documentation we have now, where a typical function has a one-paragraph description.

Would it really be readable if every single one of these descriptions were expanded into 7 subsections complete with headings? I'm skeptical.

[johnmyleswhite]

Coming from R, I find the current documentation a little too terse. For simple functions, I agree that leaving out many of those subsections seems reasonable. But for functions with even the slightest bit of subtlety (e.g. mad in StatsBase), the thoroughness of R's documentation strikes me as the right endgoal. Here's the docs for mad in R, which I think have the right level of information:

mad                   package:stats                    R Documentation

Median Absolute Deviation

Description:

     Compute the median absolute deviation, i.e., the (lo-/hi-) median
     of the absolute deviations from the median, and (by default)
     adjust by a factor for asymptotically normal consistency.

Usage:

     mad(x, center = median(x), constant = 1.4826, na.rm = FALSE,
         low = FALSE, high = FALSE)

Arguments:

       x: a numeric vector.

  center: Optionally, the centre: defaults to the median.

constant: scale factor.

   na.rm: if ‘TRUE’ then ‘NA’ values are stripped from ‘x’ before
          computation takes place.

     low: if ‘TRUE’, compute the ‘lo-median’, i.e., for even sample
          size, do not average the two middle values, but take the
          smaller one.

    high: if ‘TRUE’, compute the ‘hi-median’, i.e., take the larger of
          the two middle values for even sample size.

Details:

     The actual value calculated is ‘constant * cMedian(abs(x -
     center))’ with the default value of ‘center’ being ‘median(x)’,
     and ‘cMedian’ being the usual, the ‘low’ or ‘high’ median, see the
     arguments description for ‘low’ and ‘high’ above.

     The default ‘constant = 1.4826’ (approximately 1/ Phi^(-1)(3/4) =
     ‘1/qnorm(3/4)’) ensures consistency, i.e.,

                         E[mad(X_1,...,X_n)] = sigma                    

     for X_i distributed as N(mu, sigma^2) and large n.

     If ‘na.rm’ is ‘TRUE’ then ‘NA’ values are stripped from ‘x’ before
     computation takes place.  If this is not done then an ‘NA’ value
     in ‘x’ will cause ‘mad’ to return ‘NA’.

See Also:

     ‘IQR’ which is simpler but less robust, ‘median’, ‘var’.

Examples:

     mad(c(1:9))
     print(mad(c(1:9),     constant = 1)) ==
           mad(c(1:8, 100), constant = 1)       # = 2 ; TRUE
     x <- c(1,2,3,5,7,8)
     sort(abs(x - median(x)))
     c(mad(x, constant = 1),
       mad(x, constant = 1, low = TRUE),
       mad(x, constant = 1, high = TRUE))

[ViralBShah]

@stevengj If we have section templates, we can choose different levels of verbosity for the command line help (just show description and usage), and the manual which has all sections. Matlab also does more than what we do currently, which helps when one is new to the system.

[stevengj]

Fair enough.

[alanedelman]

I've always wished one could click somewhere and find out about numerical notes ....when
available. Some things would be links to issues about accuracy, even numerical comparisons
to other systems, links to Kahan wisdom, etc. In other words, the help becomes
more than how to use Julia, it becomes a whole numerical course.