Markdown or rST?

[matin]

Should the API specs for Balanced be written in GitHub Flavored Markdown or reStructuredText?

In creating this repo, we were having an internal debate around whether we should use Markdown or rst. Balanced is a Python shop, and it's considerably easier for us to generate rst. That means it's also easier to ensure validation and consistancy. However, the specs are being written for GitHub, and Markdown is the standard on GitHub.

[bninja]

rst seems better suited for spec docs which will likely feature things, such as tables, that md intentionally doesn't support (or requires that you drop down to html to achieve).

For reading that probably doesn't matter as the tedious work of generating say a table in html should be entirely automated when building the spec from the implementation. Additionally markdown is clearly favored by github (so e.g. nicer rendering, shortcuts, comments and issues are all .md).

For writing it does matter. The idea is for people to edit the markup files directly to show how things should be done and being forced to drop down to html to do basic things is annoying. There are probably tools for markdown that make this easier.

If you identify all the structural features you need to support and find that markdown supports them as first class markup elements then it would probably fine.

This thread raises may good points of which is preferable, when and why:

http://stackoverflow.com/questions/34276/markdown-versus-restructuredtext

In this case I think rst should win.

[matin]

@whit537 this repo largely came out of a conversation you and I had. Thoughts?

[matin]

ping @alexnotov, @bcm, @khussein, and @ajsharp

[zealoushacker]

I am going to basically point to github's own API repo as an example of stuff that can be done with gfm: http://developer.github.com/v3/. After a cursory look, I think they use a static layout file and then pull in .md content. (I may be mistaken).

I think it's a matter of goals...
If your goal is to generate the docs for the larger community to help you collaborate and edit the docs, then I would suggest gfm (because it is the preference of the community).
If your goal is to have a private repo, where you maintain your API docs in version control and don't have the larger github community participate, then use whatever you prefer (rst, or otherwise).

my humble, but very opinionated 2cents =)

[bcm]

for what it's worth, I'm very used to writing gfm for technical prose and api docs, and I have never missed the lack of table syntax. I've had to drop into html maybe a half dozen times, and it was never a pain. also, I sure do like how that github site looks.

[zealoushacker]

@bcm 👍 regarding "never missed the lack of table syntax".

I've dug a bit deeper into how github itself maintains its API docs. So 95% of their docs are written in gfm. They use nanoc to compile the site into static html from their .md files and whatever other formats they use.

@matin You and your devs have probably read this thread and may even know better alternatives...

[matin]

@alexnotov I'm a supporter of Markdown, especially GFM. I think @mahmoudimus, @bninja, and @msherry need to elaborate on their case for rST.

[mahmoudimus]

I realized that most of my arguments here were a little incohesive, so I've deleted them and added them into one post. First and foremost, I want to stress that there are serveral use cases for the power of reST when designing something like an API spec that has to be self validating. What does it mean to be self-validating? It means that the documentation is auto-generated and checked against the current version of the "master" branch of the API spec (in this case, it's the balanced-api repository).

Here are some technical reasons why I think github flavored markdown is a poor choice for auto-generated api specification from python source code:

1. Not having the list directive, which controls tables and their respective layouts, restricts the way the documentation generates its fields (i.e. look @ App.Net's spec - it is hand curated w/ custom HTML and no ability to auto-validate their API: https://github.com/appdotnet/api-spec).
2. Markdown is notorioius for having limitations, like code in lists, or finicky stuff that require you to dive into HTML (like images inside links).
3. GitHub's support for reST is just as great as GFM, but has the ability to be better.
4. Sphinx's ecosystem and extensive docutils plugins.
5. Markdown's intention was for simple marked up files like READMEs or simple documents -- not an API spec.

Given reStructuredText's extensibility, it is clearly superior to add different domains / directives that allow customized rendering of the way the API spec is set for design. It also allows us to keep our API spec in sync w/ the API spec the master branch dictates

[chadwhitacre]

@matin Love the repo, very excited to see this opening up. :-)

The primary concern is excellent end-user experience, not tools. In this case, the end-user is the developer who is using the documentation. I'm with @bcm: "I sure do like how that github site looks." Achieving that level of excellence is a function of taste and competence and hard work, which depend primarily on the people doing the work, and not necessarily on the tools employed. However, tool choice can be a barometer of taste and competence, as Linus famously, um, observed. ;-)

My personal strategy when I don't feel competent (as here) is to find examples of excellence and attempt to emulate them. In this case I see plenty of people writing reST or even creole, but the user experiences over there are rather more mundane than on GitHub, I daresay. The fact that I think so highly of GitHub in general predisposes me towards GFM, in the absence of a well-formed opinion otherwise regarding structured text formats.

What does it mean to be self-validating? It means that the documentation is auto-generated and checked against the current version of the "master" branch of the API spec (in this case, it's the balanced-api repository).

@mahmoudimus Can you help me understand this better? The idea is that the documentation is maintained in a literate style inside docstrings? What does the checking step entail? Are you validating behavior of the API against documented behavior? Or are you validating auto-generated documentation against manually maintained documentation in the balanced-api repo? What does the balanced-api repo contain? Documentation?

[bcm]

this: "The fact that I think so highly of GitHub in general predisposes me
towards GFM, in the absence of a well-formed opinion otherwise regarding
structured text formats."

I have zero experience with reST so can't comment on how powerful or easy
to use it might be. my main concerns when choosing an authoring tool are
how easy it is for me to write content and how attractive and easily
navigable the output is. I don't typically consider automated publishing
features because I don't typically use them. I've always maintained these
kinds of documents by hand simply because I prefer to have 100% control
over the content. therefore, my experience and opinion might not be super
useful in this situation :)

[matin]

Here are the two arguments I get out of this conversation:

• GFM is the GitHub standard. It adopts part of rST to make it more powerful, but it's the standard nonetheless, which makes it easier for anyone to make changes and issue pull requests
• rST is more powerful and easier to generate from Python because of docutils. Automated validation keeps Balanced honest in guaranteeing that the API is consistent with this repo. Even if GFM was generated, it would happen with rST as an intermediate anyways

I propose we move forward with rST for all generated content. It allows Balanced to move forward more quickly, and we can revisit later.

GFM will be used for all handwritten content like the TOCs in READMEs and proposals inside the repo that are not in GitHub issues obviously.

I'm not a huge fan of compromise, or splitting across two different technologies. However, this allows us to move forward.

Final thoughts before I close the issue?

[mahmoudimus]

@whit537 @bcm Thanks for your comments! I completely agree with you both, Github's documentation looks awesome. However, I have to ask @matin to elaborate on how he wants to display the API spec before I can discuss this further.

• Does @matin want the api spec to be viewable from github's repository viewer?
• Does @matin want to host the api spec on a github page?

This will help make the goal of this discussion clearer as those two choices will open up other paths of thought.

@whit537, as a reference for great, navigatable, aesthetically pleasing documentation written in reST, take a look at Flask's documentation.

@whit537, we leverage Sphinx's python parsing tools, extend it to understand our in house tools, and use the auto-doc module to spit out a very maintainable, reflective specification of the current api code. We do not do much manual orientiation for a few reasons:

• avoid cognitive overhead of developers adding a field, restriction, or changing any of the guarnatees we're trying to make with Balanced.
• if a system is guaranteed to reproduce its specifications programmatically, it follows that an external automated system can also validate these specifications by comparing them to a set of externally available automatically generated master specifications -- providing guarantees on interfaces.

[mahmoudimus]

@matin just saw your comments. If you have a moment to answer the two questions I posted above, I'd love to make another point.