Support basic charts

[damithc]

Suggestions: Provide support (likely through a third-party plugin) to add basic charts (e.g., bar/line/pie ... charts) using text sources (or csv files), similar to how we support basic UML diagrams.

---

(Updating the description to include the TODO from the attempt in #2052

• Add optional mermaid plugin for alternative diagram & chart support

---

(Updating the description to include conclusions/TODOs from the discussions below, see #1663 (comment))

1. Include documentation/examples on how it is possible to utilize some of these charting libraries
2. Extract out a few common chart types with minimalistic options to allow for really simple use cases.
   • E.g. a way to create a piechart by doing something like <pie-chart>data = [1,2,3]</pie-chart>.
   • The user need not know what is the underlying charting library in use, and the user is limited in terms of the options provided.
3. Create a general abstraction to simplify the typical steps required

Elaboration on the last point:

1. import (help import the script according to the type defined)
2. define a location to place the chart (user define the data/option right here, or pass in from an external file via data attribute)
3. init with a data/option object (help pass the data/option and generate the required script to append to the page)

Example syntax:

<chart type="chartjs">
{
    type: 'bar',
    data: {
        labels: ['Red', 'Blue', 'Yellow', 'Green', 'Purple', 'Orange'],
        datasets: [{
            label: '# of Votes',
            data: [12, 19, 3, 5, 2, 3],
        }]
    },
    options: {
        scales: {
            y: {
                beginAtZero: true
            }
        }
    }
}
</chart>
<chart type="chartjs" data="/chart2.json"></chart>

[tlylt]

Related:

• Strong support for diagrams #984
• WIP: Adding mermaid integration for support of more diagrams #1079

I did a brief search and it seems like having integration with mermaid would be nice (Gitlab has native support for mermaid.js syntax). It supports pie chart, among other things.

Also, it seems like we were trying to integrate it (in #1079 @crphang will you be continuing on it?)
If not I don't mind having a go at this if this feature is desirable

[damithc]

I did a brief search and it seems like having integration with mermaid would be nice (Gitlab has native support for mermaid.js syntax). It supports pie chart, among other things.

Thanks for looking into this @tlylt
I don't mind supporting mermaid. My concern is that it has a large overlap with plantUML that we already support. The only extra thing I could spot was the pie chart. Still, we can support it if the cost is not too high.

A higher priority for me is to add support for more charts such as pie, line, bar, histogram etc.

Also, it seems like we were trying to integrate it (in #1079 @crphang will you be continuing on it?)

Unlikely, as he hasn't been active for quite a while now.

[tlylt]

A higher priority for me is to add support for more charts such as pie, line, bar, histogram etc.

For supporting basic charts, I am thinking we can support multiple javascript charting libraries like chart.js, in the same fashion as this markdown-it plugin:

1. within the markdown, write the data as json in a code block (language info is the chart library name):

\`\`\`chartjs
{
  "type": "pie",
  "data": {
    "labels": [
      "Red",
      "Blue",
      "Yellow"
    ],
    "datasets": [
      {
        "data": [
          300,
          50,
          100
        ],
        "backgroundColor": [
          "#FF6384",
          "#36A2EB",
          "#FFCE56"
        ],
        "hoverBackgroundColor": [
          "#FF6384",
          "#36A2EB",
          "#FFCE56"
        ]
      }
    ]
  },
  "options": {}
}
\`\`\`

2. transform it into the respective HTML code
3. inject the CDN script of the required charting library in the page

I will need to test out the plugin to see if we can use it as is or make our own with a similar workflow.
Any comments or alternative plans on this are welcome :)

[crphang]

@tlylt I will not be working on it. Go ahead 💪 !

[ang-zeyu]

1. The primary thing to keep in mind with this and Strong support for diagrams #984 is author usability.

A lot of these external libraries require that you provide options as a json configuration object, from my very brief scan of chart.js this seems to be the case as well.
This flexibility is great for general app development but may be a little clunky for markbind. (versus something like inline puml where we can assume users have puml knowledge)
Not to say we can't expose the original configuration format to the author, there's also value in directly exposing it so that this is more maintainable; Abstracting everything into simpler interfaces is going to be costly.

E.g. for the data option, per the OP, we should at least consider transforming data from external sources easily, and not require that this
data: [45, 25, 20, 10], be manually written. Default configurations could be provided / added as well for the most common use cases so the json configuration object need not (or at least, less of it) be manually written.

Ergo also to keep in mind: if you see any rather complex-to-use configuration options (that have use cases in markbind) that ought to be extracted, you could consider whether we need to abstract over it.

2. Also for data: https://markbind.org/userGuide/reusingContents.html#importing-variables-from-other-external-file-formats

We currently already have the above lower level ways of importing data from json / csv files.
This could be linked up with the data option in chart.js easily.

<chart>
type: 'line',
data: {
  datasets: [{
    data: [
      {%for xx in ... %}{{xx}},{%endfor %}   // create the array and commas
    ]
  }]
}
</chart>

❗ ❗ From an author perspective, I'm not sure if this is enough or some abstraction should be done over this particular part (supplying data). For example, <line-chart src="foobar.csv">.

@damithc @tlylt

We can also consider starting out with this usage (i.e. the author needs to know how to supply the data option) then abstracting over it later.

---

write the data as json in a code block (language info is the chart library name):

https://markbind.org/devdocs/devGuide/writingPlugins.html#tag-behaviour

You could use this ^ to define your own "special tags". Its what we're using for inline <puml> tags currently, so that "conflicting syntax" (with html / md) can be inside.

It follows the same parsing rules as <script / style>, and being html tags you can provide configuration options as just simple attributes.

We should avoid mixing triple backticks syntax as its use case is well established for code blocks already.

[damithc]

❗ ❗ From an author perspective, I'm not sure if this is enough or some abstraction should be done over this particular part (supplying data). For example, <line-chart src="foobar.csv">.

@damithc @tlylt

We can also consider starting out with this usage (i.e. the author needs to know how to supply the data option) then abstracting over it later.

Fully agree. We can start with this kind of support first and add more user-friendly syntax for more common chart types later.
i.e., it's in line with the principle 'make common things easy, make rare things possible'

[tlylt]

Recent update: GitHub now has native support for Mermaid: https://github.blog/2022-02-14-include-diagrams-markdown-files-mermaid/

Perhaps it's mentioned somewhere but I can't really find it...why did we settle on creating custom HTML tags like <puml> instead of putting it as part of the code fence?
From an author's perspective writing mainly markdown:

• code fence is a common markdown feature and the understanding of specifying a language after ``` is commonplace as well
• whether is it puml or mermaid, they are actually lines of code so would be good to put them in the code fence for potential syntax highlighting as well as just easy of viewing, especially on a standard markdown renderer(e.g on GitHub/GitLab/VSCode interfaces)
• writing ```puml is arguably more "smooth" than writing <puml>
• given the general support on major platforms like GitHub/GitLab, and potentially other places going forward, ease of migration will definitely be a selling point

[damithc]

@tlylt Isn't the purpose of fenced code feature is to show syntax-highlighted code? If we use the same syntax for embedding diagrams, how do you distinguish between the two features? But yes, given that neither puml nor mermaid has syntax highlighting support as of now, seems like it can work, and I'm OK to support the same for the sake of compatibility.

[damithc]

In any case, given GitHub already supports mermaid, I think we'd better start supporting it too :-p

[tlylt]

@tlylt Isn't the purpose of fenced code feature is to show syntax-highlighted code? If we use the same syntax for embedding diagrams, how do you distinguish between the two features? But yes, given that neither puml nor mermaid has syntax highlighting support as of now, seems like it can work, and I'm OK to support the same for the sake of compatibility.

hmm I see your point now... indeed if someone wants to just show the raw puml/mermaid code with syntax highlight support (may not be available at the moment), I guess there is no way to do it if we use code fence. The only workaround is to not specify the language info.

[ang-zeyu]

Relevant PR: #968

code fence is a common markdown feature and the understanding of specifying a language after ``` is commonplace as well

Yup, but for syntax highlighting.

At the time of this PR (2 years back) ^, ```mermaid was never really "standard". <puml> syntax was our very first choice since it is more in line with MarkBind's components in general.

My personal 2 cents today as well still is that markdown code blocks shouldn't be used to generate anything other than code blocks, its mixing use cases and syntaxes that have nothing to do with each other semantically. Also, not quite commonmark compliant in that triple backticks/tildes should generate fenced code blocks.

writing ```puml is arguably more "smooth" than writing

Another smaller point raised (in the pr) was that specifying <puml> attributes/options (e.g. name) using markdown-it-attrs isn't quite as familiar / smooth as html attributes. (although, we do use it quite frequently for other code block features now)

In any case since github has started supporting this syntax, I agree we should support it as well (or at least the option of it). We can create a simple markdown plugin addition to replicate this functionality quite easily (i.e. transform ```puml to <puml>). A plugin system addition to add markdown plugins from our plugins would be welcome first too though!