✨ NEW: Add path and project link schemes

Part of implementing jupyter-book/myst-enhancement-proposals#10

Author: chrisjsewell

docs/faq/index.md

@@ -54,7 +54,7 @@ The [MyST-NB](https://myst-nb.readthedocs.io) tool provides a Sphinx extension f
 ### Include a file from outside the docs folder (like README.md)
 
 :::{seealso}
-[...](#organising-content/include)
+<project:#organising-content/include>
 :::
 
 You can include a file, including one from outside the project using e.g.:
@@ -118,7 +118,7 @@ See [](#syntax/implicit-targets) for this information.
 
 ### Suppress warnings
 
-See [...](#myst-warnings) for this information.
+See <project:#myst-warnings> for this information.
 
 ### Sphinx-specific page front matter
 

docs/intro.md

@@ -107,7 +107,7 @@ There are a range of great HTML themes that work well with MyST, such as [sphinx
 
 ## Configuring MyST-Parser
 
-The [...](configuration.md) section contains a complete list of configuration options for the MyST-Parser.
+The <project:configuration.md> section contains a complete list of configuration options for the MyST-Parser.
 
 These can be applied globally, e.g. in the sphinx `conf.py`:
 

docs/syntax/code_and_apis.md

@@ -170,7 +170,7 @@ Sphinx and MyST provide means to analyse source code and automatically generate
 
 As opposed to `sphinx.ext.autodoc`, `sphinx-autodoc2` performs static (rather than dynamic) analysis of the source code, integrates full package documenting, and also allows for docstrings to be written in both RestructureText and MyST.
 
-The `auto_mode` will automatically generate the full API documentation, as shown [...](/apidocs/index.rst).
+The `auto_mode` will automatically generate the full API documentation, as shown <project:/apidocs/index.rst>.
 
 Alternatively, the `autodoc2-object` directive can be used to generate documentation for a single object.
 To embed in a MyST document the MyST `render_plugin` should be specified, for example:

docs/syntax/cross-referencing.md

@@ -78,7 +78,7 @@ For example, using `myst_heading_anchors = 2`:
 
 ## A heading with slug
 
-[...](#a-heading-with-slug)
+<project:#a-heading-with-slug>
 
 [Explicit title](#a-heading-with-slug-1)
 ::::
@@ -136,74 +136,84 @@ or an **internal** target, such as a file, heading or figure within the same pro
 
 By default, MyST will resolve link destinations according to the following rules:
 
-1. Destinations beginning with `inv:` will be treated as links to intersphinx references ([see below](#syntax/inv_links)).
+1. Destination beginning with a scheme (e.g. `xxx:`), will be handled according to that scheme:
 
-2. Autolinks or destinations beginning with  `http:`, `https:`, `ftp:`, or `mailto:` will be treated as external [URL] links.
+   {style=lower-roman}
+   1. Destinations beginning with `project:` will be treated as internal references
+   2. Destinations beginning with `path:` will be treated as downloadable files
+   3. Destinations beginning with `inv:` will be treated as intersphinx references
+   4. Autolinks or destinations beginning with  `http:`, `https:`, `ftp:`, or `mailto:` will be treated as external [URL] links.
 
-3. Destinations which point to a local file path are treated as links to that file.
+2. Destinations which point to a local file path are treated as links to that file.
 
-   - If the destination is a relative path, it is resolved relative to the current file.
-   - If the destination is an absolute path (starts with `/`), it is resolved relative to the root of the project (i.e. the source directory).
-   - If that path relates to another document in the project (e.g. a `.md` or `.rst` file), then it will link to the first heading in that document.
-   - Links to project documents can also include a `#` fragment identifier, to link to a specific heading in that document.
-   - If the path is to a non-source file (e.g. a `.png` or `.pdf` file), then the link will be to the file itself, e.g. to download it.
+   {style=lower-roman}
+   1. If the destination is a relative path, it is resolved relative to the current file.
+   2. If the destination is an absolute path (starts with `/`), it is resolved relative to the root of the project (i.e. the source directory).
+   3. If that path relates to another document in the project (e.g. a `.md` or `.rst` file), then it will link to the first heading in that document.
+   4. Links to project documents can also include a `#` fragment identifier, to link to a specific heading in that document.
+   5. If the path is to a non-source file (e.g. a `.png` or `.pdf` file), then the link will be to the file itself, e.g. to download it.
 
-4. Destinations beginning with `#` will be treated as internal references.
+3. Destinations beginning with `#` will be treated as internal references.
 
-   - First, explicit target in the same file are searched for, if not found
-   - Then, implicit targets in the same file are searched for, if not found
-   - Then, explicit targets across the whole project are searched for, if not found
-   - Then, intersphinx references are searched for, if not found
-   - A warning is emitted and the destination is left as an external link.
+   {style=lower-roman}
+   1. First, explicit targets in the same file are searched for, if not found
+   2. Then, implicit targets in the same file are searched for, if not found
+   3. Then, explicit targets across the whole project are searched for, if not found
+   4. Then, intersphinx references are searched for, if not found
+   5. A warning is emitted and the destination is left as an external link.
 
 :::{note}
 Local file path resolution and cross-project references are not available in [single page builds](#myst-docutils)
 :::
 
 ### Explicit vs implicit link text
 
-If the link text is explicitly given, e.g. `[text](dest)`, then the link text will be the given text.
+If the link text is explicitly given, e.g. `[text](#dest)`, then the rendered text will be that.
 This text can contain nested inline markup, such as `[*emphasis*](#syntax/emphasis)`{l=md}.
 
-If no text or `...` is given, e.g. `[](dest)` or `[...](dest)`, then MyST will attempt to resolve an implicit text.
+If no text is given or it is an auto-link, e.g. `[](#dest)` or `<project:#dest>`, then MyST will attempt to resolve an implicit text.
 For example, if the destination is a heading, then the heading text will be used as the link text,
 or if the destination is a figure/table then the caption will be used as the link text.
 Otherwise, the link text will be the destination itself.
 
-### Some examples
+### Examples
 
-Here are some examples:
+#### Autolinks
 
-:::{list-table}
-:header-rows: 1
+:::{myst-example}
 
-* - Type
-  - Syntax
-  - Rendered
+:External URL: <https://example.com>
+:Internal target reference: <project:#cross-references>
+:Internal file reference: <project:../intro.md>
+:Internal file -> heading reference: <project:../intro.md#-get-started>
+:Downloadable file: <path:example.txt>
+:Intersphinx reference: <inv:sphinx:std#index>
 
-* - Autolink
-  - `<https://example.com>`
-  - <https://example.com>
+:::
+
+#### Inline links with implicit text
+
+:::{myst-example}
 
-* - External URL
-  - `[example.com](https://example.com)`
-  - [example.com](https://example.com)
+:External URL: [](https://example.com)
+:Internal target reference: [](#cross-references)
+:Internal file reference: [](../intro.md)
+:Internal file -> heading reference: [](../intro.md#-get-started)
+:Downloadable file: [](example.txt)
+:Intersphinx reference: [](inv:sphinx:std#index)
 
-* - Document file
-  - `[Source file](../intro.md)`
-  - [Source file](../intro.md)
+:::
 
-* - Non-document file
-  - `[Non-source file](example.txt)`
-  - [Non-source file](example.txt)
+#### Inline links with explicit text
 
-* - Local heading
-  - `[...](#cross-references)`
-  - [...](#cross-references)
+:::{myst-example}
 
-* - Heading in other file
-  - `[Heading](../intro.md#installation)`
-  - [Heading](../intro.md#installation)
+:External URL: [Explicit text](https://example.com)
+:Internal target reference: [Explicit text](#cross-references)
+:Internal file reference: [Explicit text](../intro.md)
+:Internal file -> heading reference: [Explicit text](../intro.md#-get-started)
+:Downloadable file: [Explicit text](example.txt)
+:Intersphinx reference: [Explicit text](inv:sphinx:std#index)
 
 :::
 
@@ -341,23 +351,23 @@ Here are some examples:
   - Rendered
 
 * - Autolink, full
-  - `<inv:sphinx:std:doc#index>`
+  - `<inv:sphinx:std:doc#index>`{l=myst}
   - <inv:sphinx:std:doc#index>
 
 * - Link, full
-  - `[Sphinx](inv:sphinx:std:doc#index)`
+  - `[Sphinx](inv:sphinx:std:doc#index)`{l=myst}
   - [Sphinx](inv:sphinx:std:doc#index)
 
 * - Autolink, no type
-  - `<inv:sphinx:std#index>`
+  - `<inv:sphinx:std#index>`{l=myst}
   - <inv:sphinx:std#index>
 
 * - Autolink, no domain
-  - `<inv:sphinx:*:doc#index>`
+  - `<inv:sphinx:*:doc#index>`{l=myst}
   - <inv:sphinx:*:doc#index>
 
 * - Autolink, only name
-  - `<inv:#*.Sphinx>`
+  - `<inv:#*.Sphinx>`{l=myst}
   - <inv:#*.Sphinx>
 
 :::
@@ -371,21 +381,21 @@ These can also within MyST documents, although it is recommended to use the Mark
 :::{myst-example}
 - {ref}`syntax/referencing`, {ref}`Explicit text <syntax/referencing>`
 - {term}`my other term`
-- {doc}`../intro`
-- {download}`example.txt`
-- {py:class}`mypackage.MyClass`
-- {external:class}`sphinx.application.Sphinx`
-- {external+sphinx:ref}`code-examples`
+- {doc}`../intro`, {doc}`Explicit text <../intro>`
+- {download}`example.txt`, {download}`Explicit text <example.txt>`
+- {py:class}`mypackage.MyClass`, {py:class}`Explicit text <mypackage.MyClass>`
+- {external:class}`sphinx.application.Sphinx`, {external:class}`Explicit text <sphinx.application.Sphinx>`
+- {external+sphinx:ref}`code-examples`, {external+sphinx:ref}`Explicit text <code-examples>`
 
 ---
 
-- [...][syntax], [Explicit text][syntax]
-- [...](<#my other term>)
-- [...](../intro.md)
-- [...](example.txt)
-- [...](#mypackage.MyClass)
-- [...](#sphinx.application.Sphinx), [...](inv:#*Sphinx)
-- [...](inv:sphinx#code-examples)
+- <project:#syntax/referencing>, [][syntax], [Explicit text][syntax]
+- [](<#my other term>)
+- <project:../intro.md>, [Explicit text](../intro.md)
+- <path:example.txt>, [Explicit text](example.txt)
+- <project:#mypackage.MyClass>, [Explicit text](#mypackage.MyClass)
+- <inv:#*Sphinx>, [Explicit text](#sphinx.application.Sphinx)
+- <inv:sphinx#code-examples>, [Explicit text](inv:sphinx#code-examples)
 
 [syntax]: #syntax/referencing
 :::

docs/syntax/images_and_figures.md

@@ -119,7 +119,7 @@ figclass : text
 
 :::::{seealso}
 
-See the [...](#syntax/md-figures) section for information on how to create figures that use native Markdown images.
+See the <project:#syntax/md-figures> section for information on how to create figures that use native Markdown images.
 
 ::::{myst-example}
 :::{figure-md}

myst_parser/_docs.py

@@ -107,7 +107,6 @@ def field_type(field):
 
 
 class MystConfigDirective(_ConfigBase):
-
     option_spec = {
         "sphinx": directives.flag,
         "extensions": directives.flag,
@@ -120,7 +119,6 @@ def run(self):
         text = self.table_header()
         count = 0
         for name, value, field in config.as_triple():
-
             if field.metadata.get("deprecated"):
                 continue
 
@@ -378,7 +376,7 @@ def run(self):
 # TODO when some more work and testing, this should be made available publicly
 
 from pygments import token  # noqa: E402
-from pygments.lexer import bygroups, inherit  # noqa: E402
+from pygments.lexer import bygroups, inherit, this, using  # noqa: E402
 from pygments.lexers.markup import MarkdownLexer  # noqa: E402
 
 
@@ -412,12 +410,12 @@ class MystLexer(MarkdownLexer):
             ),
             # :name: value
             (
-                r"^(\:)([^\n]+)(\:)([^\n]+)(\n)",
+                r"^(\:)([^\n\:]+)(\:)([^\n]+)(\n)",
                 bygroups(
                     token.Punctuation,
-                    token.Name.Label,
+                    token.Generic.Strong,
                     token.Punctuation,
-                    token.Text,
+                    using(this, state="inline"),
                     token.Text,
                 ),
             ),
@@ -429,7 +427,18 @@ class MystLexer(MarkdownLexer):
             # {name}
             (
                 r"(\{)([a-zA-Z0-9+:-]+)(\})",
-                bygroups(token.Punctuation, token.Name.Label, token.Punctuation),
+                bygroups(token.Punctuation, token.Operator.Word, token.Punctuation),
+            ),
+            # <http:example.com>
+            (
+                r"(<)(http|https|mailto|project|path|inv)(\:)([^\s>]+)(>)",
+                bygroups(
+                    token.Punctuation,
+                    token.String.Other,
+                    token.String.Other,
+                    token.Name.Label,
+                    token.Punctuation,
+                ),
             ),
             inherit,
         ],