[Documentation] Replace recommonmark by myst-parser #65664

cor3ntin · 2023-09-07T19:47:37Z

This is a bit rough, but i did that while trying to figure out if there would be a good way to do
pre commit checks on RsT files.

Recommonmark has been deprecated, then archived last year. This was tracked by: llvm/llvm-iwg#30

https://github.com/readthedocs/recommonmark

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

If you own a bot that build documentation, please run:

pip3 install sphinx myst-parser sphinx-automodapi furo

in the build environment. Thanks !

tru · 2023-09-08T06:37:42Z

Seems like a nice cleanup on top of the fact that we don't have to use a deprecated module. I am guessing it didn't cause any output errors.

Regarding build-bots, I am unsure where we automatically build the documentation. Pushing the docs to the main branch will rebuild and put them on llvm.org, But not sure where that happens. Maybe we should have a requirements.txt for documentation building and add that somewhere in the tree and have our bots always install that instead of each bot having its own list of python modules.

cc @tstellar

cor3ntin · 2023-09-08T07:10:44Z

Agreed. I have found that the full set of needed pip modules is
sphinx myst-parser sphinx-markdown-tables sphinx-automodapi furo - I think most of theses are only used by LLDB but might as well document them as requirements for everything.

tstellar

LGTM. @andreil99 Are you maintaining the documentation bots?

andreil99 · 2023-09-12T05:12:47Z

I can update the publishing bots. For the bots running on https://lab.llvm.org/buildbot/#/workers/92 we would need @gribozavr or somebody with access to gribozavr3 worker.

@cor3ntin, is there any dependency on a particular version of myst-parser?

tru · 2023-09-12T08:44:21Z

Instead of updating the module manually on the bot, I still think it's a good idea to add a requirements.txt that should be installed before running the build, ideally in a virtualenv. WDYT @andreil99

llvmbot · 2023-09-12T09:38:36Z

@llvm/pr-subscribers-github-workflow

Changes

This is a bit rough, but i did that while trying to figure out if there would be a good way to do
pre commit checks on RsT files.

Recommonmark has been deprecated, then archived last year. This was tracked by: llvm/llvm-iwg#30

https://github.com/readthedocs/recommonmark

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

Full diff: https://github.com/llvm/llvm-project/pull/65664.diff

8 Files Affected:

(modified) .github/workflows/release-tasks.yml (+4-6)
(modified) clang/docs/conf.py (+4-19)
(modified) flang/docs/conf.py (+4-52)
(modified) llvm/docs/MarkdownQuickstartTemplate.md (+2-2)
(modified) llvm/docs/SphinxQuickstartTemplate.rst (+3-11)
(modified) llvm/docs/conf.py (+5-15)
(added) llvm/docs/requirements.txt (+5)
(modified) llvm/utils/release/build-docs.sh (+4-4)

diff --git a/.github/workflows/release-tasks.yml b/.github/workflows/release-tasks.yml
index 5604b48857dbe25..4d0e6e93d57108b 100644
--- a/.github/workflows/release-tasks.yml
+++ b/.github/workflows/release-tasks.yml
@@ -24,6 +24,9 @@ jobs:
           release_version=$(echo "${{ github.ref_name }}" | sed 's/llvmorg-//g')
           echo "release-version=$release_version" >> $GITHUB_OUTPUT
 
+      - name: Checkout LLVM
+        uses: actions/checkout@v3
+
       - name: Install Dependencies
         run: |
           sudo apt-get update
@@ -31,14 +34,9 @@ jobs:
               doxygen \
               graphviz \
               python3-github \
-              python3-recommonmark \
-              python3-sphinx \
               ninja-build \
               texlive-font-utils
-          pip3 install --user sphinx-markdown-tables
-
-      - name: Checkout LLVM
-        uses: actions/checkout@v3
+          pip3 install --user -r ./llvm/docs/requirements.txt
 
       - name: Create Release
         run: |
diff --git a/clang/docs/conf.py b/clang/docs/conf.py
index de31a5dcd068eac..ca310026f53e2a9 100644
--- a/clang/docs/conf.py
+++ b/clang/docs/conf.py
@@ -32,26 +32,11 @@
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
 
-# The suffix of source filenames.
-source_suffix = {
-    ".rst": "restructuredtext",
-}
 
-try:
-    import recommonmark
-except ImportError:
-    # manpages do not use any .md sources
-    if not tags.has("builder-man"):
-        raise
-else:
-    import sphinx
-
-    if sphinx.version_info >= (3, 0):
-        # This requires 0.5 or later.
-        extensions.append("recommonmark")
-    else:
-        source_parsers = {".md": "recommonmark.parser.CommonMarkParser"}
-    source_suffix[".md"] = "markdown"
+import sphinx
+
+if sphinx.version_info >= (3, 0):
+    extensions.append("myst_parser")
 
 # The encoding of source files.
 # source_encoding = 'utf-8-sig'
diff --git a/flang/docs/conf.py b/flang/docs/conf.py
index 117cd1f1c97aea2..9950c9176c38798 100644
--- a/flang/docs/conf.py
+++ b/flang/docs/conf.py
@@ -9,10 +9,7 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 
-import sys, os
 from datetime import date
-from recommonmark.parser import CommonMarkParser
-
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
@@ -20,13 +17,6 @@
 
 # -- General configuration -----------------------------------------------------
 
-# https://github.com/readthedocs/recommonmark/issues/177
-# Method used to remove the warning message.
-class CustomCommonMarkParser(CommonMarkParser):
-    def visit_document(self, node):
-        pass
-
-
 # If your documentation needs a minimal Sphinx version, state it here.
 # needs_sphinx = '1.0'
 # Add any Sphinx extension module names here, as strings. They can be extensions
@@ -36,49 +26,11 @@ def visit_document(self, node):
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
 
-# The suffix of source filenames.
-source_suffix = {
-    ".rst": "restructuredtext",
-}
-try:
-    import recommonmark
-except ImportError:
-    # manpages do not use any .md sources
-    if not tags.has("builder-man"):
-        raise
-else:
-    import sphinx
-
-    if sphinx.version_info >= (3, 0):
-        # This requires 0.5 or later.
-        extensions.append("recommonmark")
-    else:
-        source_parsers = {".md": CustomCommonMarkParser}
-    source_suffix[".md"] = "markdown"
-    extensions.append("sphinx_markdown_tables")
-
-    # Setup AutoStructify for inline .rst toctrees in index.md
-    from recommonmark.transform import AutoStructify
-
-    # Stolen from https://github.com/readthedocs/recommonmark/issues/93
-    # Monkey patch to fix recommonmark 0.4 doc reference issues.
-    from recommonmark.states import DummyStateMachine
-
-    orig_run_role = DummyStateMachine.run_role
-
-    def run_role(self, name, options=None, content=None):
-        if name == "doc":
-            name = "any"
-            return orig_run_role(self, name, options, content)
-
-    DummyStateMachine.run_role = run_role
-
-    def setup(app):
-        # Disable inline math to avoid
-        # https://github.com/readthedocs/recommonmark/issues/120 in Extensions.md
-        app.add_config_value("recommonmark_config", {"enable_inline_math": False}, True)
-        app.add_transform(AutoStructify)
+import sphinx
 
+if sphinx.version_info >= (3, 0):
+    extensions.append("myst_parser")
+extensions.append("sphinx_markdown_tables")
 
 # The encoding of source files.
 # source_encoding = 'utf-8-sig'
diff --git a/llvm/docs/MarkdownQuickstartTemplate.md b/llvm/docs/MarkdownQuickstartTemplate.md
index 8d6434afe84ef39..add5985d5b40bb2 100644
--- a/llvm/docs/MarkdownQuickstartTemplate.md
+++ b/llvm/docs/MarkdownQuickstartTemplate.md
@@ -151,10 +151,10 @@ without any syntax highlighting like this:
 
 If you need to do fancier things than what has been shown in this document,
 you can mail the list or check the [Common Mark spec].  Sphinx specific
-integration documentation can be found in the [recommonmark docs].
+integration documentation can be found in the [myst-parser docs].
 
 [Common Mark spec]: http://spec.commonmark.org/0.28/
-[recommonmark docs]: http://recommonmark.readthedocs.io/en/latest/index.html
+[myst-parser docs]: https://myst-parser.readthedocs.io/en/latest/
 
 ## Generating the documentation
 
diff --git a/llvm/docs/SphinxQuickstartTemplate.rst b/llvm/docs/SphinxQuickstartTemplate.rst
index db9bd26bb3d9a4a..956adabce78c24b 100644
--- a/llvm/docs/SphinxQuickstartTemplate.rst
+++ b/llvm/docs/SphinxQuickstartTemplate.rst
@@ -171,20 +171,12 @@ Generating the documentation
 You can generate the HTML documentation from the sources locally if you want to
 see what they would look like. In addition to the normal
 `build tools `_
-you need to install `Sphinx`_ and the
-`recommonmark `_ extension.
-
-On Debian you can install these with:
-
-.. code-block:: console
-
-   sudo apt install -y sphinx-doc python-recommonmark-doc
-
-On Ubuntu use pip to get an up-to-date version of recommonmark:
+you need to install `Sphinx`_ and the necessary extensions
+using the following command inside the ``llvm-project`` checkout:
 
 .. code-block:: console
 
-   sudo pip install sphinx recommonmark
+   pip install --user -r ./llvm/docs/requirements.txt
 
 Then run cmake to build the documentation inside the ``llvm-project`` checkout:
 
diff --git a/llvm/docs/conf.py b/llvm/docs/conf.py
index 206f72285a830c2..5a11cc26f0e757b 100644
--- a/llvm/docs/conf.py
+++ b/llvm/docs/conf.py
@@ -36,21 +36,11 @@
     ".rst": "restructuredtext",
 }
 
-try:
-    import recommonmark
-except ImportError:
-    # manpages do not use any .md sources
-    if not tags.has("builder-man"):
-        raise
-else:
-    import sphinx
-
-    if sphinx.version_info >= (3, 0):
-        # This requires 0.5 or later.
-        extensions.append("recommonmark")
-    else:
-        source_parsers = {".md": "recommonmark.parser.CommonMarkParser"}
-    source_suffix[".md"] = "markdown"
+import sphinx
+
+if sphinx.version_info >= (3, 0):
+    extensions.append("myst_parser")
+extensions.append("sphinx_markdown_tables")
 
 # The encoding of source files.
 # source_encoding = 'utf-8-sig'
diff --git a/llvm/docs/requirements.txt b/llvm/docs/requirements.txt
new file mode 100644
index 000000000000000..fbfd258178c8ecb
--- /dev/null
+++ b/llvm/docs/requirements.txt
@@ -0,0 +1,5 @@
+sphinx
+myst-parser
+sphinx-markdown-tables
+sphinx-automodapi
+furo
diff --git a/llvm/utils/release/build-docs.sh b/llvm/utils/release/build-docs.sh
index ef3784959b4f3aa..fd226434b95a3bf 100755
--- a/llvm/utils/release/build-docs.sh
+++ b/llvm/utils/release/build-docs.sh
@@ -11,13 +11,13 @@
 #
 # Required Packages:
 # * Fedora:
-#   * dnf install doxygen python3-sphinx texlive-epstopdf ghostscript \
+#   * dnf install doxygen texlive-epstopdf ghostscript \
 #                 ninja-build gcc-c++
-#   * pip install sphinx-markdown-tables
+#   * pip install --user -r ./llvm/docs/requirements.txt
 # * Ubuntu:
-#   * apt-get install doxygen sphinx-common python3-recommonmark \
+#   * apt-get install doxygen \
 #             ninja-build graphviz texlive-font-utils
-#   * pip install sphinx-markdown-tables
+#   * pip install --user -r ./llvm/docs/requirements.txt
 #===------------------------------------------------------------------------===#
 
 set -ex

llvmbot · 2023-09-12T09:38:38Z

@llvm/pr-subscribers-clang

Changes

This is a bit rough, but i did that while trying to figure out if there would be a good way to do
pre commit checks on RsT files.

Recommonmark has been deprecated, then archived last year. This was tracked by: llvm/llvm-iwg#30

https://github.com/readthedocs/recommonmark

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

Full diff: https://github.com/llvm/llvm-project/pull/65664.diff

8 Files Affected:

(modified) .github/workflows/release-tasks.yml (+4-6)
(modified) clang/docs/conf.py (+4-19)
(modified) flang/docs/conf.py (+4-52)
(modified) llvm/docs/MarkdownQuickstartTemplate.md (+2-2)
(modified) llvm/docs/SphinxQuickstartTemplate.rst (+3-11)
(modified) llvm/docs/conf.py (+5-15)
(added) llvm/docs/requirements.txt (+5)
(modified) llvm/utils/release/build-docs.sh (+4-4)

diff --git a/.github/workflows/release-tasks.yml b/.github/workflows/release-tasks.yml
index 5604b48857dbe25..4d0e6e93d57108b 100644
--- a/.github/workflows/release-tasks.yml
+++ b/.github/workflows/release-tasks.yml
@@ -24,6 +24,9 @@ jobs:
           release_version=$(echo "${{ github.ref_name }}" | sed 's/llvmorg-//g')
           echo "release-version=$release_version" >> $GITHUB_OUTPUT
 
+      - name: Checkout LLVM
+        uses: actions/checkout@v3
+
       - name: Install Dependencies
         run: |
           sudo apt-get update
@@ -31,14 +34,9 @@ jobs:
               doxygen \
               graphviz \
               python3-github \
-              python3-recommonmark \
-              python3-sphinx \
               ninja-build \
               texlive-font-utils
-          pip3 install --user sphinx-markdown-tables
-
-      - name: Checkout LLVM
-        uses: actions/checkout@v3
+          pip3 install --user -r ./llvm/docs/requirements.txt
 
       - name: Create Release
         run: |
diff --git a/clang/docs/conf.py b/clang/docs/conf.py
index de31a5dcd068eac..ca310026f53e2a9 100644
--- a/clang/docs/conf.py
+++ b/clang/docs/conf.py
@@ -32,26 +32,11 @@
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
 
-# The suffix of source filenames.
-source_suffix = {
-    ".rst": "restructuredtext",
-}
 
-try:
-    import recommonmark
-except ImportError:
-    # manpages do not use any .md sources
-    if not tags.has("builder-man"):
-        raise
-else:
-    import sphinx
-
-    if sphinx.version_info >= (3, 0):
-        # This requires 0.5 or later.
-        extensions.append("recommonmark")
-    else:
-        source_parsers = {".md": "recommonmark.parser.CommonMarkParser"}
-    source_suffix[".md"] = "markdown"
+import sphinx
+
+if sphinx.version_info >= (3, 0):
+    extensions.append("myst_parser")
 
 # The encoding of source files.
 # source_encoding = 'utf-8-sig'
diff --git a/flang/docs/conf.py b/flang/docs/conf.py
index 117cd1f1c97aea2..9950c9176c38798 100644
--- a/flang/docs/conf.py
+++ b/flang/docs/conf.py
@@ -9,10 +9,7 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 
-import sys, os
 from datetime import date
-from recommonmark.parser import CommonMarkParser
-
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
@@ -20,13 +17,6 @@
 
 # -- General configuration -----------------------------------------------------
 
-# https://github.com/readthedocs/recommonmark/issues/177
-# Method used to remove the warning message.
-class CustomCommonMarkParser(CommonMarkParser):
-    def visit_document(self, node):
-        pass
-
-
 # If your documentation needs a minimal Sphinx version, state it here.
 # needs_sphinx = '1.0'
 # Add any Sphinx extension module names here, as strings. They can be extensions
@@ -36,49 +26,11 @@ def visit_document(self, node):
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
 
-# The suffix of source filenames.
-source_suffix = {
-    ".rst": "restructuredtext",
-}
-try:
-    import recommonmark
-except ImportError:
-    # manpages do not use any .md sources
-    if not tags.has("builder-man"):
-        raise
-else:
-    import sphinx
-
-    if sphinx.version_info >= (3, 0):
-        # This requires 0.5 or later.
-        extensions.append("recommonmark")
-    else:
-        source_parsers = {".md": CustomCommonMarkParser}
-    source_suffix[".md"] = "markdown"
-    extensions.append("sphinx_markdown_tables")
-
-    # Setup AutoStructify for inline .rst toctrees in index.md
-    from recommonmark.transform import AutoStructify
-
-    # Stolen from https://github.com/readthedocs/recommonmark/issues/93
-    # Monkey patch to fix recommonmark 0.4 doc reference issues.
-    from recommonmark.states import DummyStateMachine
-
-    orig_run_role = DummyStateMachine.run_role
-
-    def run_role(self, name, options=None, content=None):
-        if name == "doc":
-            name = "any"
-            return orig_run_role(self, name, options, content)
-
-    DummyStateMachine.run_role = run_role
-
-    def setup(app):
-        # Disable inline math to avoid
-        # https://github.com/readthedocs/recommonmark/issues/120 in Extensions.md
-        app.add_config_value("recommonmark_config", {"enable_inline_math": False}, True)
-        app.add_transform(AutoStructify)
+import sphinx
 
+if sphinx.version_info >= (3, 0):
+    extensions.append("myst_parser")
+extensions.append("sphinx_markdown_tables")
 
 # The encoding of source files.
 # source_encoding = 'utf-8-sig'
diff --git a/llvm/docs/MarkdownQuickstartTemplate.md b/llvm/docs/MarkdownQuickstartTemplate.md
index 8d6434afe84ef39..add5985d5b40bb2 100644
--- a/llvm/docs/MarkdownQuickstartTemplate.md
+++ b/llvm/docs/MarkdownQuickstartTemplate.md
@@ -151,10 +151,10 @@ without any syntax highlighting like this:
 
 If you need to do fancier things than what has been shown in this document,
 you can mail the list or check the [Common Mark spec].  Sphinx specific
-integration documentation can be found in the [recommonmark docs].
+integration documentation can be found in the [myst-parser docs].
 
 [Common Mark spec]: http://spec.commonmark.org/0.28/
-[recommonmark docs]: http://recommonmark.readthedocs.io/en/latest/index.html
+[myst-parser docs]: https://myst-parser.readthedocs.io/en/latest/
 
 ## Generating the documentation
 
diff --git a/llvm/docs/SphinxQuickstartTemplate.rst b/llvm/docs/SphinxQuickstartTemplate.rst
index db9bd26bb3d9a4a..956adabce78c24b 100644
--- a/llvm/docs/SphinxQuickstartTemplate.rst
+++ b/llvm/docs/SphinxQuickstartTemplate.rst
@@ -171,20 +171,12 @@ Generating the documentation
 You can generate the HTML documentation from the sources locally if you want to
 see what they would look like. In addition to the normal
 `build tools `_
-you need to install `Sphinx`_ and the
-`recommonmark `_ extension.
-
-On Debian you can install these with:
-
-.. code-block:: console
-
-   sudo apt install -y sphinx-doc python-recommonmark-doc
-
-On Ubuntu use pip to get an up-to-date version of recommonmark:
+you need to install `Sphinx`_ and the necessary extensions
+using the following command inside the ``llvm-project`` checkout:
 
 .. code-block:: console
 
-   sudo pip install sphinx recommonmark
+   pip install --user -r ./llvm/docs/requirements.txt
 
 Then run cmake to build the documentation inside the ``llvm-project`` checkout:
 
diff --git a/llvm/docs/conf.py b/llvm/docs/conf.py
index 206f72285a830c2..5a11cc26f0e757b 100644
--- a/llvm/docs/conf.py
+++ b/llvm/docs/conf.py
@@ -36,21 +36,11 @@
     ".rst": "restructuredtext",
 }
 
-try:
-    import recommonmark
-except ImportError:
-    # manpages do not use any .md sources
-    if not tags.has("builder-man"):
-        raise
-else:
-    import sphinx
-
-    if sphinx.version_info >= (3, 0):
-        # This requires 0.5 or later.
-        extensions.append("recommonmark")
-    else:
-        source_parsers = {".md": "recommonmark.parser.CommonMarkParser"}
-    source_suffix[".md"] = "markdown"
+import sphinx
+
+if sphinx.version_info >= (3, 0):
+    extensions.append("myst_parser")
+extensions.append("sphinx_markdown_tables")
 
 # The encoding of source files.
 # source_encoding = 'utf-8-sig'
diff --git a/llvm/docs/requirements.txt b/llvm/docs/requirements.txt
new file mode 100644
index 000000000000000..fbfd258178c8ecb
--- /dev/null
+++ b/llvm/docs/requirements.txt
@@ -0,0 +1,5 @@
+sphinx
+myst-parser
+sphinx-markdown-tables
+sphinx-automodapi
+furo
diff --git a/llvm/utils/release/build-docs.sh b/llvm/utils/release/build-docs.sh
index ef3784959b4f3aa..fd226434b95a3bf 100755
--- a/llvm/utils/release/build-docs.sh
+++ b/llvm/utils/release/build-docs.sh
@@ -11,13 +11,13 @@
 #
 # Required Packages:
 # * Fedora:
-#   * dnf install doxygen python3-sphinx texlive-epstopdf ghostscript \
+#   * dnf install doxygen texlive-epstopdf ghostscript \
 #                 ninja-build gcc-c++
-#   * pip install sphinx-markdown-tables
+#   * pip install --user -r ./llvm/docs/requirements.txt
 # * Ubuntu:
-#   * apt-get install doxygen sphinx-common python3-recommonmark \
+#   * apt-get install doxygen \
 #             ninja-build graphviz texlive-font-utils
-#   * pip install sphinx-markdown-tables
+#   * pip install --user -r ./llvm/docs/requirements.txt
 #===------------------------------------------------------------------------===#
 
 set -ex

cor3ntin · 2023-09-12T09:40:26Z

Instead of updating the module manually on the bot, I still think it's a good idea to add a requirements.txt that should be installed before running the build, ideally in a virtualenv. WDYT @andreil99

I added a requirements.txt file in llvm/docs (and updated docs, scripts)

@cor3ntin, is there any dependency on a particular version of myst-parser?

I unfortunately have no idea, besides "work on my system" (ubuntu 23.04)

andreil99 · 2023-09-13T06:39:44Z

Thanks, @cor3ntin!

I have installed the latest myst-parser (2.0.0) on the publishing bot. Will see how it will fly.

Please feel free to commit requirements.txt you have. I'll update it with the one with known-to-be-good versions frozen once we will confirm on that bot myst-parser 2.0.0.

tru

Let's ship it! Maybe post a heads-up on discourse as well so that people are aware it will happen and if some bots fail how to fix it.

This is a bit rough, but i did that while trying to figure out if there would be a good way to do pre commit checks on RsT files. Recommonmark has been deprecated, then archived last year. This was tracked by: llvm/llvm-iwg#30 https://github.com/readthedocs/recommonmark Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

- Removed uneeded sphinx-markdown-tables package (myst handles table natively) - Fix llvm docs extension configuration

llvmbot · 2023-09-14T11:19:36Z

@llvm/pr-subscribers-openacc

Changes

This is a bit rough, but i did that while trying to figure out if there would be a good way to do pre commit checks on RsT files.

Recommonmark has been deprecated, then archived last year. This was tracked by: llvm/llvm-iwg#30

https://github.com/readthedocs/recommonmark

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

Patch is 24.25 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/65664.diff

45 Files Affected:

(modified) .github/workflows/release-tasks.yml (+4-6)
(modified) clang/docs/conf.py (+4-19)
(modified) flang/docs/Aliasing.md (+4-3)
(modified) flang/docs/ArrayComposition.md (+4-3)
(modified) flang/docs/BijectiveInternalNameUniquing.md (+4-3)
(modified) flang/docs/C++17.md (+4-3)
(modified) flang/docs/C++style.md (+4-3)
(modified) flang/docs/Calls.md (+4-3)
(modified) flang/docs/Character.md (+4-3)
(modified) flang/docs/ComplexOperations.md (+2-2)
(modified) flang/docs/ControlFlowGraph.md (+4-3)
(modified) flang/docs/DesignGuideline.md (+4-3)
(modified) flang/docs/DoConcurrent.md (+4-3)
(modified) flang/docs/Extensions.md (+4-3)
(modified) flang/docs/FIRArrayOperations.md (+4-3)
(modified) flang/docs/FlangDriver.md (+4-3)
(modified) flang/docs/FortranFeatureHistory.md (+4-3)
(modified) flang/docs/FortranForCProgrammers.md (+4-3)
(modified) flang/docs/FortranIR.md (+4-3)
(modified) flang/docs/FortranLLVMTestSuite.md (+4-3)
(modified) flang/docs/GettingInvolved.md (+4-3)
(modified) flang/docs/GettingStarted.md (+6-4)
(modified) flang/docs/IORuntimeInternals.md (+4-3)
(modified) flang/docs/ImplementingASemanticCheck.md (+4-3)
(modified) flang/docs/IntrinsicTypes.md (+4-3)
(modified) flang/docs/Intrinsics.md (+4-3)
(modified) flang/docs/LabelResolution.md (+4-3)
(modified) flang/docs/ModFiles.md (+4-3)
(modified) flang/docs/OpenACC.md (+4-3)
(modified) flang/docs/OpenMP-semantics.md (+4-3)
(modified) flang/docs/Overview.md (+4-3)
(modified) flang/docs/ParserCombinators.md (+4-3)
(modified) flang/docs/Parsing.md (+4-3)
(modified) flang/docs/Preprocessing.md (+4-3)
(modified) flang/docs/RuntimeDescriptor.md (+4-3)
(modified) flang/docs/RuntimeTypeInfo.md (+4-3)
(modified) flang/docs/Semantics.md (+4-3)
(modified) flang/docs/conf.py (+6-54)
(modified) flang/docs/index.md (+4-4)
(modified) llvm/docs/GwpAsan.rst (+1)
(modified) llvm/docs/MarkdownQuickstartTemplate.md (+2-2)
(modified) llvm/docs/SphinxQuickstartTemplate.rst (+3-11)
(modified) llvm/docs/conf.py (+3-20)
(added) llvm/docs/requirements.txt (+4)
(modified) llvm/utils/release/build-docs.sh (+4-4)

diff --git a/.github/workflows/release-tasks.yml b/.github/workflows/release-tasks.yml
index 013714005d1124e..656b8c49c556173 100644
--- a/.github/workflows/release-tasks.yml
+++ b/.github/workflows/release-tasks.yml
@@ -24,6 +24,9 @@ jobs:
           release_version=$(echo &quot;${{ github.ref_name }}&quot; | sed &#x27;s/llvmorg-//g&#x27;)
           echo &quot;release-version=$release_version&quot; &gt;&gt; &quot;$GITHUB_OUTPUT&quot;
 
+      - name: Checkout LLVM
+        uses: actions/checkout@v4
+
       - name: Install Dependencies
         run: |
           sudo apt-get update
@@ -31,14 +34,9 @@ jobs:
               doxygen \
               graphviz \
               python3-github \
-              python3-recommonmark \
-              python3-sphinx \
               ninja-build \
               texlive-font-utils
-          pip3 install --user sphinx-markdown-tables
-
-      - name: Checkout LLVM
-        uses: actions/checkout@v4
+          pip3 install --user -r ./llvm/docs/requirements.txt
 
       - name: Create Release
         run: |
diff --git a/clang/docs/conf.py b/clang/docs/conf.py
index de31a5dcd068eac..ca310026f53e2a9 100644
--- a/clang/docs/conf.py
+++ b/clang/docs/conf.py
@@ -32,26 +32,11 @@
 # Add any paths that contain templates here, relative to this directory.
 templates_path = [&quot;_templates&quot;]
 
-# The suffix of source filenames.
-source_suffix = {
-    &quot;.rst&quot;: &quot;restructuredtext&quot;,
-}
 
-try:
-    import recommonmark
-except ImportError:
-    # manpages do not use any .md sources
-    if not tags.has(&quot;builder-man&quot;):
-        raise
-else:
-    import sphinx
-
-    if sphinx.version_info &gt;= (3, 0):
-        # This requires 0.5 or later.
-        extensions.append(&quot;recommonmark&quot;)
-    else:
-        source_parsers = {&quot;.md&quot;: &quot;recommonmark.parser.CommonMarkParser&quot;}
-    source_suffix[&quot;.md&quot;] = &quot;markdown&quot;
+import sphinx
+
+if sphinx.version_info &gt;= (3, 0):
+    extensions.append(&quot;myst_parser&quot;)
 
 # The encoding of source files.
 # source_encoding = &#x27;utf-8-sig&#x27;
diff --git a/flang/docs/Aliasing.md b/flang/docs/Aliasing.md
index f2805c731477160..652b766541fd467 100644
--- a/flang/docs/Aliasing.md
+++ b/flang/docs/Aliasing.md
@@ -8,9 +8,10 @@
 
 # Aliasing in Fortran
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Introduction
diff --git a/flang/docs/ArrayComposition.md b/flang/docs/ArrayComposition.md
index 9e61abe5670f370..8de1c760d281a04 100644
--- a/flang/docs/ArrayComposition.md
+++ b/flang/docs/ArrayComposition.md
@@ -8,9 +8,10 @@
 
 # Array Composition
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 This note attempts to describe the motivation for and design of an
diff --git a/flang/docs/BijectiveInternalNameUniquing.md b/flang/docs/BijectiveInternalNameUniquing.md
index 996c490e7e19481..4a11626e3870a65 100644
--- a/flang/docs/BijectiveInternalNameUniquing.md
+++ b/flang/docs/BijectiveInternalNameUniquing.md
@@ -8,9 +8,10 @@
 
 # Bijective Internal Name Uniquing
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 FIR has a flat namespace. No two objects may have the same name at the module
diff --git a/flang/docs/C++17.md b/flang/docs/C++17.md
index 9e0120d2e4c5e69..b20364b03e060a4 100644
--- a/flang/docs/C++17.md
+++ b/flang/docs/C++17.md
@@ -8,9 +8,10 @@
 
 # C++14/17 features used in f18
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 The C++ dialect used in this project constitutes a subset of the
diff --git a/flang/docs/C++style.md b/flang/docs/C++style.md
index d4d692a522d0465..04579130aa7cbe9 100644
--- a/flang/docs/C++style.md
+++ b/flang/docs/C++style.md
@@ -8,9 +8,10 @@
 
 # Flang C++ Style Guide
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 This document captures the style guide rules that are followed in the Flang codebase.
diff --git a/flang/docs/Calls.md b/flang/docs/Calls.md
index cbf689f199ef52f..f518dc00ed8e817 100644
--- a/flang/docs/Calls.md
+++ b/flang/docs/Calls.md
@@ -8,9 +8,10 @@
 
 # Representation of Fortran function calls
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Procedure reference implementation protocol
diff --git a/flang/docs/Character.md b/flang/docs/Character.md
index 603dd8848ba1b99..4e1d40774d6db21 100644
--- a/flang/docs/Character.md
+++ b/flang/docs/Character.md
@@ -8,9 +8,10 @@
 
 # Implementation of `CHARACTER` types in f18
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Kinds and Character Sets
diff --git a/flang/docs/ComplexOperations.md b/flang/docs/ComplexOperations.md
index 6faa1811fd6d0d1..d035658b90acd1c 100644
--- a/flang/docs/ComplexOperations.md
+++ b/flang/docs/ComplexOperations.md
@@ -1,7 +1,7 @@
 # Complex Operations
 
-```eval_rst
-.. contents::
+```{eval-rst}
+.. toctree::
    :local:
 ```
 
diff --git a/flang/docs/ControlFlowGraph.md b/flang/docs/ControlFlowGraph.md
index dcdecf1b77f6545..35eb4c4798d6d74 100644
--- a/flang/docs/ControlFlowGraph.md
+++ b/flang/docs/ControlFlowGraph.md
@@ -8,9 +8,10 @@
 
 # Control Flow Graph
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Concept
diff --git a/flang/docs/DesignGuideline.md b/flang/docs/DesignGuideline.md
index 5c945cbf1300141..03357f21fd07eb4 100644
--- a/flang/docs/DesignGuideline.md
+++ b/flang/docs/DesignGuideline.md
@@ -7,9 +7,10 @@
 --&gt;
 # Design Guideline
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 ## Documenting the design
 
diff --git a/flang/docs/DoConcurrent.md b/flang/docs/DoConcurrent.md
index 14f6899e2c1c575..bd1008af86f6b4e 100644
--- a/flang/docs/DoConcurrent.md
+++ b/flang/docs/DoConcurrent.md
@@ -8,9 +8,10 @@
 
 # `DO CONCURRENT` isn&#x27;t necessarily concurrent
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 A variant form of Fortran&#x27;s primary looping construct was
diff --git a/flang/docs/Extensions.md b/flang/docs/Extensions.md
index 49e78a10fa6bcdb..df9b4fd4cade4a0 100644
--- a/flang/docs/Extensions.md
+++ b/flang/docs/Extensions.md
@@ -8,9 +8,10 @@
 
 # Fortran Extensions supported by Flang
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 As a general principle, this compiler will accept by default and
diff --git a/flang/docs/FIRArrayOperations.md b/flang/docs/FIRArrayOperations.md
index 7fec24c6e783840..bea49eae37a38ab 100644
--- a/flang/docs/FIRArrayOperations.md
+++ b/flang/docs/FIRArrayOperations.md
@@ -8,9 +8,10 @@
 
 # Design: FIR Array operations
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## General
diff --git a/flang/docs/FlangDriver.md b/flang/docs/FlangDriver.md
index da4ae8c91397333..6e48a332eafcc97 100644
--- a/flang/docs/FlangDriver.md
+++ b/flang/docs/FlangDriver.md
@@ -8,9 +8,10 @@
 
 # Flang drivers
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 There are two main drivers in Flang:
diff --git a/flang/docs/FortranFeatureHistory.md b/flang/docs/FortranFeatureHistory.md
index 184af189eee13a6..2c43a56e01502ef 100644
--- a/flang/docs/FortranFeatureHistory.md
+++ b/flang/docs/FortranFeatureHistory.md
@@ -8,9 +8,10 @@
 
 # A Fortran feature history cheat sheet
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Original IBM 704 FORTRAN
diff --git a/flang/docs/FortranForCProgrammers.md b/flang/docs/FortranForCProgrammers.md
index 572433ab7c15402..50c83ed7e9bfe2e 100644
--- a/flang/docs/FortranForCProgrammers.md
+++ b/flang/docs/FortranForCProgrammers.md
@@ -8,9 +8,10 @@
 
 # Fortran For C Programmers
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 This note is limited to essential information about Fortran so that
diff --git a/flang/docs/FortranIR.md b/flang/docs/FortranIR.md
index ed322bd06fbf937..f9f8f6416b37aa1 100644
--- a/flang/docs/FortranIR.md
+++ b/flang/docs/FortranIR.md
@@ -8,9 +8,10 @@
 
 # Design: Fortran IR
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Introduction
diff --git a/flang/docs/FortranLLVMTestSuite.md b/flang/docs/FortranLLVMTestSuite.md
index a6fffc8937ed808..62459e6a7b7020e 100644
--- a/flang/docs/FortranLLVMTestSuite.md
+++ b/flang/docs/FortranLLVMTestSuite.md
@@ -1,8 +1,9 @@
 # Fortran Tests in the LLVM Test Suite
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 The [LLVM Test Suite](https://github.com/llvm/llvm-test-suite) is a
diff --git a/flang/docs/GettingInvolved.md b/flang/docs/GettingInvolved.md
index 074564ceb355621..09f10fac7cf1b4c 100644
--- a/flang/docs/GettingInvolved.md
+++ b/flang/docs/GettingInvolved.md
@@ -7,9 +7,10 @@
 --&gt;
 # Getting Involved
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 The Flang Project welcomes contributions of all kinds.
diff --git a/flang/docs/GettingStarted.md b/flang/docs/GettingStarted.md
index f470067e07f6093..f5319a7c38541fd 100644
--- a/flang/docs/GettingStarted.md
+++ b/flang/docs/GettingStarted.md
@@ -8,9 +8,10 @@
 
 # Getting Started
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Building flang
@@ -443,7 +444,8 @@ system to create HTML pages which would be hosted on the webpage of flang and
 updated periodically.
 
 If you would like to generate and view the HTML locally:
-- Install [Sphinx](http://sphinx-doc.org/), including the [sphinx-markdown-tables](https://pypi.org/project/sphinx-markdown-tables/) extension.
+- Install [Sphinx](http://sphinx-doc.org/), and the required extensions
+  using `pip install --user -r ~/llvm-projects/docs/requirements.txt`
 - Pass `-DLLVM_ENABLE_SPHINX=ON -DSPHINX_WARNINGS_AS_ERRORS=OFF` to the cmake command.
 
 ```bash
diff --git a/flang/docs/IORuntimeInternals.md b/flang/docs/IORuntimeInternals.md
index 2748fcf16fa3c21..d4e321348b2debf 100644
--- a/flang/docs/IORuntimeInternals.md
+++ b/flang/docs/IORuntimeInternals.md
@@ -8,9 +8,10 @@
 
 # Fortran I/O Runtime Library Internal Design
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 This note is meant to be an overview of the design of the *implementation*
diff --git a/flang/docs/ImplementingASemanticCheck.md b/flang/docs/ImplementingASemanticCheck.md
index 4e19b041c392017..5b583d4f8031b89 100644
--- a/flang/docs/ImplementingASemanticCheck.md
+++ b/flang/docs/ImplementingASemanticCheck.md
@@ -7,9 +7,10 @@
 --&gt;
 # How to implement a Sematic Check in Flang
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 I recently added a semantic check to the Flang compiler front end.  This document
diff --git a/flang/docs/IntrinsicTypes.md b/flang/docs/IntrinsicTypes.md
index 3706b3f3e290ad0..fa9d64b377cdbc3 100644
--- a/flang/docs/IntrinsicTypes.md
+++ b/flang/docs/IntrinsicTypes.md
@@ -8,9 +8,10 @@
 
 # Implementation of `Intrinsic` types in f18
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 Intrinsic types are integer, real, complex, character, and logical.
diff --git a/flang/docs/Intrinsics.md b/flang/docs/Intrinsics.md
index 0128b4c96a5820f..ab0a940e53e5538 100644
--- a/flang/docs/Intrinsics.md
+++ b/flang/docs/Intrinsics.md
@@ -8,9 +8,10 @@
 
 # A categorization of standard (2018) and extended Fortran intrinsic procedures
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 This note attempts to group the intrinsic procedures of Fortran into categories
diff --git a/flang/docs/LabelResolution.md b/flang/docs/LabelResolution.md
index c1227a8bc35a10b..5e2fbe72172cc8d 100644
--- a/flang/docs/LabelResolution.md
+++ b/flang/docs/LabelResolution.md
@@ -8,9 +8,10 @@
 
 # Semantics: Resolving Labels and Construct Names
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Overview
diff --git a/flang/docs/ModFiles.md b/flang/docs/ModFiles.md
index ccb849ab0decd96..e55d72fa3a705bb 100644
--- a/flang/docs/ModFiles.md
+++ b/flang/docs/ModFiles.md
@@ -8,9 +8,10 @@
 
 # Module Files
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 Module files hold information from a module that is necessary to compile 
diff --git a/flang/docs/OpenACC.md b/flang/docs/OpenACC.md
index 2becfb1aeac1a63..80258041a627b93 100644
--- a/flang/docs/OpenACC.md
+++ b/flang/docs/OpenACC.md
@@ -8,9 +8,10 @@
 
 # OpenACC in Flang
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Intentional deviation from the specification
diff --git a/flang/docs/OpenMP-semantics.md b/flang/docs/OpenMP-semantics.md
index 6f42b44726e9378..57938afba62ddb3 100644
--- a/flang/docs/OpenMP-semantics.md
+++ b/flang/docs/OpenMP-semantics.md
@@ -8,9 +8,10 @@
 
 # OpenMP Semantic Analysis
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## OpenMP for F18
diff --git a/flang/docs/Overview.md b/flang/docs/Overview.md
index a65bd9c84780098..561e9cfcf95c34d 100644
--- a/flang/docs/Overview.md
+++ b/flang/docs/Overview.md
@@ -8,9 +8,10 @@
 
 # Overview of Compiler Phases
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 The Flang compiler transforms Fortran source code into an executable file. 
 This transformation proceeds in three high level phases -- analysis, lowering,
diff --git a/flang/docs/ParserCombinators.md b/flang/docs/ParserCombinators.md
index b00347396471edd..2c5652ec36138ba 100644
--- a/flang/docs/ParserCombinators.md
+++ b/flang/docs/ParserCombinators.md
@@ -8,9 +8,10 @@
 
 # Parser Combinators
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 This document is a primer on Parser Combinators and their use in Flang.
diff --git a/flang/docs/Parsing.md b/flang/docs/Parsing.md
index e960c33dcbf34c5..bedc1ea6aee9789 100644
--- a/flang/docs/Parsing.md
+++ b/flang/docs/Parsing.md
@@ -8,9 +8,10 @@
 
 # The F18 Parser
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 This program source code implements a parser for the Fortran programming
diff --git a/flang/docs/Preprocessing.md b/flang/docs/Preprocessing.md
index 620fa568d1a7390..7465ff538e42e2c 100644
--- a/flang/docs/Preprocessing.md
+++ b/flang/docs/Preprocessing.md
@@ -8,9 +8,10 @@
 
 # Fortran Preprocessing
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Behavior common to (nearly) all compilers:
diff --git a/flang/docs/RuntimeDescriptor.md b/flang/docs/RuntimeDescriptor.md
index f0bbd2e3fedaf6f..e6ce825b044022c 100644
--- a/flang/docs/RuntimeDescriptor.md
+++ b/flang/docs/RuntimeDescriptor.md
@@ -8,9 +8,10 @@
 
 # Runtime Descriptors
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Concept
diff --git a/flang/docs/RuntimeTypeInfo.md b/flang/docs/RuntimeTypeInfo.md
index 391b6ea5f88b773..8bd5551c6667875 100644
--- a/flang/docs/RuntimeTypeInfo.md
+++ b/flang/docs/RuntimeTypeInfo.md
@@ -8,9 +8,10 @@
 
 # The derived type runtime information table
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 ## Overview
diff --git a/flang/docs/Semantics.md b/flang/docs/Semantics.md
index 270a1033c4c9580..0fc1ebe4cff1d47 100644
--- a/flang/docs/Semantics.md
+++ b/flang/docs/Semantics.md
@@ -8,9 +8,10 @@
 
 # Semantic Analysis
 
-```eval_rst
-.. contents::
-   :local:
+```{contents}
+---
+local:
+---
 ```
 
 The semantic analysis pass determines if a syntactically correct Fortran
diff --git a/flang/docs/conf.py b/flang/docs/conf.py
index 117cd1f1c97aea2..dac089eeb37e9f3 100644
--- a/flang/docs/conf.py
+++ b/flang/docs/conf.py
@@ -9,10 +9,7 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 
-import sys, os
 from datetime import date
-from recommonmark.parser import CommonMarkParser
-
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
@@ -20,65 +17,20 @@
 
 # -- General configuration -----------------------------------------------------
 
-# https://github.com/readthedocs/recommonmark/issues/177
-# Method used to remove the warning message.
-class CustomCommonMarkParser(CommonMarkParser):
-    def visit_document(self, node):
-        pass
-
-
 # If your documentation needs a minimal Sphinx version, state it here.
 # needs_sphinx = &#x27;1.0&#x27;
 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named &#x27;sphinx.ext.*&#x27;) or your custom ones.
-extensions = [&quot;sphinx.ext.todo&quot;, &quot;sphinx.ext.mathjax&quot;, &quot;sphinx.ext.intersphinx&quot;]
+extensions = [&quot;myst_parser&quot;,
+              &quot;sphinx.ext.todo&quot;,
+              &quot;sphinx.ext.mathjax&quot;,
+              &quot;sphinx.ext.intersphinx&quot;,
+              &quot;sphinx.ext.autodoc&quot;]
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = [&quot;_templates&quot;]
 
-# The suffix of source filenames.
-source_suffix = {
-    &quot;.rst&quot;: &quot;restructuredtext&quot;,
-}
-try:
-    import recommonmark
-except ImportError:
-    # manpages do not use any .md sources
-    if not tags.has(&quot;builder-man&quot;):
-        raise
-else:
-    import sphinx
-
-    if sphinx.version_info &gt;= (3, 0):
-        # This requires 0.5 or later.
-        extensions.append(&quot;recommonmark&quot;)
-    else:
-        source_parsers = {&quot;.md&quot;: CustomCommonMarkParser}
-    source_suffix[&quot;.md&quot;] = &quot;markdow...

cor3ntin · 2023-09-14T19:19:36Z

@tru I'll try to do that later this week!

cor3ntin · 2023-09-16T10:00:59Z

@andreil99 did you update all the sphinx bots or just specific ones?

andreil99 · 2023-09-16T22:57:45Z

@cor3ntin No, I did not update all the sphinx bots, only specific ones, as I mentioned above in #65664 (comment)

I can update the publishing bots. For the bots running on https://lab.llvm.org/buildbot/#/workers/92 we would need @gribozavr or somebody with access to gribozavr3 worker.

cor3ntin · 2023-09-18T12:30:44Z

@andreil99 Sorry, I missed that.
I'd like to wait a few days for the bots to have the package installed first to minimize disruption.

This is in preparation towards the move from recommonmark to myst See #65664

reverts [Documentation] Replace recommonmark by myst-parser (llvm#65664) needs pip install magic in the builder Change-Id: I99886b000c8142c772388022f2172352938e5a3a

[Documentation] Replace recommonmark by myst-parser (llvm#65664) Recommonmark has been deprecated, then archived last year. This was tracked by: llvm/llvm-iwg#30 See https://github.com/readthedocs/recommonmark This patch migrates all our doc to use myst Additional details for bot maintainers: https://discourse.llvm.org/t/maintenance-required-on-sphinx-build-bots/73612 Change-Id: Id3803e8bb22d3e0fdc7c22bca0aef879edff6b2a

mgorny · 2024-02-17T08:14:49Z

I'm sorry for being late to the party but what's the rationale for removing support for building manpages without myst-parser, that FWICS isn't used for manpages at all? I don't see any answer neither in the commit message, nor in this thread. It really feels like you've just shrugged and removed it without any consideration for the prior effort to make things work better.

AaronBallman · 2024-02-20T17:23:42Z

I'm sorry for being late to the party but what's the rationale for removing support for building manpages without myst-parser, that FWICS isn't used for manpages at all? I don't see any answer neither in the commit message, nor in this thread. It really feels like you've just shrugged and removed it without any consideration for the prior effort to make things work better.

This is the first I've heard about this breaking anyone, can you give more information on what used to work for you and is now broken? Is myst-parser package not available for you?

mgorny · 2024-02-20T17:26:59Z

What used to work is that you could build manpages (that don't use any .md files) without installing additional dependencies. Now Sphinx requires myst-parser even when no .md files are used. I don't really care. However, Gentoo users do tend to complain whenever any single unnecessary package gets pulled in.

AaronBallman · 2024-02-20T17:41:54Z

Ah, thank you for the extra information! We definitely cannot go back to recommonmark, but perhaps @cor3ntin can find a way to restore the older behavior there. However, I think finding the minimal set of dependencies might be brittle in terms of long-term maintenance (so if it turns out it's significant effort to support the old behavior, I think the extra package may be the price we pay for getting off an unmaintained package).

mgorny · 2024-02-20T17:45:21Z

I've adjusted the Gentoo packages, and can live with it. I just find it sad that even though there was an explicit comment on why the ImportError fallback was added, it was just removed without a second thought or even an attempt to communicate with me (given git blame would have attributed the original change to me).

AaronBallman · 2024-02-20T18:06:18Z

I've adjusted the Gentoo packages, and can live with it. I just find it sad that even though there was an explicit comment on why the ImportError fallback was added, it was just removed without a second thought or even an attempt to communicate with me (given git blame would have attributed the original change to me).

I think it was simply overlooked by accident; nobody "owns" this functionality in community and so when we needed to get off the archived package, we took the offered patch which was based on advice from the recommonmark maintainers on how to port to myst-parser. Given that there's no post-commit bot testing this configuration, it's easy enough for this kind of thing to slip through the cracks. FWIW, if we do end up restoring the old behavior to help Gentoo folks out, I think we need some sort of test coverage for it and we'd appreciate your help in getting that set up so we don't regress again (assuming we're able to restore the old behavior).

The pan pages do not depend on the doc present in markdown files, so they can be built without myst_parser. Doing so might allow llvm distributions to have less development dependencies. As we do not have the ennvironment to test these configuration, this capability is provided on a best-effort basis. This restores a capability accidentally lost in llvm#65664.

The man pages do not depend on the doc present in markdown files, so they can be built without myst_parser. Doing so might allow llvm distributions to have less development dependencies. As we do not have the ennvironment to test these configuration, this capability is provided on a best-effort basis. This restores a capability accidentally lost in #65664.

cor3ntin added the infrastructure Bugs about LLVM infrastructure label Sep 7, 2023

cor3ntin requested review from a team as code owners September 7, 2023 19:47

github-actions bot added the flang Flang issues not falling into any other category label Sep 7, 2023

cor3ntin mentioned this pull request Sep 8, 2023

Linting Clang documentation #65725

Open

JDevlieghere self-requested a review September 8, 2023 22:30

tstellar approved these changes Sep 12, 2023

View reviewed changes

llvmbot added clang Clang issues not falling into any other category github:workflow labels Sep 12, 2023

tru approved these changes Sep 14, 2023

View reviewed changes

cor3ntin added 3 commits September 14, 2023 10:24

Put the documentation requirements in a requirement files.

c25d901

- Fix flang content directives directives

1f58fe7

- Removed uneeded sphinx-markdown-tables package (myst handles table natively) - Fix llvm docs extension configuration

cor3ntin force-pushed the remove_recommonmark branch from f9af8d9 to 1f58fe7 Compare September 14, 2023 11:18

llvmbot added the openacc label Sep 14, 2023

formatting

8381364

cor3ntin added a commit that referenced this pull request Sep 21, 2023

[Docs] Add requirements.txt for sphinx and extension

f7c4261

This is in preparation towards the move from recommonmark to myst See #65664

cor3ntin merged commit b7ff032 into llvm:main Sep 25, 2023
3 checks passed

wegank mentioned this pull request Feb 2, 2024

llvmPackages_git: 18.0.0-unstable-2023-12-13 -> 18.1.0-rc3 NixOS/nixpkgs#285786

Merged

13 tasks

cor3ntin mentioned this pull request Feb 20, 2024

[Docs] Allow building man pages without myst_parser #82402

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Documentation] Replace recommonmark by myst-parser #65664

[Documentation] Replace recommonmark by myst-parser #65664

cor3ntin commented Sep 7, 2023 •

edited

Loading

tru commented Sep 8, 2023

cor3ntin commented Sep 8, 2023

tstellar left a comment

andreil99 commented Sep 12, 2023

tru commented Sep 12, 2023

llvmbot commented Sep 12, 2023

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

llvmbot commented Sep 12, 2023

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

cor3ntin commented Sep 12, 2023

andreil99 commented Sep 13, 2023

tru left a comment

llvmbot commented Sep 14, 2023

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

cor3ntin commented Sep 14, 2023

cor3ntin commented Sep 16, 2023

andreil99 commented Sep 16, 2023

cor3ntin commented Sep 18, 2023

mgorny commented Feb 17, 2024

AaronBallman commented Feb 20, 2024

mgorny commented Feb 20, 2024

AaronBallman commented Feb 20, 2024

mgorny commented Feb 20, 2024

AaronBallman commented Feb 20, 2024

[Documentation] Replace recommonmark by myst-parser #65664

[Documentation] Replace recommonmark by myst-parser #65664

Conversation

cor3ntin commented Sep 7, 2023 • edited Loading

tru commented Sep 8, 2023

cor3ntin commented Sep 8, 2023

tstellar left a comment

Choose a reason for hiding this comment

andreil99 commented Sep 12, 2023

tru commented Sep 12, 2023

llvmbot commented Sep 12, 2023

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

llvmbot commented Sep 12, 2023

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

cor3ntin commented Sep 12, 2023

andreil99 commented Sep 13, 2023

tru left a comment

Choose a reason for hiding this comment

llvmbot commented Sep 14, 2023

Before merging that we would have to update the bots, which I'm not in a good position to do, so feel free to take over!

cor3ntin commented Sep 14, 2023

cor3ntin commented Sep 16, 2023

andreil99 commented Sep 16, 2023

cor3ntin commented Sep 18, 2023

mgorny commented Feb 17, 2024

AaronBallman commented Feb 20, 2024

mgorny commented Feb 20, 2024

AaronBallman commented Feb 20, 2024

mgorny commented Feb 20, 2024

AaronBallman commented Feb 20, 2024

cor3ntin commented Sep 7, 2023 •

edited

Loading