Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basis for fenced code block transformations, either synchronous or asynchronous #433

Open
wants to merge 24 commits into
base: master
Choose a base branch
from

Conversation

timmaffett
Copy link
Contributor

@timmaffett timmaffett commented May 20, 2022

@srawlins here is that PR of a possible approach to supporting further transformations of typed fenced code blocks.
This could of course be done entirely 'externally' to markdown... but I think there could be some advantages to supporting a 'standard' way of accomplishing this within the markdown package. This is an example of taking advantage of the change from PR #431.

As I said I created this in order to do static transforms of fenced code blocks containing text diagrams, but it could also be used for static code highlighting (without using highlight.js on the client), or any number of things. Link to example below.
(My thinking is that if we were to support text diagrams within dartdoc/pub.dev we would want to do so with static conversions of the text diagrams at the time the markdown was converted, for security and other considerations). dart-lang/pub-dev#5763 is another issue related to supporting diagrams.


There are two primary parts to this PR.

  • The classes [TransformableFencedCodeBlockSyntax] and [CodeBlockTransformer]. The [TransformableFencedCodeBlockSyntax] class is nearly identical to [FencedCodeBlockSyntax], except it supports a list of [CodeBlockTransformer]'s which can be used to accomplish synchronous or asynchronous transforms for the specified types of fenced code blocks.
  • The files async_transforms.dart and async_html_renderer.dart contain a simple [AsyncDocument] and [AsyncText] node class which can be used when the [CodeBlockTransformer] class need to return futures for asynchronous transforms. Strictly speaking the [AsyncDocument] class and the [markdownToHtmlWithAsyncTransforms] methods are very minor changes to their existing counterparts which could be rolled into the existing class/methods. This would allow the 'automatic' support of AsyncText nodes (and transformations). (The [AsyncDocument] class keeps a list of AsyncText nodes who's futures we want to ensure have completed before preforming the final conversion to html, and the [markdownToHtmlWithAsyncTransforms] method is identical to [markdownToHtml] except it checks this list to see if Future.wait() needs to be called to wait for all AsyncText nodes to complete).

The example directory contains an example of using these new classes to support static conversion of text diagrams to SVG files.
This is an example that addresses Issue #228 by accomplishing static conversion of text diagrams at the time the markdown is converted to html.
A running version of the example in the PR can be found here https://timmaffett.github.io/markdown_kroki/.

(The package:kroki is just a simple package i wrote to access the kroki.io web service. I thought kroki was advantageous because it is open source and could be run in-house with a restricted list of support diagram types).

Nothing in the example directory clearly needs to be merged, and is included in the PR just for illustration of using the new classes.

Here is example code of what it looks like to use these classes to extend markdown to support transformations of fenced code blocks containing text diagrams.

import 'package:kroki/kroki.dart';
import 'package:markdown/markdown.dart' as md;

final Kroki kroki = Kroki();

class DiagramTransfomer extends md.CodeBlockTransformer {

  @override
  md.Node? transformCodeBlock(String codeBlockType, 
                               String rawCodeBlock, md.BlockParser parser) {
      return md.AsyncText(
           kroki.convertDiagram(codeBlockType, rawCodeBlock), parser,
                         uncompletedFutureTextValue: rawCodeBlock // fall back to showing diagram source
            );
  }

  DiagramTransfomer() :
    super(handledCodeBlockTypes:KrokiDiagramEndpoints.supportedEndpoints);
}

final diagramTransformingFencedCodeBlock =
    md.TransformableFencedCodeBlockSyntax([DiagramTransfomer()]);

main() {
    final finalHtml = await md.markdownToHtmlWithAsyncTransforms(markdownSource,
      blockSyntaxes: [diagramTransformingFencedCodeBlock],
      extensionSet: md.ExtensionSet.gitHubWeb);
}   

final String markdownSource = r'''
# Example mermaid diagram

```mermaid
graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;
```
''';

pubspec.yaml Outdated
@@ -13,6 +13,7 @@ environment:

dependencies:
args: ^2.0.0
kroki: ^0.0.2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We almost certainly can't add a dependency to kroki.

Copy link
Contributor Author

@timmaffett timmaffett May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol - yes that is only for the illustrative example - none of the example directory needs to be merged and this dependency would be removed.

I have now added unit test to exercise the new code with both synchronous and asynchronous transforms, so I can remove the changes in the example directory sooner than later. I just intended to leave that there for the sake of illustrating a real world example for discussion, and then remove that before the PR was merged.

(kroki is a package I created to handle interfacing to the kroki.io web service for converting the diagrams. It is being used by the example in the example directory as an example of asynchronous transformations. I did not intend for the any of the changes in the example directory to be merged)

@srawlins
Copy link
Member

Sorry I have a two week vacation, and won't be able to review this until June.

@timmaffett
Copy link
Contributor Author

timmaffett commented May 20, 2022

No problem. Enjoy your vacation! (and thanks for letting me know!)

@timmaffett
Copy link
Contributor Author

I fixed the kroki package so that it can accept sdk 2.12.0 which the CI is using, the test should be able to run now 🤞 (the test do not use the kroki package, only the example, but the pubspec restrictions on sdk for the kroki package stopped the last run).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants