Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename runtime-config to executor-config and add documentation for Model Library Format #8270

Merged
merged 7 commits into from
Jul 20, 2021

Conversation

areusch
Copy link
Contributor

@areusch areusch commented Jun 17, 2021

This PR:

  1. renames runtime-config to executor-config to match Rename GraphRuntime to GraphExecutor #7653.
  2. adds documentation for Model Library Format to the tree. It is preparation to introduce a Project API RFC (for which it's required reading).

@mehrdadh @leandron @gromero @manupa-arm @jwfromm @guberti @Mousius @giuseros

@areusch areusch changed the title Add documentation for Model Library Format Rename runtime-config to executor-config and add documentation for Model Library Format Jun 17, 2021
Copy link
Contributor

@leandron leandron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks very clear documentation. I think users would benefit also from a short code snippet giving an example of how to generate a MLF package.

Also a few nits and suggestions.

--------------------------

TVM traditionally exports generated libraries as Dynamic Shared Objects
(e.g. DLLs (Windows) or .so (linux)). Inference can be performed on those libraries by loading them
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(e.g. DLLs (Windows) or .so (linux)). Inference can be performed on those libraries by loading them
(e.g. DLLs (Windows) or .so (linux)). Inferences can be performed on those libraries by loading them

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

into an executable using ``libtvm_runtime.so``. This process is very dependent on services provided
by traditional OS.

For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use embedded instead?

Suggested change
For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project
For deployment to embedded platforms (e.g. those lacking traditional OS), the microTVM project

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i feel like it's not strictly limited to embedded though

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.

to that end, I think we should word this as a general output format that produces what strictly TVM (code)generates. The creation of .so/.dll requires external toolchains called after tvm compilation -- such as LLVM and C compilers.
WDYT ?

Comment on lines +18 to +19
Model Library Format
====================
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be beneficial to introduce MLF as an acronym for Model Lbrary Format early in the document, so that we can get people used to it?

Suggested change
Model Library Format
====================
Model Library Format (MLF)
==========================

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i haven't used the acronym yet in this doc though. but i agree it's an easy shorthand for the format. maybe it would make sense more in tvmc docs, where it's a command-line param? wdyt?

@Mousius
Copy link
Member

Mousius commented Jun 17, 2021

Would it be worth adding a JSON Schema for metadata.json at this point to serve as both documentation and a validation mechanism?

Copy link
Member

@mehrdadh mehrdadh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@areusch thanks for this PR!
Mostly good, I have just a few suggestions.

* ``<target>`` - Identifies the TVM target on which the code should run. Currently, only ``host``
is supported.
* ``<unique_name>`` - A unique slug identifying this file. Currently ``lib<n>``, with ``<n>>` an
autoincrementing integer.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe change to auto-incrementing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

``parameters``
^^^^^^^^^^^^^^

Contains machine-parseable parameters. A variety of formats may be provided, but at present, only
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parsable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@manupak manupak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just one more nit -- (bar other's comments)

`Memory Usage Summary`_.
- ``model_name``: The name of this model (e.g. the ``name`` parameter supplied to
``tvm.relay.build``).
- ``runtimes``: A list of runtimes supported by this model. Currently, this list is always
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe executors ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch :)

@areusch
Copy link
Contributor Author

areusch commented Jul 9, 2021

@leandron @manupa-arm please take another look!

Copy link
Contributor

@manupak manupak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did some (final) nit picking! hope you dont mind.
Mostly text changes.

into an executable using ``libtvm_runtime.so``. This process is very dependent on services provided
by traditional OS.

For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.

to that end, I think we should word this as a general output format that produces what strictly TVM (code)generates. The creation of .so/.dll requires external toolchains called after tvm compilation -- such as LLVM and C compilers.
WDYT ?


For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project
can be used to export a generated library in pieces. In this case, microTVM provides another output
format, Model Library Format. Model Library Format is a tarball containing a file for each part of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should not use "microTVM provides another output format", though we could use microTVM as an example this output format becomes useful.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

format, Model Library Format. Model Library Format is a tarball containing a file for each part of
the TVM compiler output.

What can be Exported
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
What can be Exported
What can be exported ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just nit picking :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Directory Layout
----------------

Model Library Format is traditionally contained within a tarball. All paths are relative to the root
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Model Library Format is traditionally contained within a tarball. All paths are relative to the root
Model Library Format is contained within a tarball. All paths are relative to the root

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we dont have any other way. Do we?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point :)

sub-target which describes that relay backend used for that ``device_type``.
- ``version``: A numeric version number that identifies the format used in this Model Library
Format. This number is incremented when the metadata structure or on-disk structure changes.
This document reflects version ``3``.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This document reflects version ``3``.
This document reflects version ``5``.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@areusch
Copy link
Contributor Author

areusch commented Jul 15, 2021

@manupa-arm it won't let me continue your earlier thread, but basically i'd agree with that idea in the medium term however we cannot yet export BYOC Module in MLF. once that is possible, i think we should promote it to a general TVM output format.

@areusch
Copy link
Contributor Author

areusch commented Jul 15, 2021

@manupa-arm please take another look

Copy link
Contributor

@manupak manupak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@leandron leandron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@leandron leandron merged commit ca28dff into apache:main Jul 20, 2021
@leandron
Copy link
Contributor

Thanks @areusch @mehrdadh @Mousius @manupa-arm, this is merged now!

ylc pushed a commit to ylc/tvm that referenced this pull request Sep 29, 2021
…del Library Format (apache#8270)

* Rename runtime-config to executor-config.

* Add documentation.

* address comments, make tests pass

* fix unit test

* fix sphinx doc errors

* address manupa comments
ylc pushed a commit to ylc/tvm that referenced this pull request Jan 13, 2022
…del Library Format (apache#8270)

* Rename runtime-config to executor-config.

* Add documentation.

* address comments, make tests pass

* fix unit test

* fix sphinx doc errors

* address manupa comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants