Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Graph Debugger] Expose way to benchmark individual nodes. #11000

Merged
merged 15 commits into from
Apr 28, 2022
Merged

[Graph Debugger] Expose way to benchmark individual nodes. #11000

merged 15 commits into from
Apr 28, 2022

Conversation

AndrewZhaoLuo
Copy link
Contributor

@AndrewZhaoLuo AndrewZhaoLuo commented Apr 13, 2022 •
edited
Loading

This is useful if we only care about a single node or set of nodes in our graph we want to benchmark.

Associated example script of use: https://github.com/AndrewZhaoLuo/TVM-Sandbox/blob/main/runtime/test_benchmark_funcs_graph_runtime.py

@AndrewZhaoLuo
Copy link
Contributor Author

cc @tkonolige

tkonolige
tkonolige requested changes Apr 18, 2022
View reviewed changes
Copy link
Contributor

@tkonolige tkonolige left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice utility to have, thanks @AndrewZhaoLuo! Maybe we could add this to the VM too (although it doesn't have a concept of "node").

python/tvm/contrib/debugger/debug_executor.py Show resolved Hide resolved
python/tvm/contrib/debugger/debug_executor.py Outdated Show resolved Hide resolved
python/tvm/contrib/debugger/debug_executor.py Outdated Show resolved Hide resolved
python/tvm/contrib/debugger/debug_executor.py Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
AndrewZhaoLuo
AndrewZhaoLuo commented Apr 18, 2022
View reviewed changes
Copy link
Contributor Author

@AndrewZhaoLuo AndrewZhaoLuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review! Have some questions about using time_evaluator though.

src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
python/tvm/contrib/debugger/debug_executor.py Outdated Show resolved Hide resolved
python/tvm/contrib/debugger/debug_executor.py Show resolved Hide resolved
python/tvm/contrib/debugger/debug_executor.py Outdated Show resolved Hide resolved
python/tvm/contrib/debugger/debug_executor.py Outdated Show resolved Hide resolved
src/runtime/graph_executor/debug/graph_executor_debug.cc Outdated Show resolved Hide resolved
@AndrewZhaoLuo
Copy link
Contributor Author

PTAL @tkonolige

tkonolige
tkonolige requested changes Apr 25, 2022
View reviewed changes
Copy link
Contributor

@tkonolige tkonolige left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code look good, but could you add a test case?

@AndrewZhaoLuo
Copy link
Contributor Author

The code look good, but could you add a test case?

Done

tkonolige
tkonolige requested changes Apr 25, 2022
View reviewed changes
tests/python/unittest/test_runtime_graph_debug.py Outdated Show resolved Hide resolved
tests/python/unittest/test_runtime_graph_debug.py Outdated Show resolved Hide resolved
tests/python/unittest/test_runtime_graph_debug.py Outdated Show resolved Hide resolved
tests/python/unittest/test_runtime_graph_debug.py Outdated Show resolved Hide resolved
tkonolige
tkonolige approved these changes Apr 26, 2022
View reviewed changes
Copy link
Contributor

@tkonolige tkonolige left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Not sure what is going on with CI.

@masahi masahi merged commit 9fd279b into apache:main Apr 28, 2022
@shingjan shingjan mentioned this pull request May 13, 2022
Merged
shtinsa pushed a commit to Deelvin/tvm that referenced this pull request May 17, 2022
@AndrewZhaoLuo @shtinsa
[Graph Debugger] Expose way to benchmark individual nodes. (apache#11000
12c7df0 
)

* initial

* secondary commit

* docs

* match tests

* fix test

* use std::fixed, max precision, typed pack func, fix isnan

* comments on docs

* address tristan comments

* add test

* tristan comments

* use skipif

* empty commit

* empty commit

* jostle again

* remove assert statement
juda pushed a commit to juda/tvm that referenced this pull request Jun 21, 2022
@AndrewZhaoLuo
[Graph Debugger] Expose way to benchmark individual nodes. (apache#11000
5a9b8e6 
)

* initial

* secondary commit

* docs

* match tests

* fix test

* use std::fixed, max precision, typed pack func, fix isnan

* comments on docs

* address tristan comments

* add test

* tristan comments

* use skipif

* empty commit

* empty commit

* jostle again

* remove assert statement
@driazati driazati mentioned this pull request Jul 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tkonolige tkonolige tkonolige approved these changes

@masahi masahi masahi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@AndrewZhaoLuo @tkonolige @masahi