Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OpenCL] Set flush period of command queue #5905

Merged
merged 5 commits into from
Apr 18, 2021

Conversation

zhaoyang-star
Copy link
Collaborator

@zhaoyang-star zhaoyang-star commented Apr 15, 2021

【背景】
OpenCL kernel从CPU提交到任务队列后会经历Queued/Submitted/Ready/Running等整个执行状态。对于某个算子从入队等待到真正执行是有延迟的,具体延迟情况依赖厂商平台调度。

刷新命令队列的周期是一个经验值,并没有理论指导,更多地需要结合实际测试数据选取合适的刷新周期。
其他框架设定的刷新周期也不统一,如:

  • tensorflow cl backend
    • mali gpu:开启定时刷新,周期是 24
    • powervr gpu:开启定时刷新,周期是 1
    • adreno gpu:不开启定时刷新
    • tensorflow/tensorflow/lite/delegates/gpu/cl/inference_context.h
  • mindspore
    • 不分区 gpu 类型,定时刷新,周期是 10
    • mindspore/mindspore/lite/src/runtime/gpu/opencl/opencl_runtime.cc/RunKernel
  • TNN
    • 不分区 gpu 类型,定时刷新,周期是 10
    • tnn/source/device/opencl/acc/opencl_layer_acc.cc/NeedFlush()
  • MNN
    • adreno gpu: 刷新周期 10
    • 其他类型 gpu:刷新周期 2

【本PR内容】
实测数据发现,刷新周期对性能的提升很有限,但不会降低性能。因此本次设置:不区分 gpu 类型,定时刷新命令队列,周期是 10.
下图是在 kirin990 和 865 上 armv7 执行 ssd_mobilenetv3 模型的 Paddle-Lite 耗时:
image

@paddle-bot-old
Copy link

Thanks for your contribution!

@zhaoyang-star zhaoyang-star marked this pull request as ready for review April 16, 2021 07:50
@zhaoyang-star zhaoyang-star changed the title Flush queue [OpenCL] Set flush period of command queue Apr 16, 2021
Copy link
Contributor

@ysh329 ysh329 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM,这个和program.cc里的有重复,可以考虑将program.cc里的Flush移除保留这里

@zhaoyang-star zhaoyang-star merged commit 34b96de into PaddlePaddle:develop Apr 18, 2021
@zhaoyang-star zhaoyang-star deleted the flush_queue branch April 18, 2021 09:00
@zhaoyang-star
Copy link
Collaborator Author

LGTM,这个和program.cc里的有重复,可以考虑将program.cc里的Flush移除保留这里

在 program.cc 中没有找到对命令队列人工刷新的代码,还请确认。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants