diff --git a/README.md b/README.md index e2f2b8b..ab5344a 100755 --- a/README.md +++ b/README.md @@ -63,6 +63,7 @@ #### 🚩2022-10-01 udpate: - 修复python部分下一些较小bugs - merge来自[AutumnSun1996](https://github.com/AutumnSun1996)的[OCRWeb实现的多语言部署](https://github.com/RapidAI/RapidOCR/pull/46)demo,详情参见:[ocrweb_mutli-README](./ocrweb_multi/README.md) +- 添加onnxruntime-gpu推理速度较慢于CPU的问题说明,详情参见:[onnxruntime-gpu版相关说明](./python/README.md#onnxruntime-gpu版相关说明) #### 🛴2022-09-01 update: - 由于openvino发布了2022.2.0.dev20220829版本,该版本解决了`cls`部分模型推理的问题。至此,基于openvino的rapidocr完成了统一,全部由openvino推理引擎完成。 diff --git a/docs/README_en.md b/docs/README_en.md index 0896d51..abf1af7 100644 --- a/docs/README_en.md +++ b/docs/README_en.md @@ -65,6 +65,7 @@ #### 🚩2022-10-01 udpate: - Fix some minor bugs under python section. - Merge the [OCRWeb Implementation of Multilingual Deployment](https://github.com/RapidAI/RapidOCR/pull/46) demo from [AutumnSun1996](https://github.com/AutumnSun1996), see for details: [ocrweb_mutli-README](./ocrweb_multi/README.md) +- Add a description of the problem that onnxruntime-gpu inference speed is slower than CPU. For details, please refer to: [onnxruntime-gpu version related instructions](../python/README.md#onnxruntime-gpu版相关说明) #### 🛴2022-09-01 update: - Since openvino released version 2022.2.0.dev20220829, this version solves the problem of `cls` partial model inference. So far, the openvino-based rapidocr has been unified, and it is all completed by the openvino inference engine. diff --git a/docs/change_log.md b/docs/change_log.md index b9b4ef5..234fba6 100644 --- a/docs/change_log.md +++ b/docs/change_log.md @@ -1,6 +1,7 @@ #### 🚩2022-10-01 udpate: - 修复python部分下一些较小bugs - merge来自[AutumnSun1996](https://github.com/AutumnSun1996)的[OCRWeb实现的多语言部署](https://github.com/RapidAI/RapidOCR/pull/46)demo,详情参见:[ocrweb_mutli-README](../ocrweb_multi/README.md) +- 添加onnxruntime-gpu推理速度较慢于CPU的问题说明,详情参见:[onnxruntime-gpu版相关说明](../python/README.md#onnxruntime-gpu版相关说明) #### 🛴2022-09-01 update: - 由于openvino发布了2022.2.0.dev20220829版本,该版本解决了`cls`部分模型推理的问题。至此,基于openvino的rapidocr完成了统一,全部由openvino推理引擎完成。 diff --git a/docs/change_log_en.md b/docs/change_log_en.md index 7a1900b..ab20be1 100644 --- a/docs/change_log_en.md +++ b/docs/change_log_en.md @@ -1,6 +1,7 @@ #### 🚩2022-10-01 udpate: - Fix some minor bugs under python section. - Merge the [OCRWeb Implementation of Multilingual Deployment](https://github.com/RapidAI/RapidOCR/pull/46) demo from [AutumnSun1996](https://github.com/AutumnSun1996), see for details: [ocrweb_mutli-README](../ocrweb_multi/README.md) +- Add a description of the problem that onnxruntime-gpu inference speed is slower than CPU. For details, please refer to: [onnxruntime-gpu version related instructions](../python/README.md#onnxruntime-gpu版相关说明) #### 🛴2022-09-01 update: - Since openvino released version 2022.2.0.dev20220829, this version solves the problem of `cls` partial model inference. So far, the openvino-based rapidocr has been unified, and it is all completed by the openvino inference engine. diff --git a/python/README.md b/python/README.md index ade848f..66444ba 100755 --- a/python/README.md +++ b/python/README.md @@ -12,6 +12,7 @@ - [pip安装快速使用](#pip安装快速使用) - [源码使用步骤](#源码使用步骤) - [`config.yaml`中常用参数介绍](#configyaml中常用参数介绍) + - [onnxruntime-gpu版相关说明](#onnxruntime-gpu版相关说明) - [onnxruntime-gpu版推理配置](#onnxruntime-gpu版推理配置) - [OpenVINO GPU推理配置](#openvino-gpu推理配置) @@ -240,8 +241,11 @@ print(rec_res) |`rec_img_shape`| - |`[3, 48, 320]`| 输入文本识别模型的图像Shape(CHW) | |`rec_batch_num`| - | 6 | 批次推理的batch大小,一般采用默认值即可,太大并没有明显提速,效果还可能会差 | -### onnxruntime-gpu版推理配置 +### onnxruntime-gpu版相关说明 +- 目前已知在onnxruntime-gpu上测试过的小伙伴,反映都是GPU推理速度比在CPU上慢很多。经过探索,初步确定原因为onnxruntime在推理动态图输入时,速度就会慢很多。关于该问题,已经提了相关issue,具体可参见[onnxruntime issue#13198](https://github.com/microsoft/onnxruntime/issues/13198) +- 为了便于比较onnxruntime上推理的基准比较,简单整理了一个[AI Studio: TestOrtInfer](https://aistudio.baidu.com/aistudio/projectdetail/4634684?contributionType=1&sUid=57084&shared=1&ts=1664700017761)项目,小伙伴想要测试的,可以直接Fork来运行查看。 +### onnxruntime-gpu版推理配置 1. **onnxruntime-gpu**需要严格按照与CUDA、cuDNN版本对应来安装,具体参考[文档](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements),**这一步关乎后面是否可以成功调用GPU**。 - 以下是安装示例: - 所用机器环境情况: