zhutian
Gitee
@@ -32,3 +32,7 @@ Ascend tools，昇腾工具仓库。
|
||||
6. [pt2pb](https://gitee.com/ascend/tools/tree/master/pt2pb)
|
||||
|
||||
**pytorch模型转tensorflow pb模型工具**：输入pytorch权重参数模型，转为onnx，再转为pb模型
|
||||
|
||||
7. [dnmetis](https://gitee.com/ascend/tools/tree/master/dnmetis)
|
||||
|
||||
**NPU推理精度和性能测试工具**：使用Python封装ACL的C++接口，输入om模型和原始数据集图片、标签，即可执行模型推理，输出精度数据和性能数据
|
||||
|
||||
@@ -32,5 +32,9 @@ Ascend tools
|
||||
|
||||
**pytorch model transform to tensorflow pb model tool**：input pytorch weights parameters model，transform to onnx file，then transform to pb model
|
||||
|
||||
7. [dnmetis](https://gitee.com/ascend/tools/tree/master/dnmetis)
|
||||
|
||||
**Test tool for NPU inference precision and performance**：Using Python to encapsulate the C++ interface of ACL, inputting om model and original dataset images and tags, we can execute model inference and give out precision and performance of the om model
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
中文|[英文](README_EN.md)
|
||||
|
||||
## 1.安装依赖：
|
||||
```
|
||||
pip3.7.5 install python-opencv
|
||||
cd backend_C++/dnmetis_backend
|
||||
pip3.7.5 setup.py install
|
||||
```
|
||||
安装dnmetis_backend的细节可以在backend_C++/dnmetis_backend/README.md看到，对于一个全新的Ai1推理环境，只需要安装一次依赖，不需要重复安装。
|
||||
|
||||
## 2.下载om模型(.om)
|
||||
|
||||
如下示例展示如果在NPU上运行efficientnet-b8模型:\
|
||||
1.下载efficientnet-b8 model(.om): \
|
||||
链接：[百度网盘](https://pan.baidu.com/s/1N-kpQoDe3NRxvjFKjAT9AA) \
|
||||
提取码：tvg0
|
||||
下载的om模型放到model文件夹.
|
||||
|
||||
原生的TensorFlow模型efficientnet-b8(.pb):\
|
||||
链接：[百度网盘](https://pan.baidu.com/s/1CajdSlNTh6k35RoyOn-3Ug)\
|
||||
提取码：slqm
|
||||
|
||||
如果想了解如果从pb模型如何转换成om模型，请下载efficientnet-b8.pb模型，使用ATC模型转换工具，或者执行转换命令：\
|
||||
.atc --model=MODELDIR/efficientnet−b8.pb −−framework=3 −−inputshape=′images:1,672,672,3′−−output=MODELDIR/efficientnet-b8 --mode=0 --out_nodes='Softmax:0' --soc_version=Ascend310 --input_fp16_nodes=images --output_type=FP16
|
||||
|
||||
2.Imagenet-val数据集和标签:
|
||||
|
||||
这里的示例仅仅展示了从Imagenet-val数据集挑选的10张图片：/
|
||||
|
||||
|
||||
|
||||
|
||||
## 3.执行推理:
|
||||
建议提交的PR代码统一使用run_inference.sh作为入口：\
|
||||
bash run_inference.sh
|
||||
|
||||
执行日志:
|
||||
```
|
||||
[INFO] start backend_predict is -1518493925
|
||||
[INFO] start Execute is -1518490258
|
||||
[INFO] model execute success
|
||||
[INFO] end Execute is -1518350716
|
||||
[INFO] npu compute cost 139.476000 ms
|
||||
[INFO] 1.output data success
|
||||
[INFO] 2.output data success
|
||||
[INFO] execute sample success
|
||||
[INFO] Pure device execute time is 0.000000 ms
|
||||
[INFO] end backend_predict is -1518346882
|
||||
img_orig: ILSVRC2012_val_00000010.JPEG label: 332 predictions: 332
|
||||
|
||||
Predict total jpeg: 10 Accuracy: 0.8
|
||||
```
|
||||
如上所示, "139.476 ms"是NPU的推理时间，"0.8" 是10张图片的top1精度。
|
||||
|
||||
## 4.完整的5w张Imagenet2012-val数据集精度:
|
||||
|
||||
|
||||
|
||||
|
||||
## 5.main.py修改点:
|
||||
|
||||
如果需要使用你自己的模型来推理和计算精度，请修改main.py\
|
||||
只需要关心数据集、预处理和后处理代码部分：
|
||||
|
||||
### 预处理:
|
||||
```
|
||||
def resize_with_aspectratio(img, out_height, out_width, scale=87.5, inter_pol=cv2.INTER_LINEAR):
|
||||
height, width = img.shape[:2]
|
||||
new_height = int(100. * out_height / scale)
|
||||
new_width = int(100. * out_width / scale)
|
||||
if height > width:
|
||||
w = new_width
|
||||
h = int(new_height * height / width)
|
||||
else:
|
||||
h = new_height
|
||||
w = int(new_width * width / height)
|
||||
img = cv2.resize(img, (w, h), interpolation=inter_pol)
|
||||
return img
|
||||
|
||||
def center_crop(img, out_height, out_width):
|
||||
height, width = img.shape[:2]
|
||||
left = int((width - out_width) / 2)
|
||||
right = int((width + out_width) / 2)
|
||||
top = int((height - out_height) / 2)
|
||||
bottom = int((height + out_height) / 2)
|
||||
img = img[top:bottom, left:right]
|
||||
return img
|
||||
def pre_process_noisy(img, dims=None, precision="fp32"):
|
||||
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
|
||||
output_height, output_width, _ = dims
|
||||
cv2_interpol = cv2.INTER_CUBIC
|
||||
img = resize_with_aspectratio(img, output_height, output_width, inter_pol=cv2_interpol)
|
||||
img = center_crop(img, output_height, output_width)
|
||||
MEAN_RGB = [0.485 * 255, 0.456 * 255, 0.406 * 255]
|
||||
STDDEV_RGB = [0.229 * 255, 0.224 * 255, 0.225 * 255]
|
||||
|
||||
if precision=="fp32":
|
||||
img = np.asarray(img, dtype='float32')
|
||||
if precision=="fp16":
|
||||
img = np.asarray(img, dtype='float16')
|
||||
|
||||
means = np.array([0.485 * 255, 0.456 * 255, 0.406 * 255], dtype=np.float32)
|
||||
img -= means
|
||||
stddev = np.array([0.229 * 255, 0.224 * 255, 0.225 * 255], dtype=np.float32)
|
||||
img /= stddev
|
||||
return img
|
||||
```
|
||||
|
||||
### 推理和后处理:
|
||||
```
|
||||
predictions = backend.predict(args.feed[i])
|
||||
#print(args.feed[i].shape)
|
||||
print('img_orig:',args.image_list[i],'label:',args.label_list[i],'predictions:',np.argmax(predictions),'\n')
|
||||
```
|
||||
@@ -0,0 +1,114 @@
|
||||
EN|[CH](README.md)
|
||||
|
||||
## 1.Install requirements：
|
||||
```
|
||||
pip3.7.5 install python-opencv
|
||||
cd backend_C++/dnmetis_backend
|
||||
pip3.7.5 setup.py install
|
||||
```
|
||||
Details of dnmetis_backend installation can be found in backend_C++/dnmetis_backend/README.md. Notice that, you just need to install requirements once for a brand new Ai1-Inference environment。
|
||||
|
||||
## 2.Download model(.om)
|
||||
|
||||
Here is an example of how to run npu inference of efficientnet-b8:\
|
||||
1.download efficientnet-b8 model(.om): \
|
||||
URL：[baidu pan](https://pan.baidu.com/s/1N-kpQoDe3NRxvjFKjAT9AA) \
|
||||
Extracted code：tvg0
|
||||
|
||||
Original tensorflow model of efficientnet-b8(.pb):\
|
||||
URL：[baidu pan](https://pan.baidu.com/s/1CajdSlNTh6k35RoyOn-3Ug)\
|
||||
Extracted code：slqm
|
||||
|
||||
If you want to acknowledge how to generate om from pb，pls download efficientnet-b8.pb and execute ATC cmd：\
|
||||
.atc --model=MODELDIR/efficientnet−b8.pb −−framework=3 −−inputshape=′images:1,672,672,3′−−output=MODELDIR/efficientnet-b8 --mode=0 --out_nodes='Softmax:0' --soc_version=Ascend310 --input_fp16_nodes=images --output_type=FP16
|
||||
|
||||
2.Imagenet-val dataset and labels in val_map.txt:
|
||||
|
||||
Here is an example of 10 pictures of Imagenet-val dataset：/
|
||||
|
||||
|
||||
|
||||
|
||||
## 3.Start execute the inference:
|
||||
|
||||
bash run_inference.sh
|
||||
|
||||
Log:
|
||||
```
|
||||
[INFO] start backend_predict is -1518493925
|
||||
[INFO] start Execute is -1518490258
|
||||
[INFO] model execute success
|
||||
[INFO] end Execute is -1518350716
|
||||
[INFO] npu compute cost 139.476000 ms
|
||||
[INFO] 1.output data success
|
||||
[INFO] 2.output data success
|
||||
[INFO] execute sample success
|
||||
[INFO] Pure device execute time is 0.000000 ms
|
||||
[INFO] end backend_predict is -1518346882
|
||||
img_orig: ILSVRC2012_val_00000010.JPEG label: 332 predictions: 332
|
||||
|
||||
Predict total jpeg: 10 Accuracy: 0.8
|
||||
```
|
||||
As you seen, "139.47 ms" is the npu inference time，"0.8" is the top1 Accuracy of 10 pictures。
|
||||
|
||||
## 4.Top1 Accuracy of entire Imagenet2012-val Datasets(5w pictures):
|
||||
|
||||
|
||||
|
||||
|
||||
## 5.modify main.py for your own model:
|
||||
|
||||
Only need to concern about the dataset，pre-process，post-process：
|
||||
|
||||
### pre-process:
|
||||
```
|
||||
def resize_with_aspectratio(img, out_height, out_width, scale=87.5, inter_pol=cv2.INTER_LINEAR):
|
||||
height, width = img.shape[:2]
|
||||
new_height = int(100. * out_height / scale)
|
||||
new_width = int(100. * out_width / scale)
|
||||
if height > width:
|
||||
w = new_width
|
||||
h = int(new_height * height / width)
|
||||
else:
|
||||
h = new_height
|
||||
w = int(new_width * width / height)
|
||||
img = cv2.resize(img, (w, h), interpolation=inter_pol)
|
||||
return img
|
||||
|
||||
def center_crop(img, out_height, out_width):
|
||||
height, width = img.shape[:2]
|
||||
left = int((width - out_width) / 2)
|
||||
right = int((width + out_width) / 2)
|
||||
top = int((height - out_height) / 2)
|
||||
bottom = int((height + out_height) / 2)
|
||||
img = img[top:bottom, left:right]
|
||||
return img
|
||||
def pre_process_noisy(img, dims=None, precision="fp32"):
|
||||
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
|
||||
output_height, output_width, _ = dims
|
||||
cv2_interpol = cv2.INTER_CUBIC
|
||||
img = resize_with_aspectratio(img, output_height, output_width, inter_pol=cv2_interpol)
|
||||
img = center_crop(img, output_height, output_width)
|
||||
MEAN_RGB = [0.485 * 255, 0.456 * 255, 0.406 * 255]
|
||||
STDDEV_RGB = [0.229 * 255, 0.224 * 255, 0.225 * 255]
|
||||
|
||||
if precision=="fp32":
|
||||
img = np.asarray(img, dtype='float32')
|
||||
if precision=="fp16":
|
||||
img = np.asarray(img, dtype='float16')
|
||||
|
||||
means = np.array([0.485 * 255, 0.456 * 255, 0.406 * 255], dtype=np.float32)
|
||||
img -= means
|
||||
stddev = np.array([0.229 * 255, 0.224 * 255, 0.225 * 255], dtype=np.float32)
|
||||
img /= stddev
|
||||
return img
|
||||
```
|
||||
|
||||
### inference and post-process:
|
||||
```
|
||||
predictions = backend.predict(args.feed[i])
|
||||
#print(args.feed[i].shape)
|
||||
print('img_orig:',args.image_list[i],'label:',args.label_list[i],'predictions:',np.argmax(predictions),'\n')
|
||||
```
|
||||
|
||||
|
||||
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,26 @@
|
||||
"""
|
||||
abstract backend class
|
||||
"""
|
||||
|
||||
|
||||
# pylint: disable=unused-argument,missing-docstring
|
||||
|
||||
class Backend():
|
||||
def __init__(self):
|
||||
self.inputs = []
|
||||
self.outputs = []
|
||||
|
||||
def version(self):
|
||||
raise NotImplementedError("Backend:version")
|
||||
|
||||
def name(self):
|
||||
raise NotImplementedError("Backend:name")
|
||||
|
||||
def load(self, args):
|
||||
raise NotImplementedError("Backend:load")
|
||||
|
||||
def predict(self, feed):
|
||||
raise NotImplementedError("Backend:predict")
|
||||
|
||||
def get_predict_time(self):
|
||||
return 0
|
||||
@@ -0,0 +1,64 @@
|
||||
"""
|
||||
acl backend
|
||||
"""
|
||||
|
||||
|
||||
import dnmetis_backend as dnmetis_backend
|
||||
import backend.backend as backend
|
||||
import numpy as np
|
||||
import os
|
||||
import pdb
|
||||
|
||||
class AclBackend(backend.Backend):
|
||||
def __init__(self):
|
||||
super(AclBackend, self).__init__()
|
||||
self.ACL=5
|
||||
self.outputs = ""
|
||||
self.inputs = ""
|
||||
self.model_path = ""
|
||||
self.cfg_path = ""
|
||||
|
||||
def version(self):
|
||||
return "1.0"
|
||||
|
||||
def name(self):
|
||||
return "AclBackend"
|
||||
|
||||
def image_format(self):
|
||||
# By default tensorflow uses NHWC (and the cpu implementation only does NHWC)
|
||||
return "NHWC"
|
||||
|
||||
def load(self, args):
|
||||
# there is no input/output meta data i the graph so it need to come from config.
|
||||
if not args.inputs:
|
||||
raise ValueError("AclBackend needs inputs")
|
||||
if not args.outputs:
|
||||
raise ValueError("AclBackend needs outputs")
|
||||
self.outputs = args.outputs
|
||||
self.inputs = args.inputs
|
||||
self.model_path = args.model
|
||||
self.cfg_path = args.cfg_path
|
||||
#s.path.join(args.pwd, 'backend_cfg/built-in_config.txt')
|
||||
dnmetis_backend.backend_setconfig(self.cfg_path)
|
||||
dnmetis_backend.backend_load(self.ACL,self.model_path,"")
|
||||
return self
|
||||
|
||||
def predict(self, feed):
|
||||
#fed=feed[self.inputs[0]]
|
||||
result_list=[]
|
||||
result = dnmetis_backend.backend_predict(self.ACL,self.model_path,feed)
|
||||
|
||||
for _ in range(len(self.outputs)):
|
||||
#resnet50 tf & caffe
|
||||
if 'softmax_tensor' in self.outputs[_] or 'prob' in self.outputs[_]:
|
||||
result_list.append(np.argmax(result[_],1))
|
||||
# resnet50 tf
|
||||
if 'ArgMax' in self.outputs[_]:
|
||||
result_list.append(result[_])
|
||||
if result_list == []:
|
||||
# ssd-resnet34 tf
|
||||
result_list = result
|
||||
return result_list
|
||||
|
||||
def unload(self):
|
||||
return dnmetis_backend.backend_unload(self.ACL,self.model_path,"")
|
||||
@@ -0,0 +1,36 @@
|
||||
Copyright (c) 2016 The Pybind Development Team, All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are met:
|
||||
|
||||
1. Redistributions of source code must retain the above copyright notice, this
|
||||
list of conditions and the following disclaimer.
|
||||
|
||||
2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
|
||||
3. Neither the name of the copyright holder nor the names of its contributors
|
||||
may be used to endorse or promote products derived from this software
|
||||
without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
||||
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
||||
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
||||
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
||||
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
||||
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
||||
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
||||
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
||||
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
You are under no obligation whatsoever to provide any bug fixes, patches, or
|
||||
upgrades to the features, functionality or performance of the source code
|
||||
("Enhancements") to anyone; however, if you choose to make your Enhancements
|
||||
available either publicly, or directly to the author of this software, without
|
||||
imposing a separate written license agreement for such Enhancements, then you
|
||||
hereby grant the following license: a non-exclusive, royalty-free perpetual
|
||||
license to install, use, modify, prepare derivative works, incorporate into
|
||||
other computer software, distribute, and sublicense such enhancements or
|
||||
derivative works thereof, in binary and source code form.
|
||||
@@ -0,0 +1,87 @@
|
||||
## dnmetis_backend
|
||||
|
||||
It contains one AclBackend(C++), It can be called by DNMetis when tester tests NPU perf&accuracy
|
||||
|
||||
The third-party C++ Backends can be easily added
|
||||
|
||||
1.install dnmetis_backend
|
||||
|
||||
python3.7.5 setup.py install
|
||||
|
||||
2.Install log:
|
||||
|
||||
/usr/local/python3.7.5/lib/python3.7/site-packages/setuptools/dist.py:474: UserWarning: Normalizing 'V1.0.2' to '1.0.2'
|
||||
normalized_version,
|
||||
running install
|
||||
running bdist_egg
|
||||
running egg_info
|
||||
creating dnmetis_backend.egg-info
|
||||
writing dnmetis_backend.egg-info/PKG-INFO
|
||||
writing dependency_links to dnmetis_backend.egg-info/dependency_links.txt
|
||||
writing top-level names to dnmetis_backend.egg-info/top_level.txt
|
||||
writing manifest file 'dnmetis_backend.egg-info/SOURCES.txt'
|
||||
reading manifest file 'dnmetis_backend.egg-info/SOURCES.txt'
|
||||
writing manifest file 'dnmetis_backend.egg-info/SOURCES.txt'
|
||||
installing library code to build/bdist.linux-x86_64/egg
|
||||
running install_lib
|
||||
running build_ext
|
||||
creating tmp
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/python3.7.5/include/python3.7m -c /tmp/tmp4o2xo183.cpp -o tmp/tmp4o2xo183.o -std=c++14
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/python3.7.5/include/python3.7m -c /tmp/tmptazg09at.cpp -o tmp/tmptazg09at.o -fvisibility=hidden
|
||||
building 'dnmetis_backend' extension
|
||||
creating build
|
||||
creating build/temp.linux-x86_64-3.7
|
||||
creating build/temp.linux-x86_64-3.7/src
|
||||
creating build/temp.linux-x86_64-3.7/backend
|
||||
creating build/temp.linux-x86_64-3.7/backend/built-in
|
||||
creating build/temp.linux-x86_64-3.7/backend/built-in/src
|
||||
creating build/temp.linux-x86_64-3.7/backend/custom
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c src/main.cpp -o build/temp.linux-x86_64-3.7/src/main.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c src/Config.cpp -o build/temp.linux-x86_64-3.7/src/Config.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/BaseBackend.cpp -o build/temp.linux-x86_64-3.7/backend/BaseBackend.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/BackendFactory.cpp -o build/temp.linux-x86_64-3.7/backend/BackendFactory.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/built-in/aclbackend.cpp -o build/temp.linux-x86_64-3.7/backend/built-in/aclbackend.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/built-in/src/model_process.cpp -o build/temp.linux-x86_64-3.7/backend/built-in/src/model_process.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/built-in/src/sample_process.cpp -o build/temp.linux-x86_64-3.7/backend/built-in/src/sample_process.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/built-in/src/utils.cpp -o build/temp.linux-x86_64-3.7/backend/built-in/src/utils.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/custom/trtbackend.cpp -o build/temp.linux-x86_64-3.7/backend/custom/trtbackend.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
|
||||
creating build/lib.linux-x86_64-3.7
|
||||
g++ -pthread -shared build/temp.linux-x86_64-3.7/src/main.o build/temp.linux-x86_64-3.7/src/Config.o build/temp.linux-x86_64-3.7/backend/BaseBackend.o build/temp.linux-x86_64-3.7/backend/BackendFactory.o build/temp.linux-x86_64-3.7/backend/built-in/aclbackend.o build/temp.linux-x86_64-3.7/backend/built-in/src/model_process.o build/temp.linux-x86_64-3.7/backend/built-in/src/sample_process.o build/temp.linux-x86_64-3.7/backend/built-in/src/utils.o build/temp.linux-x86_64-3.7/backend/custom/trtbackend.o -L/usr/local/Ascend/acllib/lib64/ -L/usr/local/python3.7.5/lib -lascendcl -lpython3.7m -o build/lib.linux-x86_64-3.7/dnmetis_backend.cpython-37m-x86_64-linux-gnu.so -O0
|
||||
creating build/bdist.linux-x86_64
|
||||
creating build/bdist.linux-x86_64/egg
|
||||
copying build/lib.linux-x86_64-3.7/dnmetis_backend.cpython-37m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
|
||||
creating stub loader for dnmetis_backend.cpython-37m-x86_64-linux-gnu.so
|
||||
byte-compiling build/bdist.linux-x86_64/egg/dnmetis_backend.py to dnmetis_backend.cpython-37.pyc
|
||||
creating build/bdist.linux-x86_64/egg/EGG-INFO
|
||||
copying dnmetis_backend.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
|
||||
copying dnmetis_backend.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
|
||||
copying dnmetis_backend.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
|
||||
copying dnmetis_backend.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
|
||||
copying dnmetis_backend.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
|
||||
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
|
||||
creating dist
|
||||
creating 'dist/dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
|
||||
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
|
||||
Processing dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg
|
||||
removing '/usr/local/python3.7.5/lib/python3.7/site-packages/dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg' (and everything under it)
|
||||
creating /usr/local/python3.7.5/lib/python3.7/site-packages/dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg
|
||||
Extracting dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg to /usr/local/python3.7.5/lib/python3.7/site-packages
|
||||
dnmetis-backend 1.0.2 is already the active version in easy-install.pth
|
||||
|
||||
Installed /usr/local/python3.7.5/lib/python3.7/site-packages/dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg
|
||||
Processing dependencies for dnmetis-backend==1.0.2
|
||||
Finished processing dependencies for dnmetis-backend==1.0.2
|
||||
|
||||
|
||||
3、Check result of installation:\
|
||||
"dnmetis-backend" will be seen
|
||||
````pip3.7.5 list
|
||||
Packag Version
|
||||
--------------- ---------
|
||||
attrs 20.2.0
|
||||
certifi 2020.6.20
|
||||
cffi 1.14.2
|
||||
chardet 3.0.4
|
||||
decorator 4.4.2
|
||||
dnmetis-backend 1.0.2
|
||||
grpcio 1.31.0
|
||||
@@ -0,0 +1,61 @@
|
||||
/**
|
||||
* Copyright 2020 Huawei Technologies Co., Ltd
|
||||
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
#include "BackendFactory.h"
|
||||
|
||||
BackendFactory *BackendFactory::Instance() {
|
||||
static BackendFactory instance;
|
||||
return &instance;
|
||||
}
|
||||
|
||||
int BackendFactory::Init() {
|
||||
if (!is_init_) {
|
||||
std::string skt_bin = "libaclbackend.so";
|
||||
//handle_ = dlopen(skt_bin.c_str(), RTLD_NOW | RTLD_GLOBAL);
|
||||
//if (handle_ == nullptr) {
|
||||
// GELOGE(FAILED, "SKT: open skt lib failed, please check LD_LIBRARY_PATH.");
|
||||
//}
|
||||
}
|
||||
is_init_ = true;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
std::shared_ptr<BaseBackend> BackendFactory::CreateBaseBackend(const FrameworkType type) {
|
||||
std::map<FrameworkType, BASE_BACKEND_CREATOR_FUN>::iterator iter = creator_map_.find(type);
|
||||
if (iter != creator_map_.end()) {
|
||||
return iter->second();
|
||||
}
|
||||
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
void BackendFactory::RegisterCreator(const FrameworkType type, BASE_BACKEND_CREATOR_FUN fun) {
|
||||
std::map<FrameworkType, BASE_BACKEND_CREATOR_FUN>::iterator iter = creator_map_.find(type);
|
||||
if (iter != creator_map_.end()) {
|
||||
return;
|
||||
}
|
||||
|
||||
creator_map_[type] = fun;
|
||||
}
|
||||
|
||||
BackendFactory::~BackendFactory() {
|
||||
creator_map_.clear();
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
/**
|
||||
* Copyright 2020 Huawei Technologies Co., Ltd
|
||||
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
#include "BaseBackend.h"
|
||||
|
||||
|
||||
|
||||
int BaseBackend::param()
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
int BaseBackend::runner()
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
int BaseBackend::runnerthread()
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
int BaseBackend::statistic()
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
#include "aclbackend.h"
|
||||
|
||||
|
||||
CREATE_BACKEND_INIT(AclBackend,model,data)
|
||||
{
|
||||
Result ret = processSample.InitResource(model);
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("sample init resource failed");
|
||||
return FAILED;
|
||||
}
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
CREATE_BACKEND_LOAD(AclBackend,model,data)
|
||||
{
|
||||
INFO_LOG("AclBackend LOAD model success");
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
CREATE_BACKEND_PREDICT(AclBackend,model,data,len,output,npuTime)
|
||||
{
|
||||
Result ret = processSample.Process(data,len,output,npuTime);
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("sample process failed");
|
||||
return FAILED;
|
||||
}
|
||||
INFO_LOG("execute sample success");
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
CREATE_BACKEND_UNLOAD(AclBackend,model,data)
|
||||
{
|
||||
Result ret = processSample.Unload();
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("sample unload failed");
|
||||
return FAILED;
|
||||
}
|
||||
INFO_LOG("AclBackend UNLOAD model success");
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
|
||||
|
||||
REGISTER_BASE_BACKEND_CREATOR(ACL, AclBackend)
|
||||
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
#include "BackendFactory.h"
|
||||
#include "sample_process.h"
|
||||
#include <vector>
|
||||
using namespace std;
|
||||
|
||||
CREATE_BACKEND(AclBackend)
|
||||
|
||||
SampleProcess processSample;
|
||||
@@ -0,0 +1,113 @@
|
||||
/**
|
||||
* @file model_process.h
|
||||
*
|
||||
* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
*/
|
||||
#pragma once
|
||||
#include <iostream>
|
||||
#include "utils.h"
|
||||
#include "acl/acl.h"
|
||||
#include<vector>
|
||||
using namespace std;
|
||||
|
||||
/**
|
||||
* ModelProcess
|
||||
*/
|
||||
class ModelProcess {
|
||||
public:
|
||||
/**
|
||||
* @brief Constructor
|
||||
*/
|
||||
ModelProcess();
|
||||
|
||||
/**
|
||||
* @brief Destructor
|
||||
*/
|
||||
~ModelProcess();
|
||||
|
||||
/**
|
||||
* @brief load model from file with mem
|
||||
* @param [in] modelPath: model path
|
||||
* @return result
|
||||
*/
|
||||
Result LoadModelFromFileWithMem(const char *modelPath);
|
||||
|
||||
/**
|
||||
* @brief unload model
|
||||
*/
|
||||
void Unload();
|
||||
|
||||
/**
|
||||
* @brief create model desc
|
||||
* @return result
|
||||
*/
|
||||
Result CreateDesc();
|
||||
|
||||
/**
|
||||
* @brief destroy desc
|
||||
*/
|
||||
void DestroyDesc();
|
||||
|
||||
aclmdlDesc *GetDesc(){return modelDesc_;};
|
||||
|
||||
/**
|
||||
* @brief create model input
|
||||
* @param [in] inputDataBuffer: input buffer
|
||||
* @param [in] bufferSize: input buffer size
|
||||
* @return result
|
||||
*/
|
||||
Result CreateInput(void *inputDataBuffer, size_t bufferSize);
|
||||
|
||||
/**
|
||||
* @brief destroy input resource
|
||||
*/
|
||||
void DestroyInput();
|
||||
|
||||
/**
|
||||
* @brief create output buffer
|
||||
* @return result
|
||||
*/
|
||||
Result CreateOutput();
|
||||
|
||||
/**
|
||||
* @brief destroy output resource
|
||||
*/
|
||||
void DestroyOutput();
|
||||
|
||||
//void * GetOutput(){return outputdata;};
|
||||
|
||||
/**
|
||||
* @brief model execute
|
||||
* @return result
|
||||
*/
|
||||
Result Execute();
|
||||
|
||||
/**
|
||||
* @brief dump model output result to file
|
||||
*/
|
||||
void DumpModelOutputResult();
|
||||
|
||||
/**
|
||||
* @brief get model output result
|
||||
*/
|
||||
void OutputModelResult();
|
||||
|
||||
void OutputModelResult(vector<Output_buf> &output);
|
||||
|
||||
public:
|
||||
uint32_t modelId_;
|
||||
size_t modelMemSize_;
|
||||
size_t modelWeightSize_;
|
||||
void *modelMemPtr_;
|
||||
void *modelWeightPtr_;
|
||||
bool loadFlag_; // model load flag
|
||||
aclmdlDesc *modelDesc_;
|
||||
aclmdlDataset *input_;
|
||||
aclmdlDataset *output_;
|
||||
vector<void *> outputdata;
|
||||
};
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
/**
|
||||
* @file sample_process.h
|
||||
*
|
||||
* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
*/
|
||||
#pragma once
|
||||
#include "utils.h"
|
||||
#include "acl/acl.h"
|
||||
#include "model_process.h"
|
||||
#include<vector>
|
||||
using namespace std;
|
||||
|
||||
/**
|
||||
* SampleProcess
|
||||
*/
|
||||
class SampleProcess {
|
||||
public:
|
||||
/**
|
||||
* @brief Constructor
|
||||
*/
|
||||
SampleProcess();
|
||||
|
||||
/**
|
||||
* @brief Destructor
|
||||
*/
|
||||
~SampleProcess();
|
||||
|
||||
/**
|
||||
* @brief init reousce
|
||||
* @return result
|
||||
*/
|
||||
Result InitResource(char* omModelPath);
|
||||
|
||||
/**
|
||||
* @brief sample process
|
||||
* @return result
|
||||
*/
|
||||
//Result Process(char* binfile);
|
||||
|
||||
Result Process(void* binfile,int len);
|
||||
|
||||
Result Process(void* binfile,int len, vector<Output_buf> &output, long &npuTime);
|
||||
|
||||
|
||||
Result Unload();
|
||||
|
||||
//ModelProcess GetModelProcess();
|
||||
ModelProcess GetModelProcess(){return processModel;};
|
||||
|
||||
public:
|
||||
void DestroyResource();
|
||||
|
||||
int32_t deviceId_;
|
||||
aclrtContext context_;
|
||||
aclrtStream stream_;
|
||||
ModelProcess processModel;
|
||||
};
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
/**
|
||||
* @file utils.h
|
||||
*
|
||||
* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
*/
|
||||
#pragma once
|
||||
#include <iostream>
|
||||
#include "common.h"
|
||||
#include <map>
|
||||
#include "acl/acl_base.h"
|
||||
//#include "Config.h"
|
||||
/**
|
||||
* Utils
|
||||
*/
|
||||
|
||||
struct aclmdlTensorDesc {
|
||||
aclmdlTensorDesc() : name(""), size(0), format(ACL_FORMAT_UNDEFINED), dataType(ACL_DT_UNDEFINED) {}
|
||||
std::string name;
|
||||
size_t size;
|
||||
aclFormat format;
|
||||
aclDataType dataType;
|
||||
std::vector<int64_t> dims;
|
||||
};
|
||||
|
||||
struct aclmdlDesc {
|
||||
void Clear()
|
||||
{
|
||||
inputDesc.clear();
|
||||
outputDesc.clear();
|
||||
dynamicBatch.clear();
|
||||
dynamicHW.clear();
|
||||
}
|
||||
std::vector<aclmdlTensorDesc> inputDesc;
|
||||
std::vector<aclmdlTensorDesc> outputDesc;
|
||||
std::vector<uint64_t> dynamicBatch;
|
||||
std::vector<std::vector<uint64_t>> dynamicHW;
|
||||
};
|
||||
|
||||
|
||||
|
||||
class Utils {
|
||||
public:
|
||||
/**
|
||||
* @brief create device buffer of file
|
||||
* @param [in] fileName: file name
|
||||
* @param [out] fileSize: size of file
|
||||
* @return device buffer of file
|
||||
*/
|
||||
static void *GetDeviceBufferOfFile(std::string fileName, uint32_t &fileSize);
|
||||
|
||||
/**
|
||||
* @brief create buffer of file
|
||||
* @param [in] fileName: file name
|
||||
* @param [out] fileSize: size of file
|
||||
* @return buffer of pic
|
||||
*/
|
||||
static void* ReadBinFile(std::string fileName, uint32_t& fileSize);
|
||||
|
||||
static void* GetDeviceBufferOfptr(void* fileName, uint32_t fileSize);
|
||||
|
||||
static long getCurrentTime();
|
||||
|
||||
};
|
||||
|
||||
#pragma once
|
||||
@@ -0,0 +1,444 @@
|
||||
/**
|
||||
* @file model_process.cpp
|
||||
*
|
||||
* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
*/
|
||||
#include "model_process.h"
|
||||
#include <iostream>
|
||||
#include <map>
|
||||
#include <sstream>
|
||||
#include <algorithm>
|
||||
#include "utils.h"
|
||||
using namespace std;
|
||||
extern bool g_isDevice;
|
||||
//extern Config configSettings;
|
||||
extern std::map<aclDataType, std::string> ACLdt;
|
||||
extern std::map<aclDataType, int> ACLdt_size;
|
||||
|
||||
ModelProcess::ModelProcess() :modelId_(0), modelMemSize_(0), modelWeightSize_(0), modelMemPtr_(nullptr),
|
||||
modelWeightPtr_(nullptr), loadFlag_(false), modelDesc_(nullptr), input_(nullptr), output_(nullptr)
|
||||
{
|
||||
}
|
||||
|
||||
ModelProcess::~ModelProcess()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
Result ModelProcess::LoadModelFromFileWithMem(const char *modelPath)
|
||||
{
|
||||
if (loadFlag_) {
|
||||
ERROR_LOG("has already loaded a model");
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
aclError ret = aclmdlQuerySize(modelPath, &modelMemSize_, &modelWeightSize_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("query model failed, model file is %s", modelPath);
|
||||
return FAILED;
|
||||
}
|
||||
//int mem_malloc_type=configSettings.Read("aclrtMemMallocPolicy", 0);
|
||||
ret = aclrtMalloc(&modelMemPtr_, modelMemSize_, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("malloc buffer for mem failed, require size is %zu", modelMemSize_);
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
ret = aclrtMalloc(&modelWeightPtr_, modelWeightSize_, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("malloc buffer for weight failed, require size is %zu", modelWeightSize_);
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
ret = aclmdlLoadFromFileWithMem(modelPath, &modelId_, modelMemPtr_,
|
||||
modelMemSize_, modelWeightPtr_, modelWeightSize_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("load model from file failed, model file is %s", modelPath);
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
loadFlag_ = true;
|
||||
INFO_LOG("load model %s success", modelPath);
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
Result ModelProcess::CreateDesc()
|
||||
{
|
||||
modelDesc_ = aclmdlCreateDesc();
|
||||
if (modelDesc_ == nullptr) {
|
||||
ERROR_LOG("create model description failed");
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
aclError ret = aclmdlGetDesc(modelDesc_, modelId_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("get model description failed");
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
INFO_LOG("create model description success");
|
||||
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
void ModelProcess::DestroyDesc()
|
||||
{
|
||||
if (modelDesc_ != nullptr) {
|
||||
(void)aclmdlDestroyDesc(modelDesc_);
|
||||
modelDesc_ = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Result ModelProcess::CreateInput(void *inputDataBuffer, size_t bufferSize)
|
||||
{
|
||||
input_ = aclmdlCreateDataset();
|
||||
if (input_ == nullptr) {
|
||||
ERROR_LOG("can't create dataset, create input failed");
|
||||
return FAILED;
|
||||
}
|
||||
//INFO_LOG("start CreateInput::aclCreateDataBuffer is %d", Utils::getCurrentTime());
|
||||
aclDataBuffer* inputData = aclCreateDataBuffer(inputDataBuffer, bufferSize);
|
||||
//INFO_LOG("end CreateInput::aclCreateDataBuffer is %d", Utils::getCurrentTime());
|
||||
if (inputData == nullptr) {
|
||||
ERROR_LOG("can't create data buffer, create input failed");
|
||||
return FAILED;
|
||||
}
|
||||
//INFO_LOG("start CreateInput::aclmdlAddDatasetBuffer is %d", Utils::getCurrentTime());
|
||||
aclError ret = aclmdlAddDatasetBuffer(input_, inputData);
|
||||
//INFO_LOG("end CreateInput::aclmdlAddDatasetBuffer is %d", Utils::getCurrentTime());
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("add input dataset buffer failed");
|
||||
aclDestroyDataBuffer(inputData);
|
||||
inputData = nullptr;
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
void ModelProcess::DestroyInput()
|
||||
{
|
||||
if (input_ == nullptr) {
|
||||
return;
|
||||
}
|
||||
|
||||
for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(input_); ++i) {
|
||||
aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(input_, i);
|
||||
aclDestroyDataBuffer(dataBuffer);
|
||||
}
|
||||
aclmdlDestroyDataset(input_);
|
||||
input_ = nullptr;
|
||||
}
|
||||
|
||||
Result ModelProcess::CreateOutput()
|
||||
{
|
||||
if (modelDesc_ == nullptr) {
|
||||
ERROR_LOG("no model description, create ouput failed");
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
output_ = aclmdlCreateDataset();
|
||||
if (output_ == nullptr) {
|
||||
ERROR_LOG("can't create dataset, create output failed");
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
size_t outputSize = aclmdlGetNumOutputs(modelDesc_);
|
||||
for (size_t i = 0; i < outputSize; ++i) {
|
||||
size_t buffer_size = aclmdlGetOutputSizeByIndex(modelDesc_, i);
|
||||
|
||||
void *outputBuffer = nullptr;
|
||||
aclError ret = aclrtMalloc(&outputBuffer, buffer_size, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("can't malloc buffer, size is %zu, create output failed", buffer_size);
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
aclDataBuffer* outputData = aclCreateDataBuffer(outputBuffer, buffer_size);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("can't create data buffer, create output failed");
|
||||
aclrtFree(outputBuffer);
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
ret = aclmdlAddDatasetBuffer(output_, outputData);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("can't add data buffer, create output failed");
|
||||
aclrtFree(outputBuffer);
|
||||
aclDestroyDataBuffer(outputData);
|
||||
return FAILED;
|
||||
}
|
||||
}
|
||||
|
||||
INFO_LOG("create model output success");
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
void ModelProcess::DumpModelOutputResult()
|
||||
{
|
||||
stringstream ss;
|
||||
size_t outputNum = aclmdlGetDatasetNumBuffers(output_);
|
||||
static int executeNum = 0;
|
||||
for (size_t i = 0; i < outputNum; ++i) {
|
||||
ss << "output" << ++executeNum << "_" << i << ".bin";
|
||||
string outputFileName = ss.str();
|
||||
FILE *outputFile = fopen(outputFileName.c_str(), "wb");
|
||||
if (outputFile) {
|
||||
aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(output_, i);
|
||||
void* data = aclGetDataBufferAddr(dataBuffer);
|
||||
uint32_t len = aclGetDataBufferSize(dataBuffer);
|
||||
|
||||
void* outHostData = NULL;
|
||||
aclError ret = ACL_ERROR_NONE;
|
||||
if (!g_isDevice) {
|
||||
ret = aclrtMallocHost(&outHostData, len);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("aclrtMallocHost failed, ret[%d]", ret);
|
||||
return;
|
||||
}
|
||||
|
||||
ret = aclrtMemcpy(outHostData, len, data, len, ACL_MEMCPY_DEVICE_TO_HOST);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("aclrtMemcpy failed, ret[%d]", ret);
|
||||
(void)aclrtFreeHost(outHostData);
|
||||
return;
|
||||
}
|
||||
|
||||
fwrite(outHostData, len, sizeof(char), outputFile);
|
||||
|
||||
ret = aclrtFreeHost(outHostData);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("aclrtFreeHost failed, ret[%d]", ret);
|
||||
return;
|
||||
}
|
||||
} else {
|
||||
fwrite(data, len, sizeof(char), outputFile);
|
||||
}
|
||||
fclose(outputFile);
|
||||
outputFile = nullptr;
|
||||
} else {
|
||||
ERROR_LOG("create output file [%s] failed", outputFileName.c_str());
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
INFO_LOG("dump data success");
|
||||
return;
|
||||
}
|
||||
|
||||
void ModelProcess::OutputModelResult()
|
||||
{
|
||||
|
||||
aclError ret = ACL_ERROR_NONE;
|
||||
|
||||
if(outputdata.size()!=0)
|
||||
{
|
||||
for(auto it=outputdata.begin();it!=outputdata.end();++it){
|
||||
ret = aclrtFreeHost(*it);
|
||||
}
|
||||
outputdata.clear();
|
||||
}
|
||||
for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(output_); ++i) {
|
||||
aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(output_, i);
|
||||
void* data = aclGetDataBufferAddr(dataBuffer);
|
||||
uint32_t len = aclGetDataBufferSize(dataBuffer);
|
||||
//INFO_LOG("OutputModelResult::len %d", len);
|
||||
//void *outHostData = NULL;
|
||||
outputdata.push_back(NULL);
|
||||
ret = ACL_ERROR_NONE;
|
||||
float *outData = NULL;
|
||||
if (!g_isDevice) {
|
||||
//INFO_LOG("start OutputModelResult::aclrtMallocHost is %d", Utils::getCurrentTime());
|
||||
aclError ret = aclrtMallocHost(&outputdata[i], len);
|
||||
//INFO_LOG("end OutputModelResult::aclrtMallocHost is %d", Utils::getCurrentTime());
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("aclrtMallocHost failed, ret[%d]", ret);
|
||||
return;
|
||||
}
|
||||
//INFO_LOG("start OutputModelResult::aclrtMemcpy is %d", Utils::getCurrentTime());
|
||||
ret = aclrtMemcpy(outputdata[i], len, data, len, ACL_MEMCPY_DEVICE_TO_HOST);
|
||||
//INFO_LOG("end OutputModelResult::aclrtMemcpy is %d", Utils::getCurrentTime());
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("aclrtMemcpy failed, ret[%d]", ret);
|
||||
return;
|
||||
}
|
||||
|
||||
outData = reinterpret_cast<float*>(outputdata[i]);
|
||||
//outputdata.push_back(outHostData);
|
||||
} else {
|
||||
outData = reinterpret_cast<float*>(data);
|
||||
outputdata.push_back(data);
|
||||
}
|
||||
map<float, unsigned int, greater<float> > resultMap;
|
||||
for (unsigned int j = 0; j < len / sizeof(float); ++j) {
|
||||
resultMap[*outData] = j;
|
||||
outData++;
|
||||
}
|
||||
|
||||
int cnt = 0;
|
||||
for (auto it = resultMap.begin(); it != resultMap.end(); ++it) {
|
||||
// print top 5
|
||||
if (++cnt > 5) {
|
||||
break;
|
||||
}
|
||||
|
||||
//INFO_LOG("top %d: index[%d] value[%lf]", cnt, it->second, it->first);
|
||||
}
|
||||
}
|
||||
|
||||
INFO_LOG("1.output data success");
|
||||
return;
|
||||
}
|
||||
|
||||
void ModelProcess::OutputModelResult(vector<Output_buf> &output)
|
||||
{
|
||||
|
||||
OutputModelResult();
|
||||
|
||||
size_t outputSize = aclmdlGetNumOutputs(modelDesc_);
|
||||
|
||||
|
||||
for (size_t i = 0; i < outputSize; ++i) {
|
||||
Output_buf tmp_output;
|
||||
//ptr
|
||||
tmp_output.ptr = outputdata[i];
|
||||
//size
|
||||
tmp_output.size = aclmdlGetOutputSizeByIndex(modelDesc_, i);
|
||||
//ndim & shape
|
||||
aclmdlIODims d={{'a'},0,{1,2,3}};
|
||||
aclmdlIODims *dims=&d;
|
||||
aclError ret = aclmdlGetOutputDims((const aclmdlDesc *)modelDesc_, i, dims);
|
||||
tmp_output.ndim = (*dims).dimCount;
|
||||
for(int j=0;j<(*dims).dimCount;j++)
|
||||
tmp_output.shape.push_back((*dims).dims[j]);
|
||||
|
||||
//itemsize & format
|
||||
aclDataType dt = aclmdlGetOutputDataType(modelDesc_, i);
|
||||
tmp_output.itemsize=ACLdt_size[dt];
|
||||
tmp_output.format=ACLdt[dt];
|
||||
// if(dt==ACL_FLOAT)
|
||||
// {
|
||||
// tmp_output.itemsize=4;
|
||||
// tmp_output.format="float";
|
||||
// }
|
||||
// if(dt==ACL_FLOAT16)
|
||||
// {
|
||||
// tmp_output.itemsize=2;
|
||||
// tmp_output.format="float16";
|
||||
// }
|
||||
// if(dt==ACL_UINT8)
|
||||
// {
|
||||
// tmp_output.itemsize=1;
|
||||
// tmp_output.format="uint8";
|
||||
// }
|
||||
// if(dt==ACL_INT8)
|
||||
// {
|
||||
// tmp_output.itemsize=1;
|
||||
// tmp_output.format="int8";
|
||||
// }
|
||||
|
||||
//strides
|
||||
int strides=tmp_output.itemsize;
|
||||
std::vector<int64_t> tmp_strides;
|
||||
std::vector<int64_t> tmp1_strides;
|
||||
//vector<int64_t>::reverse_iterator it;//声明一个迭代器，来访问vector容器，作用：遍历或者指向vector容器的元素
|
||||
for(auto it=tmp_output.shape.rbegin();it!=tmp_output.shape.rend();++it)
|
||||
{
|
||||
tmp_strides.push_back(strides);
|
||||
strides = strides * (*it);
|
||||
}
|
||||
//for (int index = 0 ; index < tmp_output.ndim; index++) {
|
||||
// tmp_strides.push_back(strides);
|
||||
// strides = strides * tmp_output.shape[index];
|
||||
//}
|
||||
for(auto it=tmp_strides.rbegin();it!=tmp_strides.rend();++it){
|
||||
//for (int index=0;index < tmp_strides.size(); index++) {
|
||||
tmp1_strides.push_back(*it);
|
||||
}
|
||||
tmp_output.strides = tmp1_strides;
|
||||
output.push_back(tmp_output);
|
||||
}
|
||||
INFO_LOG("2.output data success");
|
||||
}
|
||||
void ModelProcess::DestroyOutput()
|
||||
{
|
||||
if (output_ == nullptr) {
|
||||
return;
|
||||
}
|
||||
|
||||
for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(output_); ++i) {
|
||||
aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(output_, i);
|
||||
void* data = aclGetDataBufferAddr(dataBuffer);
|
||||
(void)aclrtFree(data);
|
||||
(void)aclDestroyDataBuffer(dataBuffer);
|
||||
}
|
||||
|
||||
(void)aclmdlDestroyDataset(output_);
|
||||
output_ = nullptr;
|
||||
}
|
||||
|
||||
Result ModelProcess::Execute()
|
||||
{
|
||||
aclError ret = aclmdlExecute(modelId_, input_, output_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("execute model failed, modelId is %u", modelId_);
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
INFO_LOG("model execute success");
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
void ModelProcess::Unload()
|
||||
{
|
||||
aclError ret;
|
||||
if (!g_isDevice) {
|
||||
//INFO_LOG("start Unload::aclrtFreeHost is %d", Utils::getCurrentTime());
|
||||
//for (int index = 0 ; index < outputdata.size(); index++) {
|
||||
// ret = aclrtFreeHost(outputdata[index]);
|
||||
//}
|
||||
for(auto it=outputdata.begin();it!=outputdata.end();++it){
|
||||
ret = aclrtFreeHost(*it);
|
||||
}
|
||||
//INFO_LOG("end Unload::aclrtFreeHost is %d", Utils::getCurrentTime());
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("aclrtFreeHost failed, ret[%d]", ret);
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (!loadFlag_) {
|
||||
WARN_LOG("no model had been loaded, unload failed");
|
||||
return;
|
||||
}
|
||||
|
||||
ret = aclmdlUnload(modelId_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("unload model failed, modelId is %u", modelId_);
|
||||
}
|
||||
|
||||
if (modelDesc_ != nullptr) {
|
||||
(void)aclmdlDestroyDesc(modelDesc_);
|
||||
modelDesc_ = nullptr;
|
||||
}
|
||||
|
||||
if (modelMemPtr_ != nullptr) {
|
||||
aclrtFree(modelMemPtr_);
|
||||
modelMemPtr_ = nullptr;
|
||||
modelMemSize_ = 0;
|
||||
}
|
||||
|
||||
if (modelWeightPtr_ != nullptr) {
|
||||
aclrtFree(modelWeightPtr_);
|
||||
modelWeightPtr_ = nullptr;
|
||||
modelWeightSize_ = 0;
|
||||
}
|
||||
|
||||
loadFlag_ = false;
|
||||
INFO_LOG("unload model success, modelId is %u", modelId_);
|
||||
}
|
||||
@@ -0,0 +1,249 @@
|
||||
/**
|
||||
* @file sample_process.cpp
|
||||
*
|
||||
* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
*/
|
||||
#include "sample_process.h"
|
||||
#include <iostream>
|
||||
#include "acl/acl.h"
|
||||
#include "utils.h"
|
||||
using namespace std;
|
||||
extern bool g_isDevice;
|
||||
|
||||
|
||||
SampleProcess::SampleProcess() :deviceId_(0), context_(nullptr), stream_(nullptr)
|
||||
{
|
||||
}
|
||||
|
||||
SampleProcess::~SampleProcess()
|
||||
{
|
||||
}
|
||||
|
||||
Result SampleProcess::InitResource(char* omModelPath)
|
||||
{
|
||||
// ACL init
|
||||
const char *aclConfigPath;
|
||||
if((aclConfigPath = getenv("ACL_CONFIG_PATH"))) {
|
||||
INFO_LOG("acl init with config file:%s", aclConfigPath);
|
||||
} else {
|
||||
aclConfigPath = "";
|
||||
}
|
||||
aclError ret = aclInit(aclConfigPath);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("acl init failed");
|
||||
return FAILED;
|
||||
}
|
||||
INFO_LOG("acl init success");
|
||||
|
||||
// open device
|
||||
ret = aclrtSetDevice(deviceId_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("acl open device %d failed", deviceId_);
|
||||
return FAILED;
|
||||
}
|
||||
INFO_LOG("open device %d success", deviceId_);
|
||||
|
||||
// create context (set current)
|
||||
ret = aclrtCreateContext(&context_, deviceId_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("acl create context failed");
|
||||
return FAILED;
|
||||
}
|
||||
INFO_LOG("create context success");
|
||||
|
||||
// create stream
|
||||
ret = aclrtCreateStream(&stream_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("acl create stream failed");
|
||||
return FAILED;
|
||||
}
|
||||
INFO_LOG("create stream success");
|
||||
|
||||
// get run mode
|
||||
aclrtRunMode runMode;
|
||||
ret = aclrtGetRunMode(&runMode);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("acl get run mode failed");
|
||||
return FAILED;
|
||||
}
|
||||
g_isDevice = (runMode == ACL_DEVICE);
|
||||
INFO_LOG("get run mode success");
|
||||
Result r_ret = processModel.LoadModelFromFileWithMem(omModelPath);
|
||||
if (r_ret != SUCCESS) {
|
||||
ERROR_LOG("load model from file failed");
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
ret = processModel.CreateDesc();
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("create model description failed");
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
ret = processModel.CreateOutput();
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("create model output failed");
|
||||
return FAILED;
|
||||
}
|
||||
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
Result SampleProcess::Process(void* binfile,int len)
|
||||
{
|
||||
//for (size_t index = 0; index < sizeof(testFile) / sizeof(testFile[0]); ++index) {
|
||||
//INFO_LOG("start to process file:%s", binfile.c_str());
|
||||
// model process
|
||||
//INFO_LOG("start memcpy is %d", Utils::getCurrentTime());
|
||||
uint32_t devBufferSize = len;
|
||||
void *picDevBuffer = Utils::GetDeviceBufferOfptr(binfile,len);
|
||||
//ERROR_LOG("devBufferSize is %d", len);
|
||||
//ERROR_LOG("picDevBuffer:%f.", *((float*)picDevBuffer));
|
||||
if (picDevBuffer == nullptr) {
|
||||
ERROR_LOG("get pic device buffer failed,index is %zu", 0);
|
||||
return FAILED;
|
||||
}
|
||||
//INFO_LOG("end memcpy is %d", Utils::getCurrentTime());
|
||||
|
||||
//INFO_LOG("start CreateInput is %d", Utils::getCurrentTime());
|
||||
Result ret = processModel.CreateInput(picDevBuffer, devBufferSize);
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("model create input failed");
|
||||
aclrtFree(picDevBuffer);
|
||||
return FAILED;
|
||||
}
|
||||
//INFO_LOG("end CreateInput is %d", Utils::getCurrentTime());
|
||||
|
||||
INFO_LOG("start Execute is %d", Utils::getCurrentTime());
|
||||
ret = processModel.Execute();
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("model execute failed");
|
||||
aclrtFree(picDevBuffer);
|
||||
return FAILED;
|
||||
}
|
||||
INFO_LOG("end Execute is %d", Utils::getCurrentTime());
|
||||
|
||||
//INFO_LOG("start aclrtFree is %d", Utils::getCurrentTime());
|
||||
// release model input buffer
|
||||
aclrtFree(picDevBuffer);
|
||||
//INFO_LOG("end aclrtFree is %d", Utils::getCurrentTime());
|
||||
|
||||
// print the top 5 confidence values with indexes.use function DumpModelOutputResult
|
||||
// if want to dump output result to file in the current directory
|
||||
processModel.OutputModelResult();
|
||||
|
||||
//}
|
||||
// loop end
|
||||
|
||||
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
Result SampleProcess::Process(void* binfile,int len, vector<Output_buf> &output, long &npuTime)
|
||||
{
|
||||
//for (size_t index = 0; index < sizeof(testFile) / sizeof(testFile[0]); ++index) {
|
||||
//INFO_LOG("start to process file:%s", binfile.c_str());
|
||||
// model process
|
||||
//INFO_LOG("start memcpy is %d", Utils::getCurrentTime());
|
||||
uint32_t devBufferSize = len;
|
||||
void *picDevBuffer = Utils::GetDeviceBufferOfptr(binfile,len);
|
||||
//ERROR_LOG("devBufferSize is %d", len);
|
||||
//ERROR_LOG("picDevBuffer:%f.", *((float*)picDevBuffer));
|
||||
if (picDevBuffer == nullptr) {
|
||||
ERROR_LOG("get pic device buffer failed,index is %zu", 0);
|
||||
return FAILED;
|
||||
}
|
||||
//INFO_LOG("end memcpy is %d", Utils::getCurrentTime());
|
||||
|
||||
//INFO_LOG("start CreateInput is %d", Utils::getCurrentTime());
|
||||
Result ret = processModel.CreateInput(picDevBuffer, devBufferSize);
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("model create input failed");
|
||||
aclrtFree(picDevBuffer);
|
||||
return FAILED;
|
||||
}
|
||||
//INFO_LOG("end CreateInput is %d", Utils::getCurrentTime());
|
||||
|
||||
INFO_LOG("start Execute is %d", Utils::getCurrentTime());
|
||||
long start = Utils::getCurrentTime();
|
||||
ret = processModel.Execute();
|
||||
if (ret != SUCCESS) {
|
||||
ERROR_LOG("model execute failed");
|
||||
aclrtFree(picDevBuffer);
|
||||
return FAILED;
|
||||
}
|
||||
long end = Utils::getCurrentTime();
|
||||
INFO_LOG("end Execute is %d", end);
|
||||
|
||||
npuTime = end - start;
|
||||
if (npuTime < 0) {
|
||||
ERROR_LOG("NPU time not correct: %d start: %d end: %d", npuTime, start, end);
|
||||
aclrtFree(picDevBuffer);
|
||||
return FAILED;
|
||||
}
|
||||
INFO_LOG("npu compute cost %f ms", 1.0*(npuTime)/1000.0);
|
||||
|
||||
//INFO_LOG("start aclrtFree is %d", Utils::getCurrentTime());
|
||||
// release model input buffer
|
||||
aclrtFree(picDevBuffer);
|
||||
//INFO_LOG("end aclrtFree is %d", Utils::getCurrentTime());
|
||||
|
||||
// print the top 5 confidence values with indexes.use function DumpModelOutputResult
|
||||
// if want to dump output result to file in the current directory
|
||||
processModel.OutputModelResult(output);
|
||||
|
||||
//}
|
||||
// loop end
|
||||
|
||||
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
Result SampleProcess::Unload()
|
||||
{
|
||||
processModel.Unload();
|
||||
processModel.DestroyDesc();
|
||||
processModel.DestroyInput();
|
||||
processModel.DestroyOutput();
|
||||
DestroyResource();
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
void SampleProcess::DestroyResource()
|
||||
{
|
||||
aclError ret;
|
||||
if (stream_ != nullptr) {
|
||||
ret = aclrtDestroyStream(stream_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("destroy stream failed");
|
||||
}
|
||||
stream_ = nullptr;
|
||||
INFO_LOG("end to destroy stream");
|
||||
}
|
||||
|
||||
if (context_ != nullptr) {
|
||||
ret = aclrtDestroyContext(context_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("destroy context failed");
|
||||
}
|
||||
context_ = nullptr;
|
||||
INFO_LOG("end to destroy context");
|
||||
}
|
||||
|
||||
ret = aclrtResetDevice(deviceId_);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("reset device failed");
|
||||
}
|
||||
INFO_LOG("end to reset device is %d", deviceId_);
|
||||
|
||||
ret = aclFinalize();
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("finalize acl failed");
|
||||
}
|
||||
INFO_LOG("end to finalize acl");
|
||||
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
/**
|
||||
* @file utils.cpp
|
||||
*
|
||||
* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
*/
|
||||
#include "utils.h"
|
||||
#include <iostream>
|
||||
#include <fstream>
|
||||
#include <cstring>
|
||||
#include "acl/acl.h"
|
||||
#include <stdio.h>
|
||||
#include <sys/time.h>
|
||||
|
||||
bool g_isDevice = false;
|
||||
|
||||
std::map<aclDataType, std::string> ACLdt{{ ACL_DT_UNDEFINED , "undefined"},
|
||||
{ ACL_FLOAT , "float"},
|
||||
{ ACL_FLOAT16 , "float16"},
|
||||
{ ACL_INT8 , "int8"},
|
||||
{ ACL_INT32 , "int32"},
|
||||
{ ACL_UINT8 , "uint8"},
|
||||
{ ACL_INT16 , "int16"},
|
||||
{ ACL_UINT16 , "uint16"},
|
||||
{ ACL_UINT32 , "uint32"},
|
||||
{ ACL_INT64 , "int64"},
|
||||
{ ACL_UINT64 , "uint64"},
|
||||
{ ACL_DOUBLE , "double"},
|
||||
{ ACL_BOOL , "bool"}};
|
||||
std::map<aclDataType, int> ACLdt_size{{ ACL_DT_UNDEFINED , -1},
|
||||
{ ACL_FLOAT , 4},
|
||||
{ ACL_FLOAT16 , 2},
|
||||
{ ACL_INT8 , 1},
|
||||
{ ACL_INT32 , 4},
|
||||
{ ACL_UINT8 , 1},
|
||||
{ ACL_INT16 , 2},
|
||||
{ ACL_UINT16 , 2},
|
||||
{ ACL_UINT32 , 4},
|
||||
{ ACL_INT64 , 8},
|
||||
{ ACL_UINT64 , 8},
|
||||
{ ACL_DOUBLE , 8},
|
||||
{ ACL_BOOL , 1}};
|
||||
|
||||
//Config configSettings("cfg/built-in_config.txt");
|
||||
|
||||
void* Utils::ReadBinFile(std::string fileName, uint32_t &fileSize)
|
||||
{
|
||||
std::ifstream binFile(fileName, std::ifstream::binary);
|
||||
if (binFile.is_open() == false) {
|
||||
ERROR_LOG("open file %s failed", fileName.c_str());
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
binFile.seekg(0, binFile.end);
|
||||
uint32_t binFileBufferLen = binFile.tellg();
|
||||
if (binFileBufferLen == 0) {
|
||||
ERROR_LOG("binfile is empty, filename is %s", fileName.c_str());
|
||||
binFile.close();
|
||||
return nullptr;
|
||||
}
|
||||
//ERROR_LOG("binFileBufferLen is %d", binFileBufferLen);
|
||||
|
||||
binFile.seekg(0, binFile.beg);
|
||||
|
||||
void* binFileBufferData = nullptr;
|
||||
aclError ret = ACL_ERROR_NONE;
|
||||
if (!g_isDevice) {
|
||||
ret = aclrtMallocHost(&binFileBufferData, binFileBufferLen);
|
||||
if (binFileBufferData == nullptr) {
|
||||
ERROR_LOG("malloc binFileBufferData failed");
|
||||
binFile.close();
|
||||
return nullptr;
|
||||
}
|
||||
} else {
|
||||
ret = aclrtMalloc(&binFileBufferData, binFileBufferLen, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("malloc device buffer failed. size is %u", binFileBufferLen);
|
||||
binFile.close();
|
||||
return nullptr;
|
||||
}
|
||||
}
|
||||
binFile.read(static_cast<char *>(binFileBufferData), binFileBufferLen);
|
||||
binFile.close();
|
||||
fileSize = binFileBufferLen;
|
||||
return binFileBufferData;
|
||||
}
|
||||
|
||||
void* Utils::GetDeviceBufferOfFile(std::string fileName, uint32_t &fileSize)
|
||||
{
|
||||
uint32_t inputHostBuffSize = 0;
|
||||
void* inputHostBuff = Utils::ReadBinFile(fileName, inputHostBuffSize);
|
||||
if (inputHostBuff == nullptr) {
|
||||
return nullptr;
|
||||
}
|
||||
if (!g_isDevice) {
|
||||
void *inBufferDev = nullptr;
|
||||
uint32_t inBufferSize = inputHostBuffSize;
|
||||
aclError ret = aclrtMalloc(&inBufferDev, inBufferSize, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("malloc device buffer failed. size is %u", inBufferSize);
|
||||
aclrtFreeHost(inputHostBuff);
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
ret = aclrtMemcpy(inBufferDev, inBufferSize, inputHostBuff, inputHostBuffSize, ACL_MEMCPY_HOST_TO_DEVICE);
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("memcpy failed. device buffer size is %u, input host buffer size is %u",
|
||||
inBufferSize, inputHostBuffSize);
|
||||
aclrtFree(inBufferDev);
|
||||
aclrtFreeHost(inputHostBuff);
|
||||
return nullptr;
|
||||
}
|
||||
aclrtFreeHost(inputHostBuff);
|
||||
fileSize = inBufferSize;
|
||||
return inBufferDev;
|
||||
} else {
|
||||
fileSize = inputHostBuffSize;
|
||||
return inputHostBuff;
|
||||
}
|
||||
}
|
||||
|
||||
void* Utils::GetDeviceBufferOfptr(void* fileName, uint32_t len)
|
||||
{
|
||||
uint32_t inputHostBuffSize = len;
|
||||
void* inputHostBuff = fileName;
|
||||
if (inputHostBuff == nullptr) {
|
||||
return nullptr;
|
||||
}
|
||||
if (!g_isDevice) {
|
||||
void *inBufferDev = nullptr;
|
||||
uint32_t inBufferSize = inputHostBuffSize;
|
||||
//INFO_LOG("start memcpy::aclrtMalloc is %d", Utils::getCurrentTime());
|
||||
aclError ret = aclrtMalloc(&inBufferDev, inBufferSize, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
|
||||
//INFO_LOG("end memcpy::aclrtMalloc is %d", Utils::getCurrentTime());
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("malloc device buffer failed. size is %u", inBufferSize);
|
||||
aclrtFreeHost(inputHostBuff);
|
||||
return nullptr;
|
||||
}
|
||||
//INFO_LOG("start memcpy::aclrtMemcpy is %d", Utils::getCurrentTime());
|
||||
ret = aclrtMemcpy(inBufferDev, inBufferSize, inputHostBuff, inputHostBuffSize, ACL_MEMCPY_HOST_TO_DEVICE);
|
||||
//INFO_LOG("end memcpy::aclrtMemcpy is %d", Utils::getCurrentTime());
|
||||
if (ret != ACL_ERROR_NONE) {
|
||||
ERROR_LOG("memcpy failed. device buffer size is %u, input host buffer size is %u",
|
||||
inBufferSize, inputHostBuffSize);
|
||||
aclrtFree(inBufferDev);
|
||||
aclrtFreeHost(inputHostBuff);
|
||||
return nullptr;
|
||||
}
|
||||
//INFO_LOG("start memcpy::aclrtFreeHost is %d", Utils::getCurrentTime());
|
||||
aclrtFreeHost(inputHostBuff);
|
||||
//INFO_LOG("end memcpy::aclrtFreeHost is %d", Utils::getCurrentTime());
|
||||
//fileSize = inBufferSize;
|
||||
return inBufferDev;
|
||||
} else {
|
||||
//fileSize = inputHostBuffSize;
|
||||
return inputHostBuff;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
long Utils::getCurrentTime()
|
||||
{
|
||||
struct timeval tv;
|
||||
gettimeofday(&tv,NULL);
|
||||
return tv.tv_sec * 1000000 + tv.tv_usec;
|
||||
// return tv.tv_usec;
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
#include "trtbackend.h"
|
||||
|
||||
CREATE_BACKEND_INIT(TrtBackend,model,data)
|
||||
{
|
||||
INFO_LOG("TrtBackend INIT model success");
|
||||
return 0;
|
||||
}
|
||||
|
||||
CREATE_BACKEND_LOAD(TrtBackend,model,data)
|
||||
{
|
||||
INFO_LOG("TrtBackend LOAD model success");
|
||||
return 0;
|
||||
}
|
||||
|
||||
CREATE_BACKEND_PREDICT(TrtBackend,model,data,len,output,gpuTime)
|
||||
{
|
||||
INFO_LOG("TrtBackend PREDICT model success");
|
||||
return 0;
|
||||
}
|
||||
CREATE_BACKEND_UNLOAD(TrtBackend,model,data)
|
||||
{
|
||||
INFO_LOG("TrtBackend UNLOAD model success");
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
REGISTER_BASE_BACKEND_CREATOR(TRT, TrtBackend)
|
||||
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
#include "BackendFactory.h"
|
||||
#include <vector>
|
||||
|
||||
CREATE_BACKEND(TrtBackend)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
/**
|
||||
* Copyright 2020 Huawei Technologies Co., Ltd
|
||||
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
#ifndef BASE_BACKEND_FACTORY_H_
|
||||
#define BASE_BACKEND_FACTORY_H_
|
||||
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <string>
|
||||
#include "BaseBackend.h"
|
||||
|
||||
|
||||
|
||||
typedef std::shared_ptr<BaseBackend> (*BASE_BACKEND_CREATOR_FUN)(void);
|
||||
|
||||
class BackendFactory {
|
||||
public:
|
||||
static BackendFactory *Instance();
|
||||
int Init();
|
||||
/**
|
||||
* @ingroup domi_omg
|
||||
* @brief Create a backend based on the type entered
|
||||
* @param [in] type Framework type
|
||||
* @return Created backend
|
||||
*/
|
||||
std::shared_ptr<BaseBackend> CreateBaseBackend(const FrameworkType type);
|
||||
|
||||
/**
|
||||
* @ingroup domi_omg
|
||||
* @brief Register create function
|
||||
* @param [in] type Framework type
|
||||
* @param [in] fun ModelParser's create function
|
||||
*/
|
||||
void RegisterCreator(const FrameworkType type, BASE_BACKEND_CREATOR_FUN fun);
|
||||
|
||||
protected:
|
||||
BackendFactory() {}
|
||||
~BackendFactory();
|
||||
|
||||
private:
|
||||
std::map<FrameworkType, BASE_BACKEND_CREATOR_FUN> creator_map_;
|
||||
bool is_init_ = false;
|
||||
}; // end class BackendFactory
|
||||
|
||||
class BaseBackendRegisterar {
|
||||
public:
|
||||
BaseBackendRegisterar(const FrameworkType type, BASE_BACKEND_CREATOR_FUN fun) {
|
||||
BackendFactory::Instance()->RegisterCreator(type, fun);
|
||||
}
|
||||
~BaseBackendRegisterar() {}
|
||||
};
|
||||
|
||||
// Registration macros for backend
|
||||
typedef std::shared_ptr<BaseBackend> (*BASE_BACKEND_CREATOR_FUN)(void);
|
||||
|
||||
// Registration macros for BaseBackend
|
||||
#define REGISTER_BASE_BACKEND_CREATOR(type, clazz) \
|
||||
std::shared_ptr<BaseBackend> Creator_##type##_Base_Backend() { \
|
||||
std::shared_ptr<clazz> ptr = nullptr; \
|
||||
try { \
|
||||
ptr = std::make_shared<clazz>(); \
|
||||
} catch (...) { \
|
||||
ptr = nullptr; \
|
||||
} \
|
||||
return std::shared_ptr<BaseBackend>(ptr); \
|
||||
} \
|
||||
BaseBackendRegisterar g_##type##_Base_Backend_Creator(type, Creator_##type##_Base_Backend);
|
||||
|
||||
|
||||
#endif
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
|
||||
/**
|
||||
* Copyright 2020 Huawei Technologies Co., Ltd
|
||||
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
#ifndef BASE_BACKEND_H_
|
||||
#define BASE_BACKEND_H_
|
||||
|
||||
//#include "utils.h"
|
||||
#include "common.h"
|
||||
#include <vector>
|
||||
using namespace std;
|
||||
|
||||
class BaseBackend {
|
||||
public:
|
||||
BaseBackend() {};
|
||||
/**
|
||||
* @ingroup domi_omg
|
||||
* @brief Deconstructor
|
||||
*/
|
||||
~BaseBackend() {};
|
||||
|
||||
virtual int init(char* model,char* data){};
|
||||
|
||||
virtual int load(char* model,char* data){};
|
||||
|
||||
virtual int predict(char* model,void* data,int len, std::vector<Output_buf> &output, long &deviceTime){};
|
||||
|
||||
virtual int unload(char* model,char* data){};
|
||||
|
||||
int param();
|
||||
|
||||
int runner();
|
||||
|
||||
int runnerthread();
|
||||
|
||||
int statistic();
|
||||
|
||||
};
|
||||
|
||||
|
||||
|
||||
#define CREATE_BACKEND(x) \
|
||||
class x : public BaseBackend{ \
|
||||
public: \
|
||||
x(){}; \
|
||||
~x(){}; \
|
||||
int init(char* model,char* data); \
|
||||
int load(char* model,char* data); \
|
||||
int predict(char* model,void* data,int len, vector<Output_buf> &output, long &deviceTime); \
|
||||
int unload(char* model,char* data); \
|
||||
char* model; \
|
||||
char* data; \
|
||||
}; \
|
||||
|
||||
#define CREATE_BACKEND_INIT(x,model,data) \
|
||||
int x::init(char* model,char* data)
|
||||
#define CREATE_BACKEND_LOAD(x,model,data) \
|
||||
int x::load(char* model,char* data)
|
||||
#define CREATE_BACKEND_PREDICT(x,model,data,len,output,deviceTime) \
|
||||
int x::predict(char* model,void* data,int len,vector<Output_buf> &output, long &deviceTime)
|
||||
#define CREATE_BACKEND_UNLOAD(x,model,data) \
|
||||
int x::unload(char* model,char* data)
|
||||
|
||||
#endif
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
//Config.h
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
#include <map>
|
||||
#include <iostream>
|
||||
#include <fstream>
|
||||
#include <sstream>
|
||||
|
||||
|
||||
/*
|
||||
* \brief Generic configuration Class
|
||||
*
|
||||
*/
|
||||
class Config {
|
||||
// Data
|
||||
protected:
|
||||
std::string m_Delimiter; //!< separator between key and value
|
||||
std::string m_Comment; //!< separator between value and comments
|
||||
std::map<std::string,std::string> m_Contents; //!< extracted keys and values
|
||||
|
||||
typedef std::map<std::string,std::string>::iterator mapi;
|
||||
typedef std::map<std::string,std::string>::const_iterator mapci;
|
||||
// Methods
|
||||
public:
|
||||
static Config *instance;
|
||||
static Config *setInstance(char* cfg){
|
||||
if(instance == nullptr){
|
||||
instance = new Config(cfg);
|
||||
}
|
||||
return instance;
|
||||
}
|
||||
static Config *getInstance(){
|
||||
return instance;
|
||||
}
|
||||
Config( std::string filename,std::string delimiter = "=",std::string comment = "#" );
|
||||
Config();
|
||||
template<class T> T Read( const std::string& in_key ) const; //!<Search for key and read value or optional default value, call as read<T>
|
||||
template<class T> T Read( const std::string& in_key, const T& in_value ) const;
|
||||
template<class T> bool ReadInto( T& out_var, const std::string& in_key ) const;
|
||||
template<class T>
|
||||
bool ReadInto( T& out_var, const std::string& in_key, const T& in_value ) const;
|
||||
bool FileExist(std::string filename);
|
||||
void ReadFile(std::string filename,std::string delimiter = "=",std::string comment = "#" );
|
||||
|
||||
// Check whether key exists in configuration
|
||||
bool KeyExists( const std::string& in_key ) const;
|
||||
|
||||
// Modify keys and values
|
||||
template<class T> void Add( const std::string& in_key, const T& in_value );
|
||||
void Remove( const std::string& in_key );
|
||||
|
||||
// Check or change configuration syntax
|
||||
std::string GetDelimiter() const { return m_Delimiter; }
|
||||
std::string GetComment() const { return m_Comment; }
|
||||
std::string SetDelimiter( const std::string& in_s )
|
||||
{ std::string old = m_Delimiter; m_Delimiter = in_s; return old; }
|
||||
std::string SetComment( const std::string& in_s )
|
||||
{ std::string old = m_Comment; m_Comment = in_s; return old; }
|
||||
|
||||
// Write or read configuration
|
||||
friend std::ostream& operator<<( std::ostream& os, const Config& cf );
|
||||
friend std::istream& operator>>( std::istream& is, Config& cf );
|
||||
|
||||
protected:
|
||||
template<class T> static std::string T_as_string( const T& t );
|
||||
template<class T> static T string_as_T( const std::string& s );
|
||||
static void Trim( std::string& inout_s );
|
||||
|
||||
|
||||
// Exception types
|
||||
public:
|
||||
struct File_not_found {
|
||||
std::string filename;
|
||||
File_not_found( const std::string& filename_ = std::string() )
|
||||
: filename(filename_) {} };
|
||||
struct Key_not_found { // thrown only by T read(key) variant of read()
|
||||
std::string key;
|
||||
Key_not_found( const std::string& key_ = std::string() )
|
||||
: key(key_) {} };
|
||||
};
|
||||
|
||||
|
||||
/* static */
|
||||
template<class T>
|
||||
std::string Config::T_as_string( const T& t )
|
||||
{
|
||||
// Convert from a T to a string
|
||||
// Type T must support << operator
|
||||
std::ostringstream ost;
|
||||
ost << t;
|
||||
return ost.str();
|
||||
}
|
||||
|
||||
|
||||
/* static */
|
||||
template<class T>
|
||||
T Config::string_as_T( const std::string& s )
|
||||
{
|
||||
// Convert from a string to a T
|
||||
// Type T must support >> operator
|
||||
T t;
|
||||
std::istringstream ist(s);
|
||||
ist >> t;
|
||||
return t;
|
||||
}
|
||||
|
||||
|
||||
/* static */
|
||||
template<>
|
||||
inline std::string Config::string_as_T<std::string>( const std::string& s )
|
||||
{
|
||||
// Convert from a string to a string
|
||||
// In other words, do nothing
|
||||
return s;
|
||||
}
|
||||
|
||||
|
||||
/* static */
|
||||
template<>
|
||||
inline bool Config::string_as_T<bool>( const std::string& s )
|
||||
{
|
||||
// Convert from a string to a bool
|
||||
// Interpret "false", "F", "no", "n", "0" as false
|
||||
// Interpret "true", "T", "yes", "y", "1", "-1", or anything else as true
|
||||
bool b = true;
|
||||
std::string sup = s;
|
||||
for( std::string::iterator p = sup.begin(); p != sup.end(); ++p )
|
||||
*p = toupper(*p); // make string all caps
|
||||
if( sup==std::string("FALSE") || sup==std::string("F") ||
|
||||
sup==std::string("NO") || sup==std::string("N") ||
|
||||
sup==std::string("0") || sup==std::string("NONE") )
|
||||
b = false;
|
||||
return b;
|
||||
}
|
||||
|
||||
|
||||
template<class T>
|
||||
T Config::Read( const std::string& key ) const
|
||||
{
|
||||
// Read the value corresponding to key
|
||||
mapci p = m_Contents.find(key);
|
||||
if( p == m_Contents.end() ) throw Key_not_found(key);
|
||||
return string_as_T<T>( p->second );
|
||||
}
|
||||
|
||||
|
||||
template<class T>
|
||||
T Config::Read( const std::string& key, const T& value ) const
|
||||
{
|
||||
// Return the value corresponding to key or given default value
|
||||
// if key is not found
|
||||
mapci p = m_Contents.find(key);
|
||||
if( p == m_Contents.end() ) return value;
|
||||
return string_as_T<T>( p->second );
|
||||
}
|
||||
|
||||
|
||||
template<class T>
|
||||
bool Config::ReadInto( T& var, const std::string& key ) const
|
||||
{
|
||||
// Get the value corresponding to key and store in var
|
||||
// Return true if key is found
|
||||
// Otherwise leave var untouched
|
||||
mapci p = m_Contents.find(key);
|
||||
bool found = ( p != m_Contents.end() );
|
||||
if( found ) var = string_as_T<T>( p->second );
|
||||
return found;
|
||||
}
|
||||
|
||||
|
||||
template<class T>
|
||||
bool Config::ReadInto( T& var, const std::string& key, const T& value ) const
|
||||
{
|
||||
// Get the value corresponding to key and store in var
|
||||
// Return true if key is found
|
||||
// Otherwise set var to given default
|
||||
mapci p = m_Contents.find(key);
|
||||
bool found = ( p != m_Contents.end() );
|
||||
if( found )
|
||||
var = string_as_T<T>( p->second );
|
||||
else
|
||||
var = value;
|
||||
return found;
|
||||
}
|
||||
|
||||
|
||||
template<class T>
|
||||
void Config::Add( const std::string& in_key, const T& value )
|
||||
{
|
||||
// Add a key with given value
|
||||
std::string v = T_as_string( value );
|
||||
std::string key=in_key;
|
||||
trim(key);
|
||||
trim(v);
|
||||
m_Contents[key] = v;
|
||||
return;
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
/**
|
||||
* @file common.h
|
||||
*
|
||||
* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
*/
|
||||
#pragma once
|
||||
#include <iostream>
|
||||
#include<vector>
|
||||
#include <pybind11/numpy.h>
|
||||
#include <pybind11/stl.h>
|
||||
#include "half.hpp"
|
||||
#include "Config.h"
|
||||
|
||||
#define LOG 2
|
||||
|
||||
#define INFO_LOG(fmt, args...) if(Config::getInstance()->Read("backend_loglevel", 0)>=3) fprintf(stdout, "[INFO] " fmt "\n", ##args)
|
||||
#define WARN_LOG(fmt, args...) if(Config::getInstance()->Read("backend_loglevel", 0)>=2) fprintf(stdout, "[WARN] " fmt "\n", ##args)
|
||||
#define ERROR_LOG(fmt, args...) if(Config::getInstance()->Read("backend_loglevel", 0)>=1) fprintf(stdout, "[ERROR] " fmt "\n", ##args)
|
||||
|
||||
namespace py = pybind11;
|
||||
// half_float::half behaviors like float, but with different precision
|
||||
|
||||
using float16 = half_float::half;
|
||||
|
||||
NAMESPACE_BEGIN(pybind11)
|
||||
NAMESPACE_BEGIN(detail)
|
||||
|
||||
template <>
|
||||
struct type_caster<float16> {
|
||||
public:
|
||||
PYBIND11_TYPE_CASTER(float16, _("float16"));
|
||||
using float_caster = type_caster<float>;
|
||||
|
||||
bool load(handle src, bool convert) {
|
||||
float_caster caster;
|
||||
if (caster.load(src, convert)) {
|
||||
this->value = float16(float(caster)); // Implicit cast defined by `type_caster`.
|
||||
return true;
|
||||
|
||||
}
|
||||
return false;
|
||||
}
|
||||
static handle cast(float16 src, return_value_policy policy, handle parent) {
|
||||
return float_caster::cast(float(src), policy, parent);
|
||||
}
|
||||
};
|
||||
|
||||
constexpr int NPY_FLOAT16 = 23;
|
||||
|
||||
template <>
|
||||
struct npy_format_descriptor<float16> {
|
||||
static pybind11::dtype dtype() {
|
||||
handle ptr = npy_api::get().PyArray_DescrFromType_(NPY_FLOAT16);
|
||||
return reinterpret_borrow<pybind11::dtype>(ptr);
|
||||
}
|
||||
static std::string format() {
|
||||
// following: https://docs.python.org/3/library/struct.html#format-characters
|
||||
return "e";
|
||||
}
|
||||
static constexpr auto name() {
|
||||
return _("float16");
|
||||
}
|
||||
};
|
||||
|
||||
NAMESPACE_END(detail)
|
||||
NAMESPACE_END(pybind11)
|
||||
|
||||
|
||||
|
||||
typedef enum Result {
|
||||
SUCCESS = 0,
|
||||
FAILED = 1
|
||||
} Result;
|
||||
|
||||
enum FrameworkType {
|
||||
CAFFE = 0,
|
||||
MINDSPORE = 1,
|
||||
TENSORFLOW = 3,
|
||||
ANDROID_NN,
|
||||
ACL,
|
||||
TRT,
|
||||
HIAI_ENGINE,
|
||||
FRAMEWORK_RESERVED,
|
||||
};
|
||||
|
||||
struct Output_buf {
|
||||
void *ptr = nullptr; // Pointer to the underlying storage
|
||||
int64_t itemsize = 0; // Size of individual items in bytes
|
||||
int64_t size = 0; // Total number of entries
|
||||
std::string format; // For homogeneous buffers, this should be set to format_descriptor<T>::format()
|
||||
int64_t ndim = 0; // Number of dimensions
|
||||
std::vector<int64_t> shape; // Shape of the tensor (1 entry per dimension)
|
||||
std::vector<int64_t> strides; // Number of bytes between adjacent entries (for each per dimension)
|
||||
bool readonly = false; // flag to indicate if the underlying storage may be written to
|
||||
};
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,140 @@
|
||||
from setuptools import setup, Extension
|
||||
from setuptools.command.build_ext import build_ext
|
||||
import sys
|
||||
import setuptools
|
||||
|
||||
__version__ = 'V1.0.2'
|
||||
|
||||
|
||||
class get_pybind_include(object):
|
||||
"""Helper class to determine the pybind11 include path
|
||||
|
||||
The purpose of this class is to postpone importing pybind11
|
||||
until it is actually installed, so that the ``get_include()``
|
||||
method can be invoked. """
|
||||
|
||||
def __str__(self):
|
||||
import pybind11
|
||||
return pybind11.get_include()
|
||||
|
||||
|
||||
ext_modules = [
|
||||
Extension(
|
||||
'dnmetis_backend',
|
||||
# Sort input source files to ensure bit-for-bit reproducible builds
|
||||
# (https://github.com/pybind/dnmetis_backend/pull/53)
|
||||
#sorted(['src/main.cpp','src/model_process.cpp','src/sample_process.cpp','src/utils.cpp']),
|
||||
sources=[
|
||||
'src/main.cpp',
|
||||
'src/Config.cpp',
|
||||
'backend/BaseBackend.cpp',
|
||||
'backend/BackendFactory.cpp',
|
||||
|
||||
'backend/built-in/aclbackend.cpp',
|
||||
'backend/built-in/src/model_process.cpp',
|
||||
'backend/built-in/src/sample_process.cpp',
|
||||
'backend/built-in/src/utils.cpp',
|
||||
|
||||
'backend/custom/trtbackend.cpp',
|
||||
],
|
||||
include_dirs=[
|
||||
# Path to pybind11 headers
|
||||
get_pybind_include(),
|
||||
'./inc/',
|
||||
'./backend/inc',
|
||||
'./backend/built-in',
|
||||
'./backend/built-in/inc',
|
||||
'./backend/custom',
|
||||
'./backend/custom/inc',
|
||||
'/home/HwHiAiUser/Ascend/ascend-toolkit/20.10.0.B023//acllib/include/',
|
||||
],
|
||||
library_dirs=['/home/HwHiAiUser/Ascend/ascend-toolkit/20.10.0.B023//acllib/lib64/',],
|
||||
libraries=['ascendcl',],
|
||||
language='c++'
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
# cf http://bugs.python.org/issue26689
|
||||
def has_flag(compiler, flagname):
|
||||
"""Return a boolean indicating whether a flag name is supported on
|
||||
the specified compiler.
|
||||
"""
|
||||
import tempfile
|
||||
import os
|
||||
with tempfile.NamedTemporaryFile('w', suffix='.cpp', delete=False) as f:
|
||||
f.write('int main (int argc, char **argv) { return 0; }')
|
||||
fname = f.name
|
||||
try:
|
||||
compiler.compile([fname], extra_postargs=[flagname])
|
||||
except setuptools.distutils.errors.CompileError:
|
||||
return False
|
||||
finally:
|
||||
try:
|
||||
os.remove(fname)
|
||||
except OSError:
|
||||
pass
|
||||
return True
|
||||
|
||||
|
||||
def cpp_flag(compiler):
|
||||
"""Return the -std=c++[11/14/17] compiler flag.
|
||||
|
||||
The newer version is prefered over c++11 (when it is available).
|
||||
"""
|
||||
flags = ['-std=c++14', '-std=c++17','-std=c++11']
|
||||
|
||||
for flag in flags:
|
||||
if has_flag(compiler, flag):
|
||||
return flag
|
||||
|
||||
raise RuntimeError('Unsupported compiler -- at least C++11 support '
|
||||
'is needed!')
|
||||
|
||||
|
||||
class BuildExt(build_ext):
|
||||
"""A custom build extension for adding compiler-specific options."""
|
||||
c_opts = {
|
||||
'msvc': ['/EHsc'],
|
||||
'unix': ['-w','-O0', '-fpermissive'],
|
||||
}
|
||||
l_opts = {
|
||||
'msvc': [],
|
||||
'unix': ['-O0'],
|
||||
}
|
||||
|
||||
if sys.platform == 'darwin':
|
||||
darwin_opts = ['-stdlib=libc++', '-mmacosx-version-min=10.7']
|
||||
c_opts['unix'] += darwin_opts
|
||||
l_opts['unix'] += darwin_opts
|
||||
|
||||
def build_extensions(self):
|
||||
ct = self.compiler.compiler_type
|
||||
opts = self.c_opts.get(ct, [])
|
||||
link_opts = self.l_opts.get(ct, [])
|
||||
if ct == 'unix':
|
||||
opts.append(cpp_flag(self.compiler))
|
||||
if has_flag(self.compiler, '-fvisibility=hidden'):
|
||||
opts.append('-fvisibility=hidden')
|
||||
|
||||
for ext in self.extensions:
|
||||
ext.define_macros = [('VERSION_INFO', '"{}"'.format(self.distribution.get_version()))]
|
||||
ext.extra_compile_args = opts
|
||||
ext.extra_link_args = link_opts
|
||||
build_ext.build_extensions(self)
|
||||
|
||||
|
||||
setup(
|
||||
name='dnmetis_backend',
|
||||
version=__version__,
|
||||
author='chegulu',
|
||||
author_email='chegulu@xxx.com',
|
||||
url='https://chegulu',
|
||||
description='A test tool project using pybind11',
|
||||
long_description='',
|
||||
ext_modules=ext_modules,
|
||||
setup_requires=['pybind11==2.5.0'],
|
||||
#data_files=[('config', ['cfg/config.txt'])],
|
||||
cmdclass={'build_ext': BuildExt},
|
||||
zip_safe=False,
|
||||
)
|
||||
@@ -0,0 +1,156 @@
|
||||
// Config.cpp
|
||||
|
||||
#include "Config.h"
|
||||
|
||||
using namespace std;
|
||||
|
||||
|
||||
Config::Config( string filename, string delimiter,
|
||||
string comment )
|
||||
: m_Delimiter(delimiter), m_Comment(comment)
|
||||
{
|
||||
// Construct a Config, getting keys and values from given file
|
||||
|
||||
std::ifstream in( filename.c_str() );
|
||||
|
||||
if( !in ) throw File_not_found( filename );
|
||||
|
||||
in >> (*this);
|
||||
}
|
||||
|
||||
|
||||
Config::Config()
|
||||
: m_Delimiter( string(1,'=') ), m_Comment( string(1,'#') )
|
||||
{
|
||||
// Construct a Config without a file; empty
|
||||
}
|
||||
|
||||
|
||||
|
||||
bool Config::KeyExists( const string& key ) const
|
||||
{
|
||||
// Indicate whether key is found
|
||||
mapci p = m_Contents.find( key );
|
||||
return ( p != m_Contents.end() );
|
||||
}
|
||||
|
||||
|
||||
/* static */
|
||||
void Config::Trim( string& inout_s )
|
||||
{
|
||||
// Remove leading and trailing whitespace
|
||||
static const char whitespace[] = " \n\t\v\r\f";
|
||||
inout_s.erase( 0, inout_s.find_first_not_of(whitespace) );
|
||||
inout_s.erase( inout_s.find_last_not_of(whitespace) + 1U );
|
||||
}
|
||||
|
||||
|
||||
std::ostream& operator<<( std::ostream& os, const Config& cf )
|
||||
{
|
||||
// Save a Config to os
|
||||
for( Config::mapci p = cf.m_Contents.begin();
|
||||
p != cf.m_Contents.end();
|
||||
++p )
|
||||
{
|
||||
os << p->first << " " << cf.m_Delimiter << " ";
|
||||
os << p->second << std::endl;
|
||||
}
|
||||
return os;
|
||||
}
|
||||
|
||||
void Config::Remove( const string& key )
|
||||
{
|
||||
// Remove key and its value
|
||||
m_Contents.erase( m_Contents.find( key ) );
|
||||
return;
|
||||
}
|
||||
|
||||
std::istream& operator>>( std::istream& is, Config& cf )
|
||||
{
|
||||
// Load a Config from is
|
||||
// Read in keys and values, keeping internal whitespace
|
||||
typedef string::size_type pos;
|
||||
const string& delim = cf.m_Delimiter; // separator
|
||||
const string& comm = cf.m_Comment; // comment
|
||||
const pos skip = delim.length(); // length of separator
|
||||
|
||||
string nextline = ""; // might need to read ahead to see where value ends
|
||||
|
||||
while( is || nextline.length() > 0 )
|
||||
{
|
||||
// Read an entire line at a time
|
||||
string line;
|
||||
if( nextline.length() > 0 )
|
||||
{
|
||||
line = nextline; // we read ahead; use it now
|
||||
nextline = "";
|
||||
}
|
||||
else
|
||||
{
|
||||
std::getline( is, line );
|
||||
}
|
||||
|
||||
// Ignore comments
|
||||
line = line.substr( 0, line.find(comm) );
|
||||
|
||||
// Parse the line if it contains a delimiter
|
||||
pos delimPos = line.find( delim );
|
||||
if( delimPos < string::npos )
|
||||
{
|
||||
// Extract the key
|
||||
string key = line.substr( 0, delimPos );
|
||||
line.replace( 0, delimPos+skip, "" );
|
||||
|
||||
// See if value continues on the next line
|
||||
// Stop at blank line, next line with a key, end of stream,
|
||||
// or end of file sentry
|
||||
bool terminate = false;
|
||||
while( !terminate && is )
|
||||
{
|
||||
std::getline( is, nextline );
|
||||
terminate = true;
|
||||
|
||||
string nlcopy = nextline;
|
||||
Config::Trim(nlcopy);
|
||||
if( nlcopy == "" ) continue;
|
||||
|
||||
nextline = nextline.substr( 0, nextline.find(comm) );
|
||||
if( nextline.find(delim) != string::npos )
|
||||
continue;
|
||||
|
||||
nlcopy = nextline;
|
||||
Config::Trim(nlcopy);
|
||||
if( nlcopy != "" ) line += "\n";
|
||||
line += nextline;
|
||||
terminate = false;
|
||||
}
|
||||
|
||||
// Store key and value
|
||||
Config::Trim(key);
|
||||
Config::Trim(line);
|
||||
cf.m_Contents[key] = line; // overwrites if key is repeated
|
||||
}
|
||||
}
|
||||
|
||||
return is;
|
||||
}
|
||||
bool Config::FileExist(std::string filename)
|
||||
{
|
||||
bool exist= false;
|
||||
std::ifstream in( filename.c_str() );
|
||||
if( in )
|
||||
exist = true;
|
||||
return exist;
|
||||
}
|
||||
|
||||
void Config::ReadFile( string filename, string delimiter,
|
||||
string comment )
|
||||
{
|
||||
m_Delimiter = delimiter;
|
||||
m_Comment = comment;
|
||||
std::ifstream in( filename.c_str() );
|
||||
|
||||
if( !in ) throw File_not_found( filename );
|
||||
|
||||
in >> (*this);
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
#include <pybind11/pybind11.h>
|
||||
#include "BackendFactory.h"
|
||||
#include "BaseBackend.h"
|
||||
//#include "common.h"
|
||||
#include "utils.h"
|
||||
#include <string>
|
||||
|
||||
using namespace std;
|
||||
|
||||
bool warmup=true;
|
||||
|
||||
long deviceTime=0;
|
||||
|
||||
std::shared_ptr<BaseBackend> backend;
|
||||
Config * Config::instance = nullptr;
|
||||
|
||||
int backend_setconfig(char* cfg)
|
||||
{
|
||||
Config::setInstance(cfg);
|
||||
}
|
||||
|
||||
int backend_load(int backend_type,char* omModelPath,char* binfile)
|
||||
{
|
||||
backend = BackendFactory::Instance()->CreateBaseBackend(FrameworkType(backend_type));
|
||||
if(backend == nullptr)
|
||||
{
|
||||
ERROR_LOG("FAILED, Not found the test backend, type:%d.", FrameworkType(backend_type));
|
||||
return FAILED;
|
||||
}
|
||||
backend->init(omModelPath,binfile);
|
||||
printf("[INFO] AclBackend init OK\n");
|
||||
backend->load(omModelPath,binfile);
|
||||
printf("[INFO] AclBackend load OK\n");
|
||||
}
|
||||
|
||||
//py::array backend_predict(int type, char* omModelPath, py::array binfile)
|
||||
vector<py::array> backend_predict(int type, char* omModelPath, py::array binfile)
|
||||
{
|
||||
if(warmup)
|
||||
{
|
||||
printf("[INFO] start warmup AclBackend predict\n");
|
||||
//warmup=false;
|
||||
}
|
||||
INFO_LOG("start backend_predict is %d", Utils::getCurrentTime());
|
||||
std::vector<Output_buf> result_buf;
|
||||
//INFO_LOG("binfile.nbytes is %d", binfile.nbytes());
|
||||
deviceTime = 0;
|
||||
backend->predict(omModelPath, binfile.mutable_data(), binfile.nbytes(),result_buf, deviceTime);
|
||||
INFO_LOG("Pure device execute time is %f ms", deviceTime);
|
||||
if(warmup)
|
||||
{
|
||||
printf("[INFO] end warmup AclBackend predict\n");
|
||||
warmup=false;
|
||||
}
|
||||
|
||||
INFO_LOG("end backend_predict is %d", Utils::getCurrentTime());
|
||||
|
||||
vector<py::array> vec_result;
|
||||
for(int i =0 ; i<result_buf.size();i++)
|
||||
{
|
||||
std::string str;
|
||||
if(!result_buf[i].format.compare("uint8"))
|
||||
str=py::format_descriptor<uint8_t>::format();
|
||||
if(!result_buf[i].format.compare("int8"))
|
||||
str=py::format_descriptor<int8_t>::format();
|
||||
if(!result_buf[i].format.compare("float"))
|
||||
str=py::format_descriptor<float>::format();
|
||||
if(!result_buf[i].format.compare("float16"))
|
||||
str=py::format_descriptor<float16>::format();
|
||||
if(!result_buf[i].format.compare("int64"))
|
||||
str=py::format_descriptor<int64_t>::format();
|
||||
if(!result_buf[i].format.compare("uint64"))
|
||||
str=py::format_descriptor<uint64_t>::format();
|
||||
py::buffer_info tmp=py::buffer_info(
|
||||
result_buf[i].ptr,
|
||||
(ssize_t)result_buf[i].itemsize, //itemsize
|
||||
str,
|
||||
(ssize_t)result_buf[i].ndim,// ndim
|
||||
result_buf[i].shape, // shape
|
||||
result_buf[i].strides //strides
|
||||
);
|
||||
py::dtype dt = py::dtype(str);
|
||||
py::array result = py::array(dt,tmp.shape, tmp.strides, tmp.ptr);
|
||||
vec_result.push_back(result);
|
||||
}
|
||||
return vec_result;
|
||||
//return result;
|
||||
}
|
||||
|
||||
long backend_get_device_time()
|
||||
{
|
||||
return deviceTime;
|
||||
}
|
||||
|
||||
int backend_unload(int type,char* omModelPath,char* binfile)
|
||||
{
|
||||
backend->unload(omModelPath,binfile);
|
||||
printf("[INFO] AclBackend unload OK\n");
|
||||
}
|
||||
|
||||
namespace py = pybind11;
|
||||
|
||||
PYBIND11_MODULE(dnmetis_backend, m) {
|
||||
m.doc() = R"pbdoc(
|
||||
Pybind11 example plugin
|
||||
-----------------------
|
||||
|
||||
.. currentmodule:: dnmetis_backend
|
||||
|
||||
.. autosummary::
|
||||
:toctree: _generate
|
||||
|
||||
add
|
||||
subtract
|
||||
)pbdoc";
|
||||
/*m.def("backend_main", &backend_main, R"pbdoc(
|
||||
backend
|
||||
)pbdoc");*/
|
||||
m.def("backend_setconfig", &backend_setconfig, R"pbdoc(
|
||||
backend
|
||||
)pbdoc");
|
||||
|
||||
m.def("backend_load", &backend_load, R"pbdoc(
|
||||
backend
|
||||
)pbdoc");
|
||||
|
||||
m.def("backend_predict", &backend_predict, R"pbdoc(
|
||||
backend
|
||||
)pbdoc");
|
||||
|
||||
m.def("backend_get_device_time", &backend_get_device_time, R"pbdoc(
|
||||
backend
|
||||
)pbdoc");
|
||||
|
||||
m.def("backend_unload", &backend_unload, R"pbdoc(
|
||||
backend
|
||||
)pbdoc");
|
||||
m.def("add", [](int i, int j) { return i + j; }, R"pbdoc(
|
||||
add
|
||||
)pbdoc");
|
||||
m.def("subtract", [](int i, int j) { return i - j; }, R"pbdoc(
|
||||
subtract
|
||||
)pbdoc");
|
||||
|
||||
#ifdef VERSION_INFO
|
||||
m.attr("__version__") = VERSION_INFO;
|
||||
#else
|
||||
m.attr("__version__") = "dev";
|
||||
#endif
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
import dnmetis_backend as m
|
||||
|
||||
print(m.__version__)
|
||||
assert m.add(1, 2) == 3
|
||||
print("Test add Ok!")
|
||||
assert m.subtract(1, 2) == -1
|
||||
print("Test subtract Ok!")
|
||||
print("Test Ok!")
|
||||
@@ -0,0 +1,2 @@
|
||||
aclrtMemMallocPolicy=2
|
||||
backend_loglevel=3
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 107 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 137 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 120 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 83 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 127 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 148 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 162 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 105 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 112 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 139 KiB |
@@ -0,0 +1,10 @@
|
||||
ILSVRC2012_val_00000001.JPEG 65
|
||||
ILSVRC2012_val_00000002.JPEG 970
|
||||
ILSVRC2012_val_00000003.JPEG 230
|
||||
ILSVRC2012_val_00000004.JPEG 809
|
||||
ILSVRC2012_val_00000005.JPEG 516
|
||||
ILSVRC2012_val_00000006.JPEG 57
|
||||
ILSVRC2012_val_00000007.JPEG 334
|
||||
ILSVRC2012_val_00000008.JPEG 415
|
||||
ILSVRC2012_val_00000009.JPEG 674
|
||||
ILSVRC2012_val_00000010.JPEG 332
|
||||
+150
@@ -0,0 +1,150 @@
|
||||
import argparse
|
||||
import array
|
||||
import collections
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from queue import Queue
|
||||
#import env
|
||||
import cv2
|
||||
import numpy as np
|
||||
import re
|
||||
import pdb
|
||||
|
||||
# import converter.converter as converter
|
||||
#from backend.backend_acl import AclBackend
|
||||
|
||||
|
||||
|
||||
last_timing = []
|
||||
last_device_timing = []
|
||||
|
||||
|
||||
def get_args():
|
||||
"""Parse commandline."""
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--dataset_path", default="./datasets/imagenet_10", help="path to the dataset")
|
||||
parser.add_argument("--backend", default="acl", help="runtime to use")
|
||||
parser.add_argument("--model", required=True, help="model file path")
|
||||
parser.add_argument("--image_size",default='224,224,3',help="model inputs imagesize")
|
||||
parser.add_argument("--inputs", default="input:0", help="model inputs nodes eg: data1:0 ")
|
||||
parser.add_argument("--outputs",default="softmax:0", help="model outputs nodes list eg:fc1:0,fc2:0,fc3:0 ")
|
||||
|
||||
# below will override DNMetis rules compliant settings - don't use for official submission
|
||||
parser.add_argument("--count", default=1000, type=int, help="dataset items to infer")
|
||||
parser.add_argument("--precision", default="fp16", choices=["fp32", "fp16", "int8", "uint8"],
|
||||
help="precision mode, one of " + str(["fp32", "fp16", "int8", "uint8"]))
|
||||
parser.add_argument("--feed", default=[], help="feed")
|
||||
parser.add_argument("--image_list", default=[], help="image_list")
|
||||
parser.add_argument("--label_list", default=[], help="label_list")
|
||||
parser.add_argument("--cfg_path",default="./backend_cfg/built-in_config.txt")
|
||||
args = parser.parse_args()
|
||||
|
||||
# don't use defaults in argparser. Instead we default to a dict, override that with a profile
|
||||
# and take this as default unless command line give
|
||||
|
||||
|
||||
#if args.image_size is None:
|
||||
# args.image_size = SUPPORTED_DATASETS[args.dataset][3]['image_size']
|
||||
|
||||
if args.inputs:
|
||||
args.inputs = args.inputs.split(",")
|
||||
if args.outputs:
|
||||
args.outputs = args.outputs.split(",")
|
||||
if args.image_size:
|
||||
args.image_size = list(map(int, args.image_size.split(",")))
|
||||
|
||||
return args
|
||||
|
||||
def get_backend(backend):
|
||||
if backend == "acl":
|
||||
from backend.backend_acl import AclBackend
|
||||
backend = AclBackend()
|
||||
return backend
|
||||
|
||||
def resize_with_aspectratio(img, out_height, out_width, scale=87.5, inter_pol=cv2.INTER_LINEAR):
|
||||
height, width = img.shape[:2]
|
||||
new_height = int(100. * out_height / scale)
|
||||
new_width = int(100. * out_width / scale)
|
||||
if height > width:
|
||||
w = new_width
|
||||
h = int(new_height * height / width)
|
||||
else:
|
||||
h = new_height
|
||||
w = int(new_width * width / height)
|
||||
img = cv2.resize(img, (w, h), interpolation=inter_pol)
|
||||
return img
|
||||
|
||||
def center_crop(img, out_height, out_width):
|
||||
height, width = img.shape[:2]
|
||||
left = int((width - out_width) / 2)
|
||||
right = int((width + out_width) / 2)
|
||||
top = int((height - out_height) / 2)
|
||||
bottom = int((height + out_height) / 2)
|
||||
img = img[top:bottom, left:right]
|
||||
return img
|
||||
def pre_process_noisy(img, dims=None, precision="fp32"):
|
||||
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
|
||||
output_height, output_width, _ = dims
|
||||
cv2_interpol = cv2.INTER_CUBIC
|
||||
img = resize_with_aspectratio(img, output_height, output_width, inter_pol=cv2_interpol)
|
||||
img = center_crop(img, output_height, output_width)
|
||||
MEAN_RGB = [0.485 * 255, 0.456 * 255, 0.406 * 255]
|
||||
STDDEV_RGB = [0.229 * 255, 0.224 * 255, 0.225 * 255]
|
||||
|
||||
if precision=="fp32":
|
||||
img = np.asarray(img, dtype='float32')
|
||||
if precision=="fp16":
|
||||
img = np.asarray(img, dtype='float16')
|
||||
|
||||
means = np.array([0.485 * 255, 0.456 * 255, 0.406 * 255], dtype=np.float32)
|
||||
img -= means
|
||||
stddev = np.array([0.229 * 255, 0.224 * 255, 0.225 * 255], dtype=np.float32)
|
||||
img /= stddev
|
||||
return img
|
||||
|
||||
|
||||
|
||||
def preprocess_dataset(args,offset=0):
|
||||
with open(args.dataset_path + '/val_map.txt', 'r') as f:
|
||||
for s in f:
|
||||
image_name, label = re.split(r"\s+", s.strip())
|
||||
src = os.path.join(args.dataset_path, image_name)
|
||||
img_org = cv2.imread(src)
|
||||
processed_img = pre_process_noisy(img_org, dims=args.image_size, precision = args.precision)
|
||||
args.feed.append(processed_img)
|
||||
args.image_list.append(image_name)
|
||||
args.label_list.append(int(label)+offset)
|
||||
|
||||
|
||||
def main():
|
||||
good = 0
|
||||
total = 0
|
||||
#args
|
||||
args = get_args()
|
||||
|
||||
# find backend
|
||||
backend = get_backend(args.backend)
|
||||
|
||||
# load model to backend
|
||||
model = backend.load(args)
|
||||
#
|
||||
# preprocess_dataset
|
||||
preprocess_dataset(args,offset=0)
|
||||
|
||||
#start inference:
|
||||
for i in range(len(args.feed)):
|
||||
predictions = backend.predict(args.feed[i])
|
||||
#print(args.feed[i].shape)
|
||||
print('img_orig:',args.image_list[i],'label:',args.label_list[i],'predictions:',np.argmax(predictions),'\n')
|
||||
if args.label_list[i] == np.argmax(predictions):
|
||||
good +=1
|
||||
total +=1
|
||||
print('Predict total jpeg:',len(args.image_list),' Accuracy: ',good / total)
|
||||
|
||||
backend.unload()
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,3 @@
|
||||
pip3.7.5 install python-opencv
|
||||
cd backend_C++/dnmetis_backend
|
||||
pip3.7.5 setup.py install
|
||||
@@ -0,0 +1,4 @@
|
||||
python3.7 main.py --model=./model/efficientnet-b8.om --image_size='672,672,3' --precision=fp16
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
|
||||
version = '1.0'
|
||||
Reference in New Issue
Block a user