Made English README.MD to default

Merge pull request !16 from chengchunlei/N/A

README.md

@@ -1,34 +1,40 @@
-中文|[英文](README_EN.md)
+EN|[CH](README.md)
 
 # tools
 
-#### 介绍
+#### introduce
 
-Ascend tools，昇腾工具仓库。
+Ascend tools   
+**Please go to the corresponding folder to get the tool according to your needs, or click the link below to select the tool you need to use.**
 
-**请根据自己的需要进入对应文件夹获取工具，或者点击下面的说明链接选择需要的工具进行使用。**
-
-#### 使用说明
+#### explain
 
 1.  [msame](https://gitee.com/ascend/tools/tree/master/msame)
 
-    **模型推理工具**：输入.om模型和模型所需要的输入bin文件，输出模型的输出数据文件。
+    **Model reasoning tool**:Input. Om model and model required input bin file, output model output data file.
 
 2.  [img2bin](https://gitee.com/ascend/tools/tree/master/img2bin)
 
-    **bin文件生成工具**：生成模型推理所需的输入数据，以.bin格式保存。
+    **Bin file generation tool** : Generates input data required for model reasoning, saved in.bin format.
 
 3.  [makesd](https://gitee.com/ascend/tools/tree/master/makesd)
     
-    **制卡工具**：制卡工具包，提供ubuntu下制卡功能。
+    **makesd tool**：makesd tools package，Provide card making function under ubuntu.  
 
 4.  [faster_install](https://gitee.com/ascend/tools/tree/master/faster_install)
     
-    **快速安装脚本**：环境快速安装脚本。
+    **faster_install**：environment fast install script.  
 
 5.  [configure_usb_ethernet](https://gitee.com/ascend/tools/tree/master/configure_usb_ethernet)  
-     **USB虚拟网卡连接脚本**：配置USB网卡对应的IP地址。
+     **configure_usb_ethernet**：configuring the IP address of the USB NIC 。
     
 6. [pt2pb](https://gitee.com/ascend/tools/tree/master/pt2pb)  
 
-   **pytorch模型转tensorflow pb模型工具：**输入pytorch权重参数模型，转为onnx，再转为pb模型
+   **pytorch model transform to tensorflow pb model tool**：input pytorch weights parameters model，transform to onnx file，then transform to pb model
+
+7. [dnmetis](https://gitee.com/ascend/tools/tree/master/dnmetis)  
+
+   **Test tool for NPU inference precision and performance**：Using Python to encapsulate the C++ interface of ACL, inputting om model and original dataset images and tags, we can execute model inference and give out precision and performance of the om model
+
+   
+

README_CN.md

@@ -0,0 +1,38 @@
+中文|[英文](README_EN.md)
+
+# tools
+
+#### 介绍
+
+Ascend tools，昇腾工具仓库。
+
+**请根据自己的需要进入对应文件夹获取工具，或者点击下面的说明链接选择需要的工具进行使用。**
+
+#### 使用说明
+
+1.  [msame](https://gitee.com/ascend/tools/tree/master/msame)
+
+    **模型推理工具**：输入.om模型和模型所需要的输入bin文件，输出模型的输出数据文件。
+
+2.  [img2bin](https://gitee.com/ascend/tools/tree/master/img2bin)
+
+    **bin文件生成工具**：生成模型推理所需的输入数据，以.bin格式保存。
+
+3.  [makesd](https://gitee.com/ascend/tools/tree/master/makesd)
+    
+    **制卡工具**：制卡工具包，提供ubuntu下制卡功能。
+
+4.  [faster_install](https://gitee.com/ascend/tools/tree/master/faster_install)
+    
+    **快速安装脚本**：环境快速安装脚本。
+
+5.  [configure_usb_ethernet](https://gitee.com/ascend/tools/tree/master/configure_usb_ethernet)  
+     **USB虚拟网卡连接脚本**：配置USB网卡对应的IP地址。
+    
+6. [pt2pb](https://gitee.com/ascend/tools/tree/master/pt2pb)  
+
+   **pytorch模型转tensorflow pb模型工具**：输入pytorch权重参数模型，转为onnx，再转为pb模型
+
+7. [dnmetis](https://gitee.com/ascend/tools/tree/master/dnmetis)  
+
+   **NPU推理精度和性能测试工具**：使用Python封装ACL的C++接口，输入om模型和原始数据集图片、标签，即可执行模型推理，输出精度数据和性能数据

README_EN.md

@@ -1,36 +0,0 @@
-EN|[CH](README.md)
-
-# tools
-
-#### introduce
-
-Ascend tools   
-**Please go to the corresponding folder to get the tool according to your needs, or click the link below to select the tool you need to use.**
-
-#### explain
-
-1.  [msame](https://gitee.com/ascend/tools/tree/master/msame)
-
-    **Model reasoning tool**:Input. Om model and model required input bin file, output model output data file.
-
-2.  [img2bin](https://gitee.com/ascend/tools/tree/master/img2bin)
-
-    **Bin file generation tool** : Generates input data required for model reasoning, saved in.bin format.
-
-3.  [makesd](https://gitee.com/ascend/tools/tree/master/makesd)
-    
-    **makesd tool**：makesd tools package，Provide card making function under ubuntu.  
-
-4.  [faster_install](https://gitee.com/ascend/tools/tree/master/faster_install)
-    
-    **faster_install**：environment fast install script.  
-
-5.  [configure_usb_ethernet](https://gitee.com/ascend/tools/tree/master/configure_usb_ethernet)  
-     **configure_usb_ethernet**：configuring the IP address of the USB NIC 。
-    
-6. [pt2pb](https://gitee.com/ascend/tools/tree/master/pt2pb)  
-
-   **pytorch model transform to tensorflow pb model tool：**input pytorch weights parameters model，transform to onnx file，then transform to pb model
-
-   
-

dnmetis/README.md

@@ -0,0 +1,114 @@
+中文|[英文](README_EN.md)
+
+## 1.安装依赖：
+```
+pip3.7.5 install python-opencv
+cd backend_C++/dnmetis_backend
+pip3.7.5 setup.py install
+```
+安装dnmetis_backend的细节可以在backend_C++/dnmetis_backend/README.md看到，对于一个全新的Ai1推理环境，只需要安装一次依赖，不需要重复安装。
+
+## 2.下载om模型(.om)
+
+如下示例展示如果在NPU上运行efficientnet-b8模型:\
+1.下载efficientnet-b8 model(.om): \
+链接：[百度网盘](https://pan.baidu.com/s/1N-kpQoDe3NRxvjFKjAT9AA) \
+提取码：tvg0  
+下载的om模型放到model文件夹.
+
+原生的TensorFlow模型efficientnet-b8(.pb):\
+链接：[百度网盘](https://pan.baidu.com/s/1CajdSlNTh6k35RoyOn-3Ug)\
+提取码：slqm 
+
+如果想了解如果从pb模型如何转换成om模型，请下载efficientnet-b8.pb模型，使用ATC模型转换工具，或者执行转换命令：\
+.atc --model=MODELDIR/efficientnet−b8.pb −−framework=3 −−inputshape=′images:1,672,672,3′−−output=MODELDIR/efficientnet-b8 --mode=0 --out_nodes='Softmax:0' --soc_version=Ascend310 --input_fp16_nodes=images --output_type=FP16
+
+2.Imagenet-val数据集和标签:
+
+这里的示例仅仅展示了从Imagenet-val数据集挑选的10张图片：
+![输入图片说明](https://images.gitee.com/uploads/images/2020/0918/234302_a572d632_5418572.jpeg "无标题.jpg")
+
+
+
+## 3.执行推理:
+建议提交的PR代码统一使用run_inference.sh作为入口：\
+bash run_inference.sh
+
+执行日志:
+```
+[INFO]  start backend_predict is -1518493925
+[INFO]  start Execute is -1518490258
+[INFO]  model execute success
+[INFO]  end Execute is -1518350716
+[INFO]  npu compute cost 139.476000 ms
+[INFO]  1.output data success
+[INFO]  2.output data success
+[INFO]  execute sample success
+[INFO]  Pure device execute time is 0.000000 ms
+[INFO]  end backend_predict is -1518346882
+img_orig: ILSVRC2012_val_00000010.JPEG label: 332 predictions: 332
+
+Predict total jpeg: 10  Accuracy:  0.8
+```
+如上所示, "139.476 ms"是NPU的推理时间，"0.8" 是10张图片的top1精度。
+
+## 4.完整的5w张Imagenet2012-val数据集精度:
+
+![输入图片说明](https://images.gitee.com/uploads/images/2020/0919/010210_5cf496fc_5418572.png "屏幕截图.png")
+
+
+## 5.main.py修改点:
+
+如果需要使用你自己的模型来推理和计算精度，请修改main.py\
+只需要关心数据集、预处理和后处理代码部分：
+
+### 预处理:
+```
+def resize_with_aspectratio(img, out_height, out_width, scale=87.5, inter_pol=cv2.INTER_LINEAR):
+    height, width = img.shape[:2]
+    new_height = int(100. * out_height / scale)
+    new_width = int(100. * out_width / scale)
+    if height > width:
+        w = new_width
+        h = int(new_height * height / width)
+    else:
+        h = new_height
+        w = int(new_width * width / height)
+    img = cv2.resize(img, (w, h), interpolation=inter_pol)
+    return img
+
+def center_crop(img, out_height, out_width):
+    height, width = img.shape[:2]
+    left = int((width - out_width) / 2)
+    right = int((width + out_width) / 2)
+    top = int((height - out_height) / 2)
+    bottom = int((height + out_height) / 2)
+    img = img[top:bottom, left:right]
+    return img
+def pre_process_noisy(img, dims=None, precision="fp32"):
+    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
+    output_height, output_width, _ = dims
+    cv2_interpol = cv2.INTER_CUBIC
+    img = resize_with_aspectratio(img, output_height, output_width, inter_pol=cv2_interpol)
+    img = center_crop(img, output_height, output_width)
+    MEAN_RGB = [0.485 * 255, 0.456 * 255, 0.406 * 255]
+    STDDEV_RGB = [0.229 * 255, 0.224 * 255, 0.225 * 255]
+
+    if precision=="fp32":
+        img = np.asarray(img, dtype='float32')
+    if precision=="fp16":
+        img = np.asarray(img, dtype='float16')
+
+    means = np.array([0.485 * 255, 0.456 * 255, 0.406 * 255], dtype=np.float32)
+    img -= means
+    stddev = np.array([0.229 * 255, 0.224 * 255, 0.225 * 255], dtype=np.float32)
+    img /= stddev
+    return img
+```
+
+### 推理和后处理:
+```
+        predictions = backend.predict(args.feed[i])
+        #print(args.feed[i].shape)
+        print('img_orig:',args.image_list[i],'label:',args.label_list[i],'predictions:',np.argmax(predictions),'\n')
+```

dnmetis/README_EN.md

@@ -0,0 +1,114 @@
+EN|[CH](README.md)
+
+## 1.Install requirements：
+```
+pip3.7.5 install python-opencv
+cd backend_C++/dnmetis_backend
+pip3.7.5 setup.py install
+```
+Details of dnmetis_backend installation can be found in backend_C++/dnmetis_backend/README.md. Notice that, you just need to install requirements once for a brand new Ai1-Inference environment。
+
+## 2.Download model(.om)
+
+Here is an example of how to run npu inference of efficientnet-b8:\
+1.download  efficientnet-b8 model(.om): \
+URL：[baidu pan](https://pan.baidu.com/s/1N-kpQoDe3NRxvjFKjAT9AA) \
+Extracted code：tvg0  
+
+Original tensorflow model of efficientnet-b8(.pb):\
+URL：[baidu pan](https://pan.baidu.com/s/1CajdSlNTh6k35RoyOn-3Ug)\
+Extracted code：slqm 
+
+If you want to acknowledge how to generate om from pb，pls download efficientnet-b8.pb and execute ATC cmd：\
+.atc --model=MODELDIR/efficientnet−b8.pb −−framework=3 −−inputshape=′images:1,672,672,3′−−output=MODELDIR/efficientnet-b8 --mode=0 --out_nodes='Softmax:0' --soc_version=Ascend310 --input_fp16_nodes=images --output_type=FP16
+
+2.Imagenet-val dataset and labels in val_map.txt:
+
+Here is an example of 10 pictures of Imagenet-val dataset：
+![输入图片说明](https://images.gitee.com/uploads/images/2020/0918/234302_a572d632_5418572.jpeg "无标题.jpg")
+
+
+
+## 3.Start execute the inference:
+
+bash run_inference.sh
+
+Log:
+```
+[INFO]  start backend_predict is -1518493925
+[INFO]  start Execute is -1518490258
+[INFO]  model execute success
+[INFO]  end Execute is -1518350716
+[INFO]  npu compute cost 139.476000 ms
+[INFO]  1.output data success
+[INFO]  2.output data success
+[INFO]  execute sample success
+[INFO]  Pure device execute time is 0.000000 ms
+[INFO]  end backend_predict is -1518346882
+img_orig: ILSVRC2012_val_00000010.JPEG label: 332 predictions: 332
+
+Predict total jpeg: 10  Accuracy:  0.8
+```
+As you seen, "139.47 ms" is the npu inference time，"0.8" is the top1 Accuracy of 10 pictures。
+
+## 4.Top1 Accuracy of entire Imagenet2012-val Datasets(5w pictures):
+
+![输入图片说明](https://images.gitee.com/uploads/images/2020/0919/010210_5cf496fc_5418572.png "屏幕截图.png")
+
+
+## 5.modify main.py for your own model:
+
+Only need to concern about the dataset，pre-process，post-process：
+
+### pre-process:
+```
+def resize_with_aspectratio(img, out_height, out_width, scale=87.5, inter_pol=cv2.INTER_LINEAR):
+    height, width = img.shape[:2]
+    new_height = int(100. * out_height / scale)
+    new_width = int(100. * out_width / scale)
+    if height > width:
+        w = new_width
+        h = int(new_height * height / width)
+    else:
+        h = new_height
+        w = int(new_width * width / height)
+    img = cv2.resize(img, (w, h), interpolation=inter_pol)
+    return img
+
+def center_crop(img, out_height, out_width):
+    height, width = img.shape[:2]
+    left = int((width - out_width) / 2)
+    right = int((width + out_width) / 2)
+    top = int((height - out_height) / 2)
+    bottom = int((height + out_height) / 2)
+    img = img[top:bottom, left:right]
+    return img
+def pre_process_noisy(img, dims=None, precision="fp32"):
+    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
+    output_height, output_width, _ = dims
+    cv2_interpol = cv2.INTER_CUBIC
+    img = resize_with_aspectratio(img, output_height, output_width, inter_pol=cv2_interpol)
+    img = center_crop(img, output_height, output_width)
+    MEAN_RGB = [0.485 * 255, 0.456 * 255, 0.406 * 255]
+    STDDEV_RGB = [0.229 * 255, 0.224 * 255, 0.225 * 255]
+
+    if precision=="fp32":
+        img = np.asarray(img, dtype='float32')
+    if precision=="fp16":
+        img = np.asarray(img, dtype='float16')
+
+    means = np.array([0.485 * 255, 0.456 * 255, 0.406 * 255], dtype=np.float32)
+    img -= means
+    stddev = np.array([0.229 * 255, 0.224 * 255, 0.225 * 255], dtype=np.float32)
+    img /= stddev
+    return img
+```
+
+### inference and post-process:
+```
+        predictions = backend.predict(args.feed[i])
+        #print(args.feed[i].shape)
+        print('img_orig:',args.image_list[i],'label:',args.label_list[i],'predictions:',np.argmax(predictions),'\n')
+```
+
+

dnmetis/backend/backend.py

@@ -0,0 +1,26 @@
+"""
+abstract backend class
+"""
+
+
+# pylint: disable=unused-argument,missing-docstring
+
+class Backend():
+    def __init__(self):
+        self.inputs = []
+        self.outputs = []
+
+    def version(self):
+        raise NotImplementedError("Backend:version")
+
+    def name(self):
+        raise NotImplementedError("Backend:name")
+
+    def load(self, args):
+        raise NotImplementedError("Backend:load")
+
+    def predict(self, feed):
+        raise NotImplementedError("Backend:predict")
+
+    def get_predict_time(self):
+        return 0

dnmetis/backend/backend_acl.py

@@ -0,0 +1,64 @@
+"""
+acl backend
+"""
+
+
+import dnmetis_backend as dnmetis_backend
+import backend.backend as backend
+import numpy as np
+import os
+import pdb
+
+class AclBackend(backend.Backend):
+    def __init__(self):
+        super(AclBackend, self).__init__()
+        self.ACL=5
+        self.outputs = ""
+        self.inputs = ""
+        self.model_path = ""
+        self.cfg_path = ""
+
+    def version(self):
+        return "1.0"
+
+    def name(self):
+        return "AclBackend"
+
+    def image_format(self):
+        # By default tensorflow uses NHWC (and the cpu implementation only does NHWC)
+        return "NHWC"
+
+    def load(self, args):
+        # there is no input/output meta data i the graph so it need to come from config.
+        if not args.inputs:
+            raise ValueError("AclBackend needs inputs")
+        if not args.outputs:
+            raise ValueError("AclBackend needs outputs")
+        self.outputs = args.outputs
+        self.inputs = args.inputs
+        self.model_path = args.model
+        self.cfg_path = args.cfg_path
+        #s.path.join(args.pwd, 'backend_cfg/built-in_config.txt')
+        dnmetis_backend.backend_setconfig(self.cfg_path)
+        dnmetis_backend.backend_load(self.ACL,self.model_path,"")
+        return self
+
+    def predict(self, feed):
+        #fed=feed[self.inputs[0]]
+        result_list=[]
+        result = dnmetis_backend.backend_predict(self.ACL,self.model_path,feed)
+
+        for _ in range(len(self.outputs)):
+            #resnet50 tf & caffe
+            if 'softmax_tensor' in self.outputs[_] or 'prob' in self.outputs[_]:
+                result_list.append(np.argmax(result[_],1))
+            # resnet50 tf
+            if 'ArgMax' in self.outputs[_]:
+                result_list.append(result[_])
+        if result_list == []:
+            # ssd-resnet34 tf
+            result_list = result
+        return result_list
+
+    def unload(self):
+        return dnmetis_backend.backend_unload(self.ACL,self.model_path,"")

dnmetis/backend_C++/dnmetis_backend/LICENSE

@@ -0,0 +1,36 @@
+Copyright (c) 2016 The Pybind Development Team, All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors
+   may be used to endorse or promote products derived from this software
+   without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+You are under no obligation whatsoever to provide any bug fixes, patches, or
+upgrades to the features, functionality or performance of the source code
+("Enhancements") to anyone; however, if you choose to make your Enhancements
+available either publicly, or directly to the author of this software, without
+imposing a separate written license agreement for such Enhancements, then you
+hereby grant the following license: a non-exclusive, royalty-free perpetual
+license to install, use, modify, prepare derivative works, incorporate into
+other computer software, distribute, and sublicense such enhancements or
+derivative works thereof, in binary and source code form.

dnmetis/backend_C++/dnmetis_backend/README.md

@@ -0,0 +1,87 @@
+## dnmetis_backend
+
+  It contains one AclBackend(C++), It can be called by DNMetis when tester tests NPU perf&accuracy
+  
+  The third-party C++ Backends can be easily added 
+
+1.install dnmetis_backend
+
+    python3.7.5 setup.py  install
+
+2.Install log:
+
+/usr/local/python3.7.5/lib/python3.7/site-packages/setuptools/dist.py:474: UserWarning: Normalizing 'V1.0.2' to '1.0.2'
+  normalized_version,
+running install
+running bdist_egg
+running egg_info
+creating dnmetis_backend.egg-info
+writing dnmetis_backend.egg-info/PKG-INFO
+writing dependency_links to dnmetis_backend.egg-info/dependency_links.txt
+writing top-level names to dnmetis_backend.egg-info/top_level.txt
+writing manifest file 'dnmetis_backend.egg-info/SOURCES.txt'
+reading manifest file 'dnmetis_backend.egg-info/SOURCES.txt'
+writing manifest file 'dnmetis_backend.egg-info/SOURCES.txt'
+installing library code to build/bdist.linux-x86_64/egg
+running install_lib
+running build_ext
+creating tmp
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/python3.7.5/include/python3.7m -c /tmp/tmp4o2xo183.cpp -o tmp/tmp4o2xo183.o -std=c++14
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/python3.7.5/include/python3.7m -c /tmp/tmptazg09at.cpp -o tmp/tmptazg09at.o -fvisibility=hidden
+building 'dnmetis_backend' extension
+creating build
+creating build/temp.linux-x86_64-3.7
+creating build/temp.linux-x86_64-3.7/src
+creating build/temp.linux-x86_64-3.7/backend
+creating build/temp.linux-x86_64-3.7/backend/built-in
+creating build/temp.linux-x86_64-3.7/backend/built-in/src
+creating build/temp.linux-x86_64-3.7/backend/custom
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c src/main.cpp -o build/temp.linux-x86_64-3.7/src/main.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c src/Config.cpp -o build/temp.linux-x86_64-3.7/src/Config.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/BaseBackend.cpp -o build/temp.linux-x86_64-3.7/backend/BaseBackend.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/BackendFactory.cpp -o build/temp.linux-x86_64-3.7/backend/BackendFactory.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/built-in/aclbackend.cpp -o build/temp.linux-x86_64-3.7/backend/built-in/aclbackend.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/built-in/src/model_process.cpp -o build/temp.linux-x86_64-3.7/backend/built-in/src/model_process.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/built-in/src/sample_process.cpp -o build/temp.linux-x86_64-3.7/backend/built-in/src/sample_process.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/built-in/src/utils.cpp -o build/temp.linux-x86_64-3.7/backend/built-in/src/utils.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DVERSION_INFO="1.0.2" -I/home/c00506053/dnmetis/backend_C++/dnmetis_backend/.eggs/pybind11-2.5.0-py3.7.egg/pybind11/include -I./inc/ -I./backend/inc -I./backend/built-in -I./backend/built-in/inc -I./backend/custom -I./backend/custom/inc -I/usr/local/Ascend/acllib/include/ -I/usr/local/python3.7.5/include/python3.7m -c backend/custom/trtbackend.cpp -o build/temp.linux-x86_64-3.7/backend/custom/trtbackend.o -w -O0 -fpermissive -std=c++14 -fvisibility=hidden
+creating build/lib.linux-x86_64-3.7
+g++ -pthread -shared build/temp.linux-x86_64-3.7/src/main.o build/temp.linux-x86_64-3.7/src/Config.o build/temp.linux-x86_64-3.7/backend/BaseBackend.o build/temp.linux-x86_64-3.7/backend/BackendFactory.o build/temp.linux-x86_64-3.7/backend/built-in/aclbackend.o build/temp.linux-x86_64-3.7/backend/built-in/src/model_process.o build/temp.linux-x86_64-3.7/backend/built-in/src/sample_process.o build/temp.linux-x86_64-3.7/backend/built-in/src/utils.o build/temp.linux-x86_64-3.7/backend/custom/trtbackend.o -L/usr/local/Ascend/acllib/lib64/ -L/usr/local/python3.7.5/lib -lascendcl -lpython3.7m -o build/lib.linux-x86_64-3.7/dnmetis_backend.cpython-37m-x86_64-linux-gnu.so -O0
+creating build/bdist.linux-x86_64
+creating build/bdist.linux-x86_64/egg
+copying build/lib.linux-x86_64-3.7/dnmetis_backend.cpython-37m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
+creating stub loader for dnmetis_backend.cpython-37m-x86_64-linux-gnu.so
+byte-compiling build/bdist.linux-x86_64/egg/dnmetis_backend.py to dnmetis_backend.cpython-37.pyc
+creating build/bdist.linux-x86_64/egg/EGG-INFO
+copying dnmetis_backend.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
+copying dnmetis_backend.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
+copying dnmetis_backend.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
+copying dnmetis_backend.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
+copying dnmetis_backend.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
+writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
+creating dist
+creating 'dist/dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
+removing 'build/bdist.linux-x86_64/egg' (and everything under it)
+Processing dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg
+removing '/usr/local/python3.7.5/lib/python3.7/site-packages/dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg' (and everything under it)
+creating /usr/local/python3.7.5/lib/python3.7/site-packages/dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg
+Extracting dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg to /usr/local/python3.7.5/lib/python3.7/site-packages
+dnmetis-backend 1.0.2 is already the active version in easy-install.pth
+
+Installed /usr/local/python3.7.5/lib/python3.7/site-packages/dnmetis_backend-1.0.2-py3.7-linux-x86_64.egg
+Processing dependencies for dnmetis-backend==1.0.2
+Finished processing dependencies for dnmetis-backend==1.0.2
+
+
+3、Check result of installation:\
+"dnmetis-backend" will be seen
+````pip3.7.5 list                                                                                             
+Packag      Version
+--------------- ---------
+attrs           20.2.0
+certifi         2020.6.20
+cffi            1.14.2
+chardet         3.0.4
+decorator       4.4.2
+dnmetis-backend 1.0.2
+grpcio          1.31.0

dnmetis/backend_C++/dnmetis_backend/backend/BackendFactory.cpp

@@ -0,0 +1,61 @@
+/**
+ * Copyright 2020 Huawei Technologies Co., Ltd
+
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+
+ * http://www.apache.org/licenses/LICENSE-2.0
+
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#include "BackendFactory.h"
+
+BackendFactory *BackendFactory::Instance() {
+  static BackendFactory instance;
+  return &instance;
+}
+
+int BackendFactory::Init() {
+  if (!is_init_) {
+    std::string skt_bin = "libaclbackend.so";
+    //handle_ = dlopen(skt_bin.c_str(), RTLD_NOW | RTLD_GLOBAL);
+    //if (handle_ == nullptr) {
+    //  GELOGE(FAILED, "SKT: open skt lib failed, please check LD_LIBRARY_PATH.");
+    //}
+  }
+  is_init_ = true;
+
+  return 0;
+}
+
+
+std::shared_ptr<BaseBackend> BackendFactory::CreateBaseBackend(const FrameworkType type) {
+  std::map<FrameworkType, BASE_BACKEND_CREATOR_FUN>::iterator iter = creator_map_.find(type);
+  if (iter != creator_map_.end()) {
+    return iter->second();
+  }
+
+  return nullptr;
+}
+
+void BackendFactory::RegisterCreator(const FrameworkType type, BASE_BACKEND_CREATOR_FUN fun) {
+  std::map<FrameworkType, BASE_BACKEND_CREATOR_FUN>::iterator iter = creator_map_.find(type);
+  if (iter != creator_map_.end()) {
+    return;
+  }
+
+  creator_map_[type] = fun;
+}
+
+BackendFactory::~BackendFactory() {
+  creator_map_.clear();
+}
+
+
+

dnmetis/backend_C++/dnmetis_backend/backend/BaseBackend.cpp

@@ -0,0 +1,49 @@
+/**
+ * Copyright 2020 Huawei Technologies Co., Ltd
+
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+
+ * http://www.apache.org/licenses/LICENSE-2.0
+
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#include "BaseBackend.h"
+
+
+
+int BaseBackend::param()
+{
+  return 0;
+}
+
+int BaseBackend::runner()
+{
+  return 0;
+}
+
+int BaseBackend::runnerthread()
+{
+  return 0;
+}
+
+int BaseBackend::statistic()
+{
+  return 0;
+}
+
+
+
+
+
+
+
+
+
+

dnmetis/backend_C++/dnmetis_backend/backend/built-in/aclbackend.cpp

@@ -0,0 +1,46 @@
+#include "aclbackend.h"
+
+
+CREATE_BACKEND_INIT(AclBackend,model,data)
+{
+    Result ret = processSample.InitResource(model);
+    if (ret != SUCCESS) {
+        ERROR_LOG("sample init resource failed");
+        return FAILED;
+    }
+    return SUCCESS;   
+}
+
+CREATE_BACKEND_LOAD(AclBackend,model,data)
+{
+    INFO_LOG("AclBackend LOAD model success");
+    return SUCCESS;
+}
+
+CREATE_BACKEND_PREDICT(AclBackend,model,data,len,output,npuTime)
+{
+    Result ret = processSample.Process(data,len,output,npuTime);
+    if (ret != SUCCESS) {
+        ERROR_LOG("sample process failed");
+        return FAILED;
+    }
+    INFO_LOG("execute sample success");
+    return SUCCESS;
+}
+
+CREATE_BACKEND_UNLOAD(AclBackend,model,data)
+{
+     Result ret = processSample.Unload();
+    if (ret != SUCCESS) {
+        ERROR_LOG("sample unload failed");
+        return FAILED;
+    }
+    INFO_LOG("AclBackend UNLOAD model success");
+    return SUCCESS;
+}
+
+
+
+REGISTER_BASE_BACKEND_CREATOR(ACL, AclBackend)
+
+

dnmetis/backend_C++/dnmetis_backend/backend/built-in/aclbackend.h

@@ -0,0 +1,8 @@
+#include "BackendFactory.h"
+#include "sample_process.h"
+#include <vector>
+using namespace std;
+
+CREATE_BACKEND(AclBackend)
+
+SampleProcess processSample;

dnmetis/backend_C++/dnmetis_backend/backend/built-in/inc/model_process.h

@@ -0,0 +1,113 @@
+/**
+* @file model_process.h
+*
+* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+*/
+#pragma once
+#include <iostream>
+#include "utils.h"
+#include "acl/acl.h"
+#include<vector>
+using namespace std;
+
+/**
+* ModelProcess
+*/
+class ModelProcess {
+public:
+    /**
+    * @brief Constructor
+    */
+    ModelProcess();
+
+    /**
+    * @brief Destructor
+    */
+    ~ModelProcess();
+
+    /**
+    * @brief load model from file with mem
+    * @param [in] modelPath: model path
+    * @return result
+    */
+    Result LoadModelFromFileWithMem(const char *modelPath);
+
+    /**
+    * @brief unload model
+    */
+    void Unload();
+
+    /**
+    * @brief create model desc
+    * @return result
+    */
+    Result CreateDesc();
+
+    /**
+    * @brief destroy desc
+    */
+    void DestroyDesc();
+
+    aclmdlDesc *GetDesc(){return modelDesc_;};
+
+    /**
+    * @brief create model input
+    * @param [in] inputDataBuffer: input buffer
+    * @param [in] bufferSize: input buffer size
+    * @return result
+    */
+    Result CreateInput(void *inputDataBuffer, size_t bufferSize);
+
+    /**
+    * @brief destroy input resource
+    */
+    void DestroyInput();
+
+    /**
+    * @brief create output buffer
+    * @return result
+    */
+    Result CreateOutput();
+
+    /**
+    * @brief destroy output resource
+    */
+    void DestroyOutput();
+
+    //void * GetOutput(){return outputdata;};
+
+    /**
+    * @brief model execute
+    * @return result
+    */
+    Result Execute();
+
+    /**
+    * @brief dump model output result to file
+    */
+    void DumpModelOutputResult();
+
+    /**
+    * @brief get model output result
+    */
+    void OutputModelResult();
+
+    void OutputModelResult(vector<Output_buf> &output);
+
+public:
+    uint32_t modelId_;
+    size_t modelMemSize_;
+    size_t modelWeightSize_;
+    void *modelMemPtr_;
+    void *modelWeightPtr_;
+    bool loadFlag_;  // model load flag
+    aclmdlDesc *modelDesc_;
+    aclmdlDataset *input_;
+    aclmdlDataset *output_;
+    vector<void *> outputdata;
+};
+

dnmetis/backend_C++/dnmetis_backend/backend/built-in/inc/sample_process.h

@@ -0,0 +1,62 @@
+/**
+* @file sample_process.h
+*
+* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+*/
+#pragma once
+#include "utils.h"
+#include "acl/acl.h"
+#include "model_process.h"
+#include<vector>
+using namespace std;
+
+/**
+* SampleProcess
+*/
+class SampleProcess {
+public:
+    /**
+    * @brief Constructor
+    */
+    SampleProcess();
+
+    /**
+    * @brief Destructor
+    */
+    ~SampleProcess();
+
+    /**
+    * @brief init reousce
+    * @return result
+    */
+    Result InitResource(char* omModelPath);
+
+    /**
+    * @brief sample process
+    * @return result
+    */
+    //Result Process(char* binfile);
+
+    Result Process(void* binfile,int len);
+
+    Result Process(void* binfile,int len, vector<Output_buf> &output, long &npuTime);
+
+
+    Result Unload();
+
+    //ModelProcess GetModelProcess();
+    ModelProcess GetModelProcess(){return processModel;};
+
+public:
+    void DestroyResource();
+
+    int32_t deviceId_;
+    aclrtContext context_;
+    aclrtStream stream_;
+    ModelProcess processModel;
+};
+

dnmetis/backend_C++/dnmetis_backend/backend/built-in/inc/utils.h

@@ -0,0 +1,69 @@
+/**
+* @file utils.h
+*
+* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+*/
+#pragma once
+#include <iostream>
+#include "common.h"
+#include <map>
+#include "acl/acl_base.h"
+//#include "Config.h"
+/**
+* Utils
+*/
+
+struct aclmdlTensorDesc {
+    aclmdlTensorDesc() : name(""), size(0), format(ACL_FORMAT_UNDEFINED), dataType(ACL_DT_UNDEFINED) {}
+    std::string name;
+    size_t size;
+    aclFormat format;
+    aclDataType dataType;
+    std::vector<int64_t> dims;
+};
+
+struct aclmdlDesc {
+    void Clear()
+    {
+        inputDesc.clear();
+        outputDesc.clear();
+        dynamicBatch.clear();
+        dynamicHW.clear();
+    }
+    std::vector<aclmdlTensorDesc> inputDesc;
+    std::vector<aclmdlTensorDesc> outputDesc;
+    std::vector<uint64_t> dynamicBatch;
+    std::vector<std::vector<uint64_t>> dynamicHW;
+};
+
+
+
+class Utils {
+public:
+    /**
+    * @brief create device buffer of file
+    * @param [in] fileName: file name
+    * @param [out] fileSize: size of file
+    * @return device buffer of file
+    */
+    static void *GetDeviceBufferOfFile(std::string fileName, uint32_t &fileSize);
+
+    /**
+    * @brief create buffer of file
+    * @param [in] fileName: file name
+    * @param [out] fileSize: size of file
+    * @return buffer of pic
+    */
+    static void* ReadBinFile(std::string fileName, uint32_t& fileSize);
+
+    static void* GetDeviceBufferOfptr(void* fileName, uint32_t fileSize);
+
+    static long getCurrentTime();
+
+};
+
+#pragma once

dnmetis/backend_C++/dnmetis_backend/backend/built-in/src/model_process.cpp

@@ -0,0 +1,444 @@
+/**
+* @file model_process.cpp
+*
+* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+*/
+#include "model_process.h"
+#include <iostream>
+#include <map>
+#include <sstream>
+#include <algorithm>
+#include "utils.h"
+using namespace std;
+extern bool g_isDevice;
+//extern Config configSettings;
+extern std::map<aclDataType, std::string> ACLdt;
+extern std::map<aclDataType, int> ACLdt_size;
+
+ModelProcess::ModelProcess() :modelId_(0), modelMemSize_(0), modelWeightSize_(0), modelMemPtr_(nullptr),
+modelWeightPtr_(nullptr), loadFlag_(false), modelDesc_(nullptr), input_(nullptr), output_(nullptr)
+{
+}
+
+ModelProcess::~ModelProcess()
+{
+}
+
+
+
+Result ModelProcess::LoadModelFromFileWithMem(const char *modelPath)
+{
+    if (loadFlag_) {
+        ERROR_LOG("has already loaded a model");
+        return FAILED;
+    }
+
+    aclError ret = aclmdlQuerySize(modelPath, &modelMemSize_, &modelWeightSize_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("query model failed, model file is %s", modelPath);
+        return FAILED;
+    }
+    //int mem_malloc_type=configSettings.Read("aclrtMemMallocPolicy", 0);
+    ret = aclrtMalloc(&modelMemPtr_, modelMemSize_, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("malloc buffer for mem failed, require size is %zu", modelMemSize_);
+        return FAILED;
+    }
+
+    ret = aclrtMalloc(&modelWeightPtr_, modelWeightSize_, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("malloc buffer for weight failed, require size is %zu", modelWeightSize_);
+        return FAILED;
+    }
+
+    ret = aclmdlLoadFromFileWithMem(modelPath, &modelId_, modelMemPtr_,
+        modelMemSize_, modelWeightPtr_, modelWeightSize_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("load model from file failed, model file is %s", modelPath);
+        return FAILED;
+    }
+
+    loadFlag_ = true;
+    INFO_LOG("load model %s success", modelPath);
+    return SUCCESS;
+}
+
+Result ModelProcess::CreateDesc()
+{
+    modelDesc_ = aclmdlCreateDesc();
+    if (modelDesc_ == nullptr) {
+        ERROR_LOG("create model description failed");
+        return FAILED;
+    }
+
+    aclError ret = aclmdlGetDesc(modelDesc_, modelId_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("get model description failed");
+        return FAILED;
+    }
+
+    INFO_LOG("create model description success");
+
+    return SUCCESS;
+}
+
+void ModelProcess::DestroyDesc()
+{
+    if (modelDesc_ != nullptr) {
+        (void)aclmdlDestroyDesc(modelDesc_);
+        modelDesc_ = nullptr;
+    }
+}
+
+
+Result ModelProcess::CreateInput(void *inputDataBuffer, size_t bufferSize)
+{
+    input_ = aclmdlCreateDataset();
+    if (input_ == nullptr) {
+        ERROR_LOG("can't create dataset, create input failed");
+        return FAILED;
+    }
+    //INFO_LOG("start CreateInput::aclCreateDataBuffer is %d", Utils::getCurrentTime());
+    aclDataBuffer* inputData = aclCreateDataBuffer(inputDataBuffer, bufferSize);
+    //INFO_LOG("end CreateInput::aclCreateDataBuffer is %d", Utils::getCurrentTime());
+    if (inputData == nullptr) {
+        ERROR_LOG("can't create data buffer, create input failed");
+        return FAILED;
+    }
+    //INFO_LOG("start CreateInput::aclmdlAddDatasetBuffer is %d", Utils::getCurrentTime());
+    aclError ret = aclmdlAddDatasetBuffer(input_, inputData);
+    //INFO_LOG("end CreateInput::aclmdlAddDatasetBuffer is %d", Utils::getCurrentTime());
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("add input dataset buffer failed");
+        aclDestroyDataBuffer(inputData);
+        inputData = nullptr;
+        return FAILED;
+    }
+
+    return SUCCESS;
+}
+
+void ModelProcess::DestroyInput()
+{
+    if (input_ == nullptr) {
+        return;
+    }
+
+    for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(input_); ++i) {
+        aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(input_, i);
+        aclDestroyDataBuffer(dataBuffer);
+    }
+    aclmdlDestroyDataset(input_);
+    input_ = nullptr;
+}
+
+Result ModelProcess::CreateOutput()
+{
+    if (modelDesc_ == nullptr) {
+        ERROR_LOG("no model description, create ouput failed");
+        return FAILED;
+    }
+
+    output_ = aclmdlCreateDataset();
+    if (output_ == nullptr) {
+        ERROR_LOG("can't create dataset, create output failed");
+        return FAILED;
+    }
+
+    size_t outputSize = aclmdlGetNumOutputs(modelDesc_);
+    for (size_t i = 0; i < outputSize; ++i) {
+        size_t buffer_size = aclmdlGetOutputSizeByIndex(modelDesc_, i);
+
+        void *outputBuffer = nullptr;
+        aclError ret = aclrtMalloc(&outputBuffer, buffer_size, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("can't malloc buffer, size is %zu, create output failed", buffer_size);
+            return FAILED;
+        }
+
+        aclDataBuffer* outputData = aclCreateDataBuffer(outputBuffer, buffer_size);
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("can't create data buffer, create output failed");
+            aclrtFree(outputBuffer);
+            return FAILED;
+        }
+
+        ret = aclmdlAddDatasetBuffer(output_, outputData);
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("can't add data buffer, create output failed");
+            aclrtFree(outputBuffer);
+            aclDestroyDataBuffer(outputData);
+            return FAILED;
+        }
+    }
+
+    INFO_LOG("create model output success");
+    return SUCCESS;
+}
+
+void ModelProcess::DumpModelOutputResult()
+{
+    stringstream ss;
+    size_t outputNum = aclmdlGetDatasetNumBuffers(output_);
+    static int executeNum = 0;
+    for (size_t i = 0; i < outputNum; ++i) {
+        ss << "output" << ++executeNum << "_" << i << ".bin";
+        string outputFileName = ss.str();
+        FILE *outputFile = fopen(outputFileName.c_str(), "wb");
+        if (outputFile) {
+            aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(output_, i);
+            void* data = aclGetDataBufferAddr(dataBuffer);
+            uint32_t len = aclGetDataBufferSize(dataBuffer);
+
+            void* outHostData = NULL;
+            aclError ret = ACL_ERROR_NONE;
+            if (!g_isDevice) {
+                ret = aclrtMallocHost(&outHostData, len);
+                if (ret != ACL_ERROR_NONE) {
+                    ERROR_LOG("aclrtMallocHost failed, ret[%d]", ret);
+                    return;
+                }
+
+                ret = aclrtMemcpy(outHostData, len, data, len, ACL_MEMCPY_DEVICE_TO_HOST);
+                if (ret != ACL_ERROR_NONE) {
+                    ERROR_LOG("aclrtMemcpy failed, ret[%d]", ret);
+                    (void)aclrtFreeHost(outHostData);
+                    return;
+                }
+
+                fwrite(outHostData, len, sizeof(char), outputFile);
+
+                ret = aclrtFreeHost(outHostData);
+                if (ret != ACL_ERROR_NONE) {
+                    ERROR_LOG("aclrtFreeHost failed, ret[%d]", ret);
+                    return;
+                }
+            } else {
+                fwrite(data, len, sizeof(char), outputFile);
+            }
+            fclose(outputFile);
+            outputFile = nullptr;
+        } else {
+            ERROR_LOG("create output file [%s] failed", outputFileName.c_str());
+            return;
+        }
+    }
+
+    INFO_LOG("dump data success");
+    return;
+}
+
+void ModelProcess::OutputModelResult()
+{
+
+    aclError ret = ACL_ERROR_NONE;
+
+    if(outputdata.size()!=0)
+    {
+        for(auto it=outputdata.begin();it!=outputdata.end();++it){
+                ret = aclrtFreeHost(*it);
+            }
+            outputdata.clear();
+    }
+    for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(output_); ++i) {
+        aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(output_, i);
+        void* data = aclGetDataBufferAddr(dataBuffer);
+        uint32_t len = aclGetDataBufferSize(dataBuffer);
+        //INFO_LOG("OutputModelResult::len %d", len);
+        //void *outHostData = NULL;
+        outputdata.push_back(NULL);
+         ret = ACL_ERROR_NONE;
+        float *outData = NULL;
+        if (!g_isDevice) {
+            //INFO_LOG("start OutputModelResult::aclrtMallocHost is %d", Utils::getCurrentTime());
+            aclError ret = aclrtMallocHost(&outputdata[i], len);
+            //INFO_LOG("end OutputModelResult::aclrtMallocHost is %d", Utils::getCurrentTime());
+            if (ret != ACL_ERROR_NONE) {
+                ERROR_LOG("aclrtMallocHost failed, ret[%d]", ret);
+                return;
+            }
+            //INFO_LOG("start OutputModelResult::aclrtMemcpy is %d", Utils::getCurrentTime());
+            ret = aclrtMemcpy(outputdata[i], len, data, len, ACL_MEMCPY_DEVICE_TO_HOST);
+            //INFO_LOG("end OutputModelResult::aclrtMemcpy is %d", Utils::getCurrentTime());
+            if (ret != ACL_ERROR_NONE) {
+                ERROR_LOG("aclrtMemcpy failed, ret[%d]", ret);
+                return;
+            }
+
+            outData = reinterpret_cast<float*>(outputdata[i]);
+            //outputdata.push_back(outHostData);
+        } else {
+            outData = reinterpret_cast<float*>(data);
+            outputdata.push_back(data);
+        }
+        map<float, unsigned int, greater<float> > resultMap;
+        for (unsigned int j = 0; j < len / sizeof(float); ++j) {
+            resultMap[*outData] = j;
+            outData++;
+        }
+
+        int cnt = 0;
+        for (auto it = resultMap.begin(); it != resultMap.end(); ++it) {
+            // print top 5
+            if (++cnt > 5) {
+                break;
+            }
+
+            //INFO_LOG("top %d: index[%d] value[%lf]", cnt, it->second, it->first);
+        }
+    }
+
+    INFO_LOG("1.output data success");
+    return;
+}
+
+void ModelProcess::OutputModelResult(vector<Output_buf> &output)
+{
+
+    OutputModelResult();
+
+    size_t outputSize = aclmdlGetNumOutputs(modelDesc_);
+
+
+    for (size_t i = 0; i < outputSize; ++i) {
+        Output_buf tmp_output;
+        //ptr
+        tmp_output.ptr = outputdata[i];
+        //size
+        tmp_output.size = aclmdlGetOutputSizeByIndex(modelDesc_, i);
+        //ndim & shape
+        aclmdlIODims d={{'a'},0,{1,2,3}};
+        aclmdlIODims *dims=&d;
+        aclError ret = aclmdlGetOutputDims((const aclmdlDesc *)modelDesc_, i, dims);
+        tmp_output.ndim = (*dims).dimCount;
+        for(int j=0;j<(*dims).dimCount;j++)
+            tmp_output.shape.push_back((*dims).dims[j]);
+
+        //itemsize & format
+        aclDataType dt =  aclmdlGetOutputDataType(modelDesc_, i);
+        tmp_output.itemsize=ACLdt_size[dt];
+        tmp_output.format=ACLdt[dt];
+//        if(dt==ACL_FLOAT)
+//        {
+//            tmp_output.itemsize=4;
+//            tmp_output.format="float";
+//        }
+//        if(dt==ACL_FLOAT16)
+//        {
+//            tmp_output.itemsize=2;
+//            tmp_output.format="float16";
+//        }
+//        if(dt==ACL_UINT8)
+//        {
+//            tmp_output.itemsize=1;
+//            tmp_output.format="uint8";
+//        }
+//        if(dt==ACL_INT8)
+//        {
+//            tmp_output.itemsize=1;
+//            tmp_output.format="int8";
+//        }
+
+        //strides
+        int strides=tmp_output.itemsize;
+        std::vector<int64_t> tmp_strides;
+        std::vector<int64_t> tmp1_strides;
+        //vector<int64_t>::reverse_iterator it;//声明一个迭代器，来访问vector容器，作用：遍历或者指向vector容器的元素
+        for(auto it=tmp_output.shape.rbegin();it!=tmp_output.shape.rend();++it)
+        {
+            tmp_strides.push_back(strides);
+            strides = strides * (*it);
+        }
+        //for (int index = 0 ; index < tmp_output.ndim; index++) {
+        //   tmp_strides.push_back(strides);
+        //   strides = strides * tmp_output.shape[index];
+        //}
+        for(auto it=tmp_strides.rbegin();it!=tmp_strides.rend();++it){
+        //for (int index=0;index < tmp_strides.size(); index++) {
+           tmp1_strides.push_back(*it);
+        }
+        tmp_output.strides = tmp1_strides;
+        output.push_back(tmp_output);
+    }
+    INFO_LOG("2.output data success");
+}
+void ModelProcess::DestroyOutput()
+{
+    if (output_ == nullptr) {
+        return;
+    }
+
+    for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(output_); ++i) {
+        aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(output_, i);
+        void* data = aclGetDataBufferAddr(dataBuffer);
+        (void)aclrtFree(data);
+        (void)aclDestroyDataBuffer(dataBuffer);
+    }
+
+    (void)aclmdlDestroyDataset(output_);
+    output_ = nullptr;
+}
+
+Result ModelProcess::Execute()
+{
+    aclError ret = aclmdlExecute(modelId_, input_, output_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("execute model failed, modelId is %u", modelId_);
+        return FAILED;
+    }
+
+    INFO_LOG("model execute success");
+    return SUCCESS;
+}
+
+void ModelProcess::Unload()
+{
+    aclError ret;
+    if (!g_isDevice) {
+            //INFO_LOG("start Unload::aclrtFreeHost is %d", Utils::getCurrentTime());
+            //for (int index = 0 ; index < outputdata.size(); index++) {
+            //    ret = aclrtFreeHost(outputdata[index]);
+            //}
+            for(auto it=outputdata.begin();it!=outputdata.end();++it){
+                ret = aclrtFreeHost(*it);
+            }
+            //INFO_LOG("end Unload::aclrtFreeHost is %d", Utils::getCurrentTime());
+            if (ret != ACL_ERROR_NONE) {
+                ERROR_LOG("aclrtFreeHost failed, ret[%d]", ret);
+                return;
+            }
+        }
+    if (!loadFlag_) {
+        WARN_LOG("no model had been loaded, unload failed");
+        return;
+    }
+
+    ret = aclmdlUnload(modelId_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("unload model failed, modelId is %u", modelId_);
+    }
+
+    if (modelDesc_ != nullptr) {
+        (void)aclmdlDestroyDesc(modelDesc_);
+        modelDesc_ = nullptr;
+    }
+
+    if (modelMemPtr_ != nullptr) {
+        aclrtFree(modelMemPtr_);
+        modelMemPtr_ = nullptr;
+        modelMemSize_ = 0;
+    }
+
+    if (modelWeightPtr_ != nullptr) {
+        aclrtFree(modelWeightPtr_);
+        modelWeightPtr_ = nullptr;
+        modelWeightSize_ = 0;
+    }
+
+    loadFlag_ = false;
+    INFO_LOG("unload model success, modelId is %u", modelId_);
+}

dnmetis/backend_C++/dnmetis_backend/backend/built-in/src/sample_process.cpp

@@ -0,0 +1,249 @@
+/**
+* @file sample_process.cpp
+*
+* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+*/
+#include "sample_process.h"
+#include <iostream>
+#include "acl/acl.h"
+#include "utils.h"
+using namespace std;
+extern bool g_isDevice;
+
+
+SampleProcess::SampleProcess() :deviceId_(0), context_(nullptr), stream_(nullptr)
+{
+}
+
+SampleProcess::~SampleProcess()
+{
+}
+
+Result SampleProcess::InitResource(char* omModelPath)
+{
+    // ACL init
+    const char *aclConfigPath;
+    if((aclConfigPath = getenv("ACL_CONFIG_PATH"))) {
+        INFO_LOG("acl init with config file:%s", aclConfigPath);
+    } else {
+        aclConfigPath = "";
+    }
+    aclError ret = aclInit(aclConfigPath);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("acl init failed");
+        return FAILED;
+    }
+    INFO_LOG("acl init success");
+
+    // open device
+    ret = aclrtSetDevice(deviceId_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("acl open device %d failed", deviceId_);
+        return FAILED;
+    }
+    INFO_LOG("open device %d success", deviceId_);
+
+    // create context (set current)
+    ret = aclrtCreateContext(&context_, deviceId_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("acl create context failed");
+        return FAILED;
+    }
+    INFO_LOG("create context success");
+
+    // create stream
+    ret = aclrtCreateStream(&stream_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("acl create stream failed");
+        return FAILED;
+    }
+    INFO_LOG("create stream success");
+
+    // get run mode
+    aclrtRunMode runMode;
+    ret = aclrtGetRunMode(&runMode);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("acl get run mode failed");
+        return FAILED;
+    }
+    g_isDevice = (runMode == ACL_DEVICE);
+    INFO_LOG("get run mode success");
+    Result r_ret = processModel.LoadModelFromFileWithMem(omModelPath);
+    if (r_ret != SUCCESS) {
+        ERROR_LOG("load model from file failed");
+        return FAILED;
+    }
+
+    ret = processModel.CreateDesc();
+    if (ret != SUCCESS) {
+        ERROR_LOG("create model description failed");
+        return FAILED;
+    }
+
+    ret = processModel.CreateOutput();
+    if (ret != SUCCESS) {
+        ERROR_LOG("create model output failed");
+        return FAILED;
+    }
+
+    return SUCCESS;
+}
+
+Result SampleProcess::Process(void* binfile,int len)
+{
+    //for (size_t index = 0; index < sizeof(testFile) / sizeof(testFile[0]); ++index) {
+        //INFO_LOG("start to process file:%s", binfile.c_str());
+        // model process
+        //INFO_LOG("start memcpy is %d", Utils::getCurrentTime());
+        uint32_t devBufferSize = len;
+        void *picDevBuffer = Utils::GetDeviceBufferOfptr(binfile,len);
+        //ERROR_LOG("devBufferSize is %d", len);
+        //ERROR_LOG("picDevBuffer:%f.", *((float*)picDevBuffer));
+        if (picDevBuffer == nullptr) {
+            ERROR_LOG("get pic device buffer failed,index is %zu", 0);
+            return FAILED;
+        }
+        //INFO_LOG("end memcpy is %d", Utils::getCurrentTime());
+
+        //INFO_LOG("start CreateInput is %d", Utils::getCurrentTime());
+        Result ret = processModel.CreateInput(picDevBuffer, devBufferSize);
+        if (ret != SUCCESS) {
+            ERROR_LOG("model create input failed");
+            aclrtFree(picDevBuffer);
+            return FAILED;
+        }
+        //INFO_LOG("end CreateInput is %d", Utils::getCurrentTime());
+
+        INFO_LOG("start Execute is %d", Utils::getCurrentTime());
+        ret = processModel.Execute();
+        if (ret != SUCCESS) {
+            ERROR_LOG("model execute failed");
+            aclrtFree(picDevBuffer);
+            return FAILED;
+        }
+        INFO_LOG("end Execute is %d", Utils::getCurrentTime());
+
+        //INFO_LOG("start aclrtFree is %d", Utils::getCurrentTime());
+        // release model input buffer
+        aclrtFree(picDevBuffer);
+        //INFO_LOG("end aclrtFree is %d", Utils::getCurrentTime());
+
+        // print the top 5 confidence values with indexes.use function DumpModelOutputResult
+        // if want to dump output result to file in the current directory
+        processModel.OutputModelResult();
+
+    //}
+    // loop end
+
+
+    return SUCCESS;
+}
+
+Result SampleProcess::Process(void* binfile,int len, vector<Output_buf> &output, long &npuTime)
+{
+    //for (size_t index = 0; index < sizeof(testFile) / sizeof(testFile[0]); ++index) {
+        //INFO_LOG("start to process file:%s", binfile.c_str());
+        // model process
+        //INFO_LOG("start memcpy is %d", Utils::getCurrentTime());
+        uint32_t devBufferSize = len;
+        void *picDevBuffer = Utils::GetDeviceBufferOfptr(binfile,len);
+        //ERROR_LOG("devBufferSize is %d", len);
+        //ERROR_LOG("picDevBuffer:%f.", *((float*)picDevBuffer));
+        if (picDevBuffer == nullptr) {
+            ERROR_LOG("get pic device buffer failed,index is %zu", 0);
+            return FAILED;
+        }
+        //INFO_LOG("end memcpy is %d", Utils::getCurrentTime());
+
+        //INFO_LOG("start CreateInput is %d", Utils::getCurrentTime());
+        Result ret = processModel.CreateInput(picDevBuffer, devBufferSize);
+        if (ret != SUCCESS) {
+            ERROR_LOG("model create input failed");
+            aclrtFree(picDevBuffer);
+            return FAILED;
+        }
+        //INFO_LOG("end CreateInput is %d", Utils::getCurrentTime());
+
+        INFO_LOG("start Execute is %d", Utils::getCurrentTime());
+        long start = Utils::getCurrentTime();
+        ret = processModel.Execute();
+        if (ret != SUCCESS) {
+            ERROR_LOG("model execute failed");
+            aclrtFree(picDevBuffer);
+            return FAILED;
+        }
+        long end = Utils::getCurrentTime();
+        INFO_LOG("end Execute is %d", end);
+        
+        npuTime = end - start;
+        if (npuTime < 0) {
+            ERROR_LOG("NPU time not correct: %d start: %d end: %d", npuTime, start, end);
+            aclrtFree(picDevBuffer);
+            return FAILED;
+        }
+        INFO_LOG("npu compute cost %f ms", 1.0*(npuTime)/1000.0);
+
+        //INFO_LOG("start aclrtFree is %d", Utils::getCurrentTime());
+        // release model input buffer
+        aclrtFree(picDevBuffer);
+        //INFO_LOG("end aclrtFree is %d", Utils::getCurrentTime());
+
+        // print the top 5 confidence values with indexes.use function DumpModelOutputResult
+        // if want to dump output result to file in the current directory
+        processModel.OutputModelResult(output);
+
+    //}
+    // loop end
+
+
+    return SUCCESS;
+}
+
+Result SampleProcess::Unload()
+{
+    processModel.Unload();
+    processModel.DestroyDesc();
+    processModel.DestroyInput();
+    processModel.DestroyOutput();
+    DestroyResource();
+    return SUCCESS;
+}
+
+void SampleProcess::DestroyResource()
+{
+    aclError ret;
+    if (stream_ != nullptr) {
+        ret = aclrtDestroyStream(stream_);
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("destroy stream failed");
+        }
+        stream_ = nullptr;
+        INFO_LOG("end to destroy stream");
+    }
+
+    if (context_ != nullptr) {
+        ret = aclrtDestroyContext(context_);
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("destroy context failed");
+        }
+        context_ = nullptr;
+        INFO_LOG("end to destroy context");
+    }
+
+    ret = aclrtResetDevice(deviceId_);
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("reset device failed");
+    }
+    INFO_LOG("end to reset device is %d", deviceId_);
+
+    ret = aclFinalize();
+    if (ret != ACL_ERROR_NONE) {
+        ERROR_LOG("finalize acl failed");
+    }
+    INFO_LOG("end to finalize acl");
+
+}

dnmetis/backend_C++/dnmetis_backend/backend/built-in/src/utils.cpp

@@ -0,0 +1,171 @@
+/**
+* @file utils.cpp
+*
+* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+*/
+#include "utils.h"
+#include <iostream>
+#include <fstream>
+#include <cstring>
+#include "acl/acl.h"
+#include <stdio.h>
+#include <sys/time.h>
+
+bool g_isDevice = false;
+
+std::map<aclDataType, std::string> ACLdt{{ ACL_DT_UNDEFINED , "undefined"},
+{ ACL_FLOAT , "float"},
+{ ACL_FLOAT16 , "float16"},
+{ ACL_INT8 , "int8"},
+{ ACL_INT32 , "int32"},
+{ ACL_UINT8 , "uint8"},
+{ ACL_INT16 , "int16"},
+{ ACL_UINT16 , "uint16"},
+{ ACL_UINT32 , "uint32"},
+{ ACL_INT64 , "int64"},
+{ ACL_UINT64 , "uint64"},
+{ ACL_DOUBLE , "double"},
+{ ACL_BOOL , "bool"}};
+std::map<aclDataType, int> ACLdt_size{{ ACL_DT_UNDEFINED , -1},
+{ ACL_FLOAT , 4},
+{ ACL_FLOAT16 , 2},
+{ ACL_INT8 , 1},
+{ ACL_INT32 , 4},
+{ ACL_UINT8 , 1},
+{ ACL_INT16 , 2},
+{ ACL_UINT16 , 2},
+{ ACL_UINT32 , 4},
+{ ACL_INT64 , 8},
+{ ACL_UINT64 , 8},
+{ ACL_DOUBLE , 8},
+{ ACL_BOOL , 1}};
+
+//Config configSettings("cfg/built-in_config.txt");
+
+void* Utils::ReadBinFile(std::string fileName, uint32_t &fileSize)
+{
+    std::ifstream binFile(fileName, std::ifstream::binary);
+    if (binFile.is_open() == false) {
+        ERROR_LOG("open file %s failed", fileName.c_str());
+        return nullptr;
+    }
+
+    binFile.seekg(0, binFile.end);
+    uint32_t binFileBufferLen = binFile.tellg();
+    if (binFileBufferLen == 0) {
+        ERROR_LOG("binfile is empty, filename is %s", fileName.c_str());
+        binFile.close();
+        return nullptr;
+    }
+    //ERROR_LOG("binFileBufferLen is %d", binFileBufferLen);
+
+    binFile.seekg(0, binFile.beg);
+
+    void* binFileBufferData = nullptr;
+    aclError ret = ACL_ERROR_NONE;
+    if (!g_isDevice) {
+        ret = aclrtMallocHost(&binFileBufferData, binFileBufferLen);
+        if (binFileBufferData == nullptr) {
+            ERROR_LOG("malloc binFileBufferData failed");
+            binFile.close();
+            return nullptr;
+        }
+    } else {
+        ret = aclrtMalloc(&binFileBufferData, binFileBufferLen, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("malloc device buffer failed. size is %u", binFileBufferLen);
+            binFile.close();
+            return nullptr;
+        }
+    }
+    binFile.read(static_cast<char *>(binFileBufferData), binFileBufferLen);
+    binFile.close();
+    fileSize = binFileBufferLen;
+    return binFileBufferData;
+}
+
+void* Utils::GetDeviceBufferOfFile(std::string fileName, uint32_t &fileSize)
+{
+    uint32_t inputHostBuffSize = 0;
+    void* inputHostBuff = Utils::ReadBinFile(fileName, inputHostBuffSize);
+    if (inputHostBuff == nullptr) {
+        return nullptr;
+    }
+    if (!g_isDevice) {
+        void *inBufferDev = nullptr;
+        uint32_t inBufferSize = inputHostBuffSize;
+        aclError ret = aclrtMalloc(&inBufferDev, inBufferSize, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("malloc device buffer failed. size is %u", inBufferSize);
+            aclrtFreeHost(inputHostBuff);
+            return nullptr;
+        }
+
+        ret = aclrtMemcpy(inBufferDev, inBufferSize, inputHostBuff, inputHostBuffSize, ACL_MEMCPY_HOST_TO_DEVICE);
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("memcpy failed. device buffer size is %u, input host buffer size is %u",
+                inBufferSize, inputHostBuffSize);
+            aclrtFree(inBufferDev);
+            aclrtFreeHost(inputHostBuff);
+            return nullptr;
+        }
+        aclrtFreeHost(inputHostBuff);
+        fileSize = inBufferSize;
+        return inBufferDev;
+    } else {
+        fileSize = inputHostBuffSize;
+        return inputHostBuff;
+    }
+}
+
+void* Utils::GetDeviceBufferOfptr(void* fileName, uint32_t len)
+{
+    uint32_t inputHostBuffSize = len;
+    void* inputHostBuff = fileName;
+    if (inputHostBuff == nullptr) {
+        return nullptr;
+    }
+    if (!g_isDevice) {
+        void *inBufferDev = nullptr;
+        uint32_t inBufferSize = inputHostBuffSize;
+        //INFO_LOG("start memcpy::aclrtMalloc is %d", Utils::getCurrentTime());
+        aclError ret = aclrtMalloc(&inBufferDev, inBufferSize, (aclrtMemMallocPolicy)(Config::getInstance()->Read("aclrtMemMallocPolicy", 0)));
+        //INFO_LOG("end  memcpy::aclrtMalloc is %d", Utils::getCurrentTime());
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("malloc device buffer failed. size is %u", inBufferSize);
+            aclrtFreeHost(inputHostBuff);
+            return nullptr;
+        }
+        //INFO_LOG("start memcpy::aclrtMemcpy is %d", Utils::getCurrentTime());
+        ret = aclrtMemcpy(inBufferDev, inBufferSize, inputHostBuff, inputHostBuffSize, ACL_MEMCPY_HOST_TO_DEVICE);
+        //INFO_LOG("end memcpy::aclrtMemcpy is %d", Utils::getCurrentTime());
+        if (ret != ACL_ERROR_NONE) {
+            ERROR_LOG("memcpy failed. device buffer size is %u, input host buffer size is %u",
+                inBufferSize, inputHostBuffSize);
+            aclrtFree(inBufferDev);
+            aclrtFreeHost(inputHostBuff);
+            return nullptr;
+        }
+        //INFO_LOG("start memcpy::aclrtFreeHost is %d", Utils::getCurrentTime());
+        aclrtFreeHost(inputHostBuff);
+        //INFO_LOG("end memcpy::aclrtFreeHost is %d", Utils::getCurrentTime());
+        //fileSize = inBufferSize;
+        return inBufferDev;
+    } else {
+        //fileSize = inputHostBuffSize;
+        return inputHostBuff;
+    }
+}
+
+
+long Utils::getCurrentTime()
+{
+  struct timeval tv;
+  gettimeofday(&tv,NULL);
+  return tv.tv_sec * 1000000 + tv.tv_usec;
+  // return tv.tv_usec;
+}

dnmetis/backend_C++/dnmetis_backend/backend/custom/trtbackend.cpp

@@ -0,0 +1,29 @@
+#include "trtbackend.h"
+
+CREATE_BACKEND_INIT(TrtBackend,model,data)
+{
+  INFO_LOG("TrtBackend INIT model success");
+  return 0;
+}
+
+CREATE_BACKEND_LOAD(TrtBackend,model,data)
+{
+  INFO_LOG("TrtBackend LOAD model success");
+  return 0;
+}
+
+CREATE_BACKEND_PREDICT(TrtBackend,model,data,len,output,gpuTime)
+{
+  INFO_LOG("TrtBackend PREDICT model success");
+  return 0;
+}
+CREATE_BACKEND_UNLOAD(TrtBackend,model,data)
+{
+  INFO_LOG("TrtBackend UNLOAD model success");
+  return 0;
+}
+
+
+REGISTER_BASE_BACKEND_CREATOR(TRT, TrtBackend)
+
+

dnmetis/backend_C++/dnmetis_backend/backend/custom/trtbackend.h

@@ -0,0 +1,7 @@
+#include "BackendFactory.h"
+#include <vector>
+
+CREATE_BACKEND(TrtBackend)
+
+
+

dnmetis/backend_C++/dnmetis_backend/backend/inc/BackendFactory.h

@@ -0,0 +1,91 @@
+/**
+ * Copyright 2020 Huawei Technologies Co., Ltd
+
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+
+ * http://www.apache.org/licenses/LICENSE-2.0
+
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#ifndef BASE_BACKEND_FACTORY_H_
+#define BASE_BACKEND_FACTORY_H_
+
+#include <map>
+#include <memory>
+#include <mutex>
+#include <string>
+#include "BaseBackend.h"
+
+
+
+typedef std::shared_ptr<BaseBackend> (*BASE_BACKEND_CREATOR_FUN)(void);
+
+class BackendFactory {
+ public:
+  static BackendFactory *Instance();
+  int Init();
+  /**
+  * @ingroup domi_omg
+  * @brief Create a backend based on the type entered
+  * @param [in] type Framework type
+  * @return Created backend
+  */
+  std::shared_ptr<BaseBackend> CreateBaseBackend(const FrameworkType type);
+
+  /**
+  * @ingroup domi_omg
+  * @brief Register create function
+  * @param [in] type Framework type
+  * @param [in] fun ModelParser's create function
+  */
+  void RegisterCreator(const FrameworkType type, BASE_BACKEND_CREATOR_FUN fun);
+
+ protected:
+  BackendFactory() {}
+  ~BackendFactory();
+
+ private:
+  std::map<FrameworkType, BASE_BACKEND_CREATOR_FUN> creator_map_;
+  bool is_init_ = false;
+};  // end class BackendFactory
+
+class BaseBackendRegisterar {
+ public:
+  BaseBackendRegisterar(const FrameworkType type, BASE_BACKEND_CREATOR_FUN fun) {
+    BackendFactory::Instance()->RegisterCreator(type, fun);
+  }
+  ~BaseBackendRegisterar() {}
+};
+
+// Registration macros for backend
+typedef std::shared_ptr<BaseBackend> (*BASE_BACKEND_CREATOR_FUN)(void);
+
+// Registration macros for BaseBackend
+#define REGISTER_BASE_BACKEND_CREATOR(type, clazz)               \
+  std::shared_ptr<BaseBackend> Creator_##type##_Base_Backend() { \
+    std::shared_ptr<clazz> ptr = nullptr;                             \
+    try {                                                        \
+      ptr = std::make_shared<clazz>();                                \
+    } catch (...) {                                              \
+      ptr = nullptr;                                             \
+    }                                                            \
+    return std::shared_ptr<BaseBackend>(ptr);                    \
+  }                                                              \
+  BaseBackendRegisterar g_##type##_Base_Backend_Creator(type, Creator_##type##_Base_Backend);
+
+
+#endif 
+
+
+
+
+
+
+

dnmetis/backend_C++/dnmetis_backend/backend/inc/BaseBackend.h

@@ -0,0 +1,78 @@
+
+/**
+ * Copyright 2020 Huawei Technologies Co., Ltd
+
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+
+ * http://www.apache.org/licenses/LICENSE-2.0
+
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#ifndef BASE_BACKEND_H_
+#define BASE_BACKEND_H_
+
+//#include "utils.h"
+#include "common.h"
+#include <vector>
+using namespace std;
+
+class BaseBackend {
+ public:
+  BaseBackend() {};
+  /**
+   * @ingroup domi_omg
+   * @brief Deconstructor
+   */
+  ~BaseBackend() {};
+
+  virtual int init(char* model,char* data){};
+
+  virtual int load(char* model,char* data){};
+
+  virtual int predict(char* model,void* data,int len, std::vector<Output_buf> &output, long &deviceTime){};
+ 
+  virtual int unload(char* model,char* data){};
+
+  int param();
+
+  int runner();
+
+  int runnerthread();
+
+  int statistic();
+
+};
+
+
+
+#define CREATE_BACKEND(x)  \
+class x : public BaseBackend{ \
+public:                       \
+        x(){};                  \
+        ~x(){};       \
+       int init(char* model,char* data);   \
+       int load(char* model,char* data);      \
+       int predict(char* model,void* data,int len, vector<Output_buf> &output, long &deviceTime);  \
+       int unload(char* model,char* data);   \
+       char* model;   \
+       char* data;     \
+ }; \
+
+#define CREATE_BACKEND_INIT(x,model,data)  \
+   int x::init(char* model,char* data)
+#define CREATE_BACKEND_LOAD(x,model,data)   \
+   int x::load(char* model,char* data)
+#define CREATE_BACKEND_PREDICT(x,model,data,len,output,deviceTime)   \
+   int x::predict(char* model,void* data,int len,vector<Output_buf> &output, long &deviceTime)
+#define CREATE_BACKEND_UNLOAD(x,model,data) \
+   int x::unload(char* model,char* data)
+
+#endif 
+

dnmetis/backend_C++/dnmetis_backend/inc/Config.h

@@ -0,0 +1,198 @@
+//Config.h
+#pragma once
+
+#include <string>
+#include <map>
+#include <iostream>
+#include <fstream>
+#include <sstream>
+
+
+/*
+* \brief Generic configuration Class
+*
+*/
+class Config {
+    // Data
+protected:
+    std::string m_Delimiter;  //!< separator between key and value
+    std::string m_Comment;    //!< separator between value and comments
+    std::map<std::string,std::string> m_Contents;  //!< extracted keys and values
+
+    typedef std::map<std::string,std::string>::iterator mapi;
+    typedef std::map<std::string,std::string>::const_iterator mapci;
+    // Methods
+public:
+    static Config *instance;
+    static Config *setInstance(char* cfg){
+        if(instance == nullptr){
+            instance = new Config(cfg);
+        }
+        return instance;
+    }
+    static Config *getInstance(){
+        return instance;
+    }
+    Config( std::string filename,std::string delimiter = "=",std::string comment = "#" );
+    Config();
+    template<class T> T Read( const std::string& in_key ) const;  //!<Search for key and read value or optional default value, call as read<T>
+    template<class T> T Read( const std::string& in_key, const T& in_value ) const;
+    template<class T> bool ReadInto( T& out_var, const std::string& in_key ) const;
+    template<class T>
+    bool ReadInto( T& out_var, const std::string& in_key, const T& in_value ) const;
+    bool FileExist(std::string filename);
+    void ReadFile(std::string filename,std::string delimiter = "=",std::string comment = "#" );
+
+    // Check whether key exists in configuration
+    bool KeyExists( const std::string& in_key ) const;
+
+    // Modify keys and values
+    template<class T> void Add( const std::string& in_key, const T& in_value );
+    void Remove( const std::string& in_key );
+
+    // Check or change configuration syntax
+    std::string GetDelimiter() const { return m_Delimiter; }
+    std::string GetComment() const { return m_Comment; }
+    std::string SetDelimiter( const std::string& in_s )
+    { std::string old = m_Delimiter;  m_Delimiter = in_s;  return old; }
+    std::string SetComment( const std::string& in_s )
+    { std::string old = m_Comment;  m_Comment =  in_s;  return old; }
+
+    // Write or read configuration
+    friend std::ostream& operator<<( std::ostream& os, const Config& cf );
+    friend std::istream& operator>>( std::istream& is, Config& cf );
+
+protected:
+    template<class T> static std::string T_as_string( const T& t );
+    template<class T> static T string_as_T( const std::string& s );
+    static void Trim( std::string& inout_s );
+
+
+    // Exception types
+public:
+    struct File_not_found {
+        std::string filename;
+        File_not_found( const std::string& filename_ = std::string() )
+            : filename(filename_) {} };
+        struct Key_not_found {  // thrown only by T read(key) variant of read()
+            std::string key;
+            Key_not_found( const std::string& key_ = std::string() )
+                : key(key_) {} };
+};
+
+
+/* static */
+template<class T>
+std::string Config::T_as_string( const T& t )
+{
+    // Convert from a T to a string
+    // Type T must support << operator
+    std::ostringstream ost;
+    ost << t;
+    return ost.str();
+}
+
+
+/* static */
+template<class T>
+T Config::string_as_T( const std::string& s )
+{
+    // Convert from a string to a T
+    // Type T must support >> operator
+    T t;
+    std::istringstream ist(s);
+    ist >> t;
+    return t;
+}
+
+
+/* static */
+template<>
+inline std::string Config::string_as_T<std::string>( const std::string& s )
+{
+    // Convert from a string to a string
+    // In other words, do nothing
+    return s;
+}
+
+
+/* static */
+template<>
+inline bool Config::string_as_T<bool>( const std::string& s )
+{
+    // Convert from a string to a bool
+    // Interpret "false", "F", "no", "n", "0" as false
+    // Interpret "true", "T", "yes", "y", "1", "-1", or anything else as true
+    bool b = true;
+    std::string sup = s;
+    for( std::string::iterator p = sup.begin(); p != sup.end(); ++p )
+        *p = toupper(*p);  // make string all caps
+    if( sup==std::string("FALSE") || sup==std::string("F") ||
+        sup==std::string("NO") || sup==std::string("N") ||
+        sup==std::string("0") || sup==std::string("NONE") )
+        b = false;
+    return b;
+}
+
+
+template<class T>
+T Config::Read( const std::string& key ) const
+{
+    // Read the value corresponding to key
+    mapci p = m_Contents.find(key);
+    if( p == m_Contents.end() ) throw Key_not_found(key);
+    return string_as_T<T>( p->second );
+}
+
+
+template<class T>
+T Config::Read( const std::string& key, const T& value ) const
+{
+    // Return the value corresponding to key or given default value
+    // if key is not found
+    mapci p = m_Contents.find(key);
+    if( p == m_Contents.end() ) return value;
+    return string_as_T<T>( p->second );
+}
+
+
+template<class T>
+bool Config::ReadInto( T& var, const std::string& key ) const
+{
+    // Get the value corresponding to key and store in var
+    // Return true if key is found
+    // Otherwise leave var untouched
+    mapci p = m_Contents.find(key);
+    bool found = ( p != m_Contents.end() );
+    if( found ) var = string_as_T<T>( p->second );
+    return found;
+}
+
+
+template<class T>
+bool Config::ReadInto( T& var, const std::string& key, const T& value ) const
+{
+    // Get the value corresponding to key and store in var
+    // Return true if key is found
+    // Otherwise set var to given default
+    mapci p = m_Contents.find(key);
+    bool found = ( p != m_Contents.end() );
+    if( found )
+        var = string_as_T<T>( p->second );
+    else
+        var = value;
+    return found;
+}
+
+
+template<class T>
+void Config::Add( const std::string& in_key, const T& value )
+{
+    // Add a key with given value
+    std::string v = T_as_string( value );
+    std::string key=in_key;
+    trim(key);
+    trim(v);
+    m_Contents[key] = v;
+    return;
+}

dnmetis/backend_C++/dnmetis_backend/inc/common.h

@@ -0,0 +1,101 @@
+/**
+* @file common.h
+*
+* Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+*/
+#pragma once
+#include <iostream>
+#include<vector>
+#include <pybind11/numpy.h>
+#include <pybind11/stl.h>
+#include "half.hpp"
+#include "Config.h"
+
+#define LOG 2
+
+#define INFO_LOG(fmt, args...)   if(Config::getInstance()->Read("backend_loglevel", 0)>=3) fprintf(stdout, "[INFO]  " fmt "\n", ##args)
+#define WARN_LOG(fmt, args...)   if(Config::getInstance()->Read("backend_loglevel", 0)>=2) fprintf(stdout, "[WARN]  " fmt "\n", ##args)
+#define ERROR_LOG(fmt, args...)  if(Config::getInstance()->Read("backend_loglevel", 0)>=1) fprintf(stdout, "[ERROR] " fmt "\n", ##args)
+
+namespace py = pybind11;
+// half_float::half behaviors like float, but with different precision
+
+using float16 = half_float::half;
+
+NAMESPACE_BEGIN(pybind11)
+NAMESPACE_BEGIN(detail)
+
+template <>
+struct type_caster<float16> {
+public:
+    PYBIND11_TYPE_CASTER(float16, _("float16"));
+    using float_caster = type_caster<float>;
+
+    bool load(handle src, bool convert) {
+        float_caster caster;
+        if (caster.load(src, convert)) {
+            this->value = float16(float(caster));  // Implicit cast defined by `type_caster`.
+            return true;
+
+        }
+        return false;
+    }
+    static handle cast(float16 src, return_value_policy policy, handle parent) {
+        return float_caster::cast(float(src), policy, parent);
+    }
+};
+
+constexpr int NPY_FLOAT16 = 23;
+
+template <>
+struct npy_format_descriptor<float16> {
+  static pybind11::dtype dtype() {
+    handle ptr = npy_api::get().PyArray_DescrFromType_(NPY_FLOAT16);
+    return reinterpret_borrow<pybind11::dtype>(ptr);
+  }
+  static std::string format() {
+    // following: https://docs.python.org/3/library/struct.html#format-characters
+    return "e";
+  }
+  static constexpr auto name() {
+    return _("float16");
+  }
+};
+
+NAMESPACE_END(detail)
+NAMESPACE_END(pybind11)
+
+
+
+typedef enum Result {
+    SUCCESS = 0,
+    FAILED = 1
+} Result;
+
+enum FrameworkType {
+  CAFFE = 0,
+  MINDSPORE = 1,
+  TENSORFLOW = 3,
+  ANDROID_NN,
+  ACL,
+  TRT,
+  HIAI_ENGINE,
+  FRAMEWORK_RESERVED,
+};
+
+struct Output_buf {
+    void *ptr = nullptr;          // Pointer to the underlying storage
+    int64_t itemsize = 0;         // Size of individual items in bytes
+    int64_t size = 0;             // Total number of entries
+    std::string format;           // For homogeneous buffers, this should be set to format_descriptor<T>::format()
+    int64_t ndim = 0;             // Number of dimensions
+    std::vector<int64_t> shape;   // Shape of the tensor (1 entry per dimension)
+    std::vector<int64_t> strides; // Number of bytes between adjacent entries (for each per dimension)
+    bool readonly = false;        // flag to indicate if the underlying storage may be written to
+};
+
+

dnmetis/backend_C++/dnmetis_backend/setup.py

@@ -0,0 +1,140 @@
+from setuptools import setup, Extension
+from setuptools.command.build_ext import build_ext
+import sys
+import setuptools
+
+__version__ = 'V1.0.2'
+
+
+class get_pybind_include(object):
+    """Helper class to determine the pybind11 include path
+
+    The purpose of this class is to postpone importing pybind11
+    until it is actually installed, so that the ``get_include()``
+    method can be invoked. """
+
+    def __str__(self):
+        import pybind11
+        return pybind11.get_include()
+
+
+ext_modules = [
+    Extension(
+        'dnmetis_backend',
+        # Sort input source files to ensure bit-for-bit reproducible builds
+        # (https://github.com/pybind/dnmetis_backend/pull/53)
+        #sorted(['src/main.cpp','src/model_process.cpp','src/sample_process.cpp','src/utils.cpp']),
+        sources=[
+            'src/main.cpp',
+            'src/Config.cpp',
+            'backend/BaseBackend.cpp',
+            'backend/BackendFactory.cpp',
+
+            'backend/built-in/aclbackend.cpp',
+            'backend/built-in/src/model_process.cpp',
+            'backend/built-in/src/sample_process.cpp',
+            'backend/built-in/src/utils.cpp',
+
+            'backend/custom/trtbackend.cpp',
+        ],
+        include_dirs=[
+            # Path to pybind11 headers
+            get_pybind_include(),
+            './inc/',
+            './backend/inc',
+            './backend/built-in',
+            './backend/built-in/inc',
+            './backend/custom',
+            './backend/custom/inc',
+            '/home/HwHiAiUser/Ascend/ascend-toolkit/20.10.0.B023//acllib/include/',
+        ],
+        library_dirs=['/home/HwHiAiUser/Ascend/ascend-toolkit/20.10.0.B023//acllib/lib64/',],
+        libraries=['ascendcl',],
+        language='c++'
+    ),
+]
+
+
+# cf http://bugs.python.org/issue26689
+def has_flag(compiler, flagname):
+    """Return a boolean indicating whether a flag name is supported on
+    the specified compiler.
+    """
+    import tempfile
+    import os
+    with tempfile.NamedTemporaryFile('w', suffix='.cpp', delete=False) as f:
+        f.write('int main (int argc, char **argv) { return 0; }')
+        fname = f.name
+    try:
+        compiler.compile([fname], extra_postargs=[flagname])
+    except setuptools.distutils.errors.CompileError:
+        return False
+    finally:
+        try:
+            os.remove(fname)
+        except OSError:
+            pass
+    return True
+
+
+def cpp_flag(compiler):
+    """Return the -std=c++[11/14/17] compiler flag.
+
+    The newer version is prefered over c++11 (when it is available).
+    """
+    flags = ['-std=c++14', '-std=c++17','-std=c++11']
+
+    for flag in flags:
+        if has_flag(compiler, flag):
+            return flag
+
+    raise RuntimeError('Unsupported compiler -- at least C++11 support '
+                       'is needed!')
+
+
+class BuildExt(build_ext):
+    """A custom build extension for adding compiler-specific options."""
+    c_opts = {
+        'msvc': ['/EHsc'],
+        'unix': ['-w','-O0', '-fpermissive'],
+    }
+    l_opts = {
+        'msvc': [],
+        'unix': ['-O0'],
+    }
+
+    if sys.platform == 'darwin':
+        darwin_opts = ['-stdlib=libc++', '-mmacosx-version-min=10.7']
+        c_opts['unix'] += darwin_opts
+        l_opts['unix'] += darwin_opts
+
+    def build_extensions(self):
+        ct = self.compiler.compiler_type
+        opts = self.c_opts.get(ct, [])
+        link_opts = self.l_opts.get(ct, [])
+        if ct == 'unix':
+            opts.append(cpp_flag(self.compiler))
+            if has_flag(self.compiler, '-fvisibility=hidden'):
+                opts.append('-fvisibility=hidden')
+
+        for ext in self.extensions:
+            ext.define_macros = [('VERSION_INFO', '"{}"'.format(self.distribution.get_version()))]
+            ext.extra_compile_args = opts
+            ext.extra_link_args = link_opts
+        build_ext.build_extensions(self)
+
+
+setup(
+    name='dnmetis_backend',
+    version=__version__,
+    author='chegulu',
+    author_email='chegulu@xxx.com',
+    url='https://chegulu',
+    description='A test tool project using pybind11',
+    long_description='',
+    ext_modules=ext_modules,
+    setup_requires=['pybind11==2.5.0'],
+    #data_files=[('config', ['cfg/config.txt'])],
+    cmdclass={'build_ext': BuildExt},
+    zip_safe=False,
+)

dnmetis/backend_C++/dnmetis_backend/src/Config.cpp

@@ -0,0 +1,156 @@
+// Config.cpp
+
+#include "Config.h"
+
+using namespace std;
+
+
+Config::Config( string filename, string delimiter,
+               string comment )
+               : m_Delimiter(delimiter), m_Comment(comment)
+{
+    // Construct a Config, getting keys and values from given file
+
+    std::ifstream in( filename.c_str() );
+
+    if( !in ) throw File_not_found( filename );
+
+    in >> (*this);
+}
+
+
+Config::Config()
+: m_Delimiter( string(1,'=') ), m_Comment( string(1,'#') )
+{
+    // Construct a Config without a file; empty
+}
+
+
+
+bool Config::KeyExists( const string& key ) const
+{
+    // Indicate whether key is found
+    mapci p = m_Contents.find( key );
+    return ( p != m_Contents.end() );
+}
+
+
+/* static */
+void Config::Trim( string& inout_s )
+{
+    // Remove leading and trailing whitespace
+    static const char whitespace[] = " \n\t\v\r\f";
+    inout_s.erase( 0, inout_s.find_first_not_of(whitespace) );
+    inout_s.erase( inout_s.find_last_not_of(whitespace) + 1U );
+}
+
+
+std::ostream& operator<<( std::ostream& os, const Config& cf )
+{
+    // Save a Config to os
+    for( Config::mapci p = cf.m_Contents.begin();
+        p != cf.m_Contents.end();
+        ++p )
+    {
+        os << p->first << " " << cf.m_Delimiter << " ";
+        os << p->second << std::endl;
+    }
+    return os;
+}
+
+void Config::Remove( const string& key )
+{
+    // Remove key and its value
+    m_Contents.erase( m_Contents.find( key ) );
+    return;
+}
+
+std::istream& operator>>( std::istream& is, Config& cf )
+{
+    // Load a Config from is
+    // Read in keys and values, keeping internal whitespace
+    typedef string::size_type pos;
+    const string& delim  = cf.m_Delimiter;  // separator
+    const string& comm   = cf.m_Comment;    // comment
+    const pos skip = delim.length();        // length of separator
+
+    string nextline = "";  // might need to read ahead to see where value ends
+
+    while( is || nextline.length() > 0 )
+    {
+        // Read an entire line at a time
+        string line;
+        if( nextline.length() > 0 )
+        {
+            line = nextline;  // we read ahead; use it now
+            nextline = "";
+        }
+        else
+        {
+            std::getline( is, line );
+        }
+
+        // Ignore comments
+        line = line.substr( 0, line.find(comm) );
+
+        // Parse the line if it contains a delimiter
+        pos delimPos = line.find( delim );
+        if( delimPos < string::npos )
+        {
+            // Extract the key
+            string key = line.substr( 0, delimPos );
+            line.replace( 0, delimPos+skip, "" );
+
+            // See if value continues on the next line
+            // Stop at blank line, next line with a key, end of stream,
+            // or end of file sentry
+            bool terminate = false;
+            while( !terminate && is )
+            {
+                std::getline( is, nextline );
+                terminate = true;
+
+                string nlcopy = nextline;
+                Config::Trim(nlcopy);
+                if( nlcopy == "" ) continue;
+
+                nextline = nextline.substr( 0, nextline.find(comm) );
+                if( nextline.find(delim) != string::npos )
+                    continue;
+
+                nlcopy = nextline;
+                Config::Trim(nlcopy);
+                if( nlcopy != "" ) line += "\n";
+                line += nextline;
+                terminate = false;
+            }
+
+            // Store key and value
+            Config::Trim(key);
+            Config::Trim(line);
+            cf.m_Contents[key] = line;  // overwrites if key is repeated
+        }
+    }
+
+    return is;
+}
+bool Config::FileExist(std::string filename)
+{
+    bool exist= false;
+    std::ifstream in( filename.c_str() );
+    if( in )
+        exist = true;
+    return exist;
+}
+
+void Config::ReadFile( string filename, string delimiter,
+                      string comment )
+{
+    m_Delimiter = delimiter;
+    m_Comment = comment;
+    std::ifstream in( filename.c_str() );
+
+    if( !in ) throw File_not_found( filename );
+
+    in >> (*this);
+}

dnmetis/backend_C++/dnmetis_backend/src/main.cpp

@@ -0,0 +1,150 @@
+#include <pybind11/pybind11.h>
+#include "BackendFactory.h"
+#include "BaseBackend.h"
+//#include "common.h"
+#include "utils.h"
+#include <string>
+
+using namespace std;
+
+bool warmup=true;
+
+long deviceTime=0;
+
+std::shared_ptr<BaseBackend> backend;
+Config * Config::instance = nullptr;
+
+int backend_setconfig(char* cfg)
+{
+   Config::setInstance(cfg);
+}
+
+int backend_load(int backend_type,char* omModelPath,char* binfile)
+{
+   backend = BackendFactory::Instance()->CreateBaseBackend(FrameworkType(backend_type));
+   if(backend == nullptr)
+   {
+      ERROR_LOG("FAILED, Not found the test backend, type:%d.", FrameworkType(backend_type));
+      return FAILED;
+   }
+   backend->init(omModelPath,binfile);
+   printf("[INFO] AclBackend init OK\n");
+   backend->load(omModelPath,binfile);
+   printf("[INFO] AclBackend load OK\n");
+}
+
+//py::array backend_predict(int type, char* omModelPath, py::array binfile)
+vector<py::array> backend_predict(int type, char* omModelPath, py::array binfile)
+{
+   if(warmup)
+   {
+      printf("[INFO] start warmup AclBackend predict\n");
+      //warmup=false;
+   }
+   INFO_LOG("start backend_predict is %d", Utils::getCurrentTime());
+   std::vector<Output_buf> result_buf;
+   //INFO_LOG("binfile.nbytes is %d", binfile.nbytes());
+   deviceTime = 0;
+   backend->predict(omModelPath, binfile.mutable_data(), binfile.nbytes(),result_buf, deviceTime);
+   INFO_LOG("Pure device execute time is %f ms", deviceTime);
+   if(warmup)
+   {
+      printf("[INFO] end warmup AclBackend predict\n");
+      warmup=false;
+   }
+
+   INFO_LOG("end backend_predict is %d", Utils::getCurrentTime());
+
+   vector<py::array> vec_result;
+   for(int i =0 ; i<result_buf.size();i++)
+   {
+       std::string str;
+       if(!result_buf[i].format.compare("uint8"))
+           str=py::format_descriptor<uint8_t>::format();
+       if(!result_buf[i].format.compare("int8"))
+           str=py::format_descriptor<int8_t>::format();
+       if(!result_buf[i].format.compare("float"))
+           str=py::format_descriptor<float>::format();
+       if(!result_buf[i].format.compare("float16"))
+           str=py::format_descriptor<float16>::format();
+       if(!result_buf[i].format.compare("int64"))
+           str=py::format_descriptor<int64_t>::format();
+       if(!result_buf[i].format.compare("uint64"))
+           str=py::format_descriptor<uint64_t>::format();
+       py::buffer_info tmp=py::buffer_info(
+	           result_buf[i].ptr,
+		       (ssize_t)result_buf[i].itemsize, //itemsize
+		       str,
+		       (ssize_t)result_buf[i].ndim,// ndim
+		       result_buf[i].shape, // shape
+		       result_buf[i].strides  //strides
+           );
+       py::dtype dt = py::dtype(str);
+       py::array result = py::array(dt,tmp.shape, tmp.strides, tmp.ptr);
+       vec_result.push_back(result);
+   }
+   return vec_result;
+   //return result;
+}
+
+long backend_get_device_time()
+{
+   return deviceTime;
+}
+
+int backend_unload(int type,char* omModelPath,char* binfile)
+{
+   backend->unload(omModelPath,binfile);
+   printf("[INFO] AclBackend unload OK\n");
+}
+
+namespace py = pybind11;
+
+PYBIND11_MODULE(dnmetis_backend, m) {
+    m.doc() = R"pbdoc(
+        Pybind11 example plugin
+        -----------------------
+
+        .. currentmodule:: dnmetis_backend
+
+        .. autosummary::
+           :toctree: _generate
+
+           add
+           subtract
+    )pbdoc";
+    /*m.def("backend_main", &backend_main, R"pbdoc(
+        backend
+    )pbdoc");*/
+    m.def("backend_setconfig", &backend_setconfig, R"pbdoc(
+        backend
+    )pbdoc");
+
+    m.def("backend_load", &backend_load, R"pbdoc(
+        backend
+    )pbdoc");
+
+    m.def("backend_predict", &backend_predict, R"pbdoc(
+        backend
+    )pbdoc");
+
+    m.def("backend_get_device_time", &backend_get_device_time, R"pbdoc(
+        backend
+    )pbdoc");
+
+    m.def("backend_unload", &backend_unload, R"pbdoc(
+        backend
+    )pbdoc");
+    m.def("add", [](int i, int j) { return i + j; }, R"pbdoc(
+        add
+    )pbdoc");
+    m.def("subtract", [](int i, int j) { return i - j; }, R"pbdoc(
+        subtract
+    )pbdoc");
+
+#ifdef VERSION_INFO
+    m.attr("__version__") = VERSION_INFO;
+#else
+    m.attr("__version__") = "dev";
+#endif
+}

dnmetis/backend_C++/dnmetis_backend/tests/test.py

@@ -0,0 +1,8 @@
+import dnmetis_backend as m
+
+print(m.__version__)
+assert m.add(1, 2) == 3
+print("Test add Ok!")
+assert m.subtract(1, 2) == -1
+print("Test subtract Ok!")
+print("Test Ok!")

dnmetis/backend_cfg/built-in_config.txt

@@ -0,0 +1,2 @@
+aclrtMemMallocPolicy=2
+backend_loglevel=3

dnmetis/datasets/imagenet_10/val_map.txt

@@ -0,0 +1,10 @@
+ILSVRC2012_val_00000001.JPEG 65
+ILSVRC2012_val_00000002.JPEG 970
+ILSVRC2012_val_00000003.JPEG 230
+ILSVRC2012_val_00000004.JPEG 809
+ILSVRC2012_val_00000005.JPEG 516
+ILSVRC2012_val_00000006.JPEG 57
+ILSVRC2012_val_00000007.JPEG 334
+ILSVRC2012_val_00000008.JPEG 415
+ILSVRC2012_val_00000009.JPEG 674
+ILSVRC2012_val_00000010.JPEG 332

dnmetis/main.py

@@ -0,0 +1,150 @@
+import argparse
+import array
+import collections
+import json
+import os
+import sys
+import threading
+import time
+from queue import Queue
+#import env
+import cv2
+import numpy as np
+import re
+import pdb
+
+# import converter.converter as converter
+#from backend.backend_acl import AclBackend
+
+
+
+last_timing = []
+last_device_timing = []
+
+
+def get_args():
+    """Parse commandline."""
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--dataset_path", default="./datasets/imagenet_10", help="path to the dataset")
+    parser.add_argument("--backend", default="acl", help="runtime to use")
+    parser.add_argument("--model", required=True, help="model file path")
+    parser.add_argument("--image_size",default='224,224,3',help="model inputs imagesize")
+    parser.add_argument("--inputs", default="input:0", help="model inputs nodes eg: data1:0 ")
+    parser.add_argument("--outputs",default="softmax:0", help="model outputs nodes list eg:fc1:0,fc2:0,fc3:0 ")
+
+    # below will override DNMetis rules compliant settings - don't use for official submission
+    parser.add_argument("--count", default=1000, type=int, help="dataset items to infer")
+    parser.add_argument("--precision", default="fp16", choices=["fp32", "fp16", "int8", "uint8"],
+                         help="precision mode, one of " + str(["fp32", "fp16", "int8", "uint8"]))
+    parser.add_argument("--feed", default=[], help="feed")
+    parser.add_argument("--image_list", default=[], help="image_list")
+    parser.add_argument("--label_list", default=[], help="label_list")
+    parser.add_argument("--cfg_path",default="./backend_cfg/built-in_config.txt")
+    args = parser.parse_args()
+
+    # don't use defaults in argparser. Instead we default to a dict, override that with a profile
+    # and take this as default unless command line give
+
+
+    #if args.image_size is None:
+    #    args.image_size = SUPPORTED_DATASETS[args.dataset][3]['image_size']
+
+    if args.inputs:
+        args.inputs = args.inputs.split(",")
+    if args.outputs:
+        args.outputs = args.outputs.split(",")
+    if args.image_size:
+        args.image_size = list(map(int, args.image_size.split(",")))
+
+    return args
+
+def get_backend(backend):
+    if backend == "acl":
+        from backend.backend_acl import AclBackend
+        backend = AclBackend()
+    return backend
+
+def resize_with_aspectratio(img, out_height, out_width, scale=87.5, inter_pol=cv2.INTER_LINEAR):
+    height, width = img.shape[:2]
+    new_height = int(100. * out_height / scale)
+    new_width = int(100. * out_width / scale)
+    if height > width:
+        w = new_width
+        h = int(new_height * height / width)
+    else:
+        h = new_height
+        w = int(new_width * width / height)
+    img = cv2.resize(img, (w, h), interpolation=inter_pol)
+    return img
+
+def center_crop(img, out_height, out_width):
+    height, width = img.shape[:2]
+    left = int((width - out_width) / 2)
+    right = int((width + out_width) / 2)
+    top = int((height - out_height) / 2)
+    bottom = int((height + out_height) / 2)
+    img = img[top:bottom, left:right]
+    return img
+def pre_process_noisy(img, dims=None, precision="fp32"):
+    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
+    output_height, output_width, _ = dims
+    cv2_interpol = cv2.INTER_CUBIC
+    img = resize_with_aspectratio(img, output_height, output_width, inter_pol=cv2_interpol)
+    img = center_crop(img, output_height, output_width)
+    MEAN_RGB = [0.485 * 255, 0.456 * 255, 0.406 * 255]
+    STDDEV_RGB = [0.229 * 255, 0.224 * 255, 0.225 * 255]
+
+    if precision=="fp32":
+        img = np.asarray(img, dtype='float32')
+    if precision=="fp16":
+        img = np.asarray(img, dtype='float16')
+
+    means = np.array([0.485 * 255, 0.456 * 255, 0.406 * 255], dtype=np.float32)
+    img -= means
+    stddev = np.array([0.229 * 255, 0.224 * 255, 0.225 * 255], dtype=np.float32)
+    img /= stddev
+    return img
+
+
+
+def preprocess_dataset(args,offset=0):
+    with open(args.dataset_path + '/val_map.txt', 'r') as f:
+        for s in f:
+            image_name, label = re.split(r"\s+", s.strip())
+            src = os.path.join(args.dataset_path, image_name)
+            img_org = cv2.imread(src)
+            processed_img = pre_process_noisy(img_org, dims=args.image_size, precision = args.precision)
+            args.feed.append(processed_img)
+            args.image_list.append(image_name)
+            args.label_list.append(int(label)+offset)
+
+
+def main():
+    good = 0
+    total = 0
+    #args
+    args = get_args()
+
+    # find backend
+    backend = get_backend(args.backend)
+
+    # load model to backend
+    model = backend.load(args)
+    #
+    # preprocess_dataset
+    preprocess_dataset(args,offset=0)
+
+    #start inference:
+    for i in range(len(args.feed)):
+        predictions = backend.predict(args.feed[i])
+        #print(args.feed[i].shape)
+        print('img_orig:',args.image_list[i],'label:',args.label_list[i],'predictions:',np.argmax(predictions),'\n')
+        if args.label_list[i] == np.argmax(predictions):
+            good +=1
+        total +=1
+    print('Predict total jpeg:',len(args.image_list),' Accuracy: ',good / total)
+
+    backend.unload()
+
+if __name__ == "__main__":
+    main()

dnmetis/requirements.txt

@@ -0,0 +1,3 @@
+pip3.7.5 install python-opencv
+cd backend_C++/dnmetis_backend
+pip3.7.5 setup.py install

dnmetis/run_inference.sh

@@ -0,0 +1,4 @@
+python3.7 main.py --model=./model/efficientnet-b8.om --image_size='672,672,3' --precision=fp16
+
+
+

dnmetis/version.py

@@ -0,0 +1,2 @@
+
+version = '1.0'

faster_install/for_1.7x.0.0/README.md

@@ -41,7 +41,7 @@
 
 执行脚本。
 
- **bash faster-install.sh** 
+ **bash faster_install.sh** 
 
 2、安装完成之后会显示以下界面。选择下图中红框选项（第三行）并点击OK。
 

faster_install/for_1.7x.0.0/README_EN.md

@@ -42,7 +42,7 @@ Copy the **faster_install.sh** script from the **faster-install** directory to t
 
 Run the script.
 
-**bash faster-install.sh**
+**bash faster_install.sh**
 
 2. After the installation is complete, the following page is displayed. Select the option (the third line) in the red box in the following figure and click **OK**.
 

img2bin/README.md

@@ -9,7 +9,7 @@ img2bin能够生成模型推理所需的输入数据，以.bin格式保存。当
 ## 前提条件  
 
 - 脚本可在Centos和Ubuntu环境下使用,只支持x86架构。  
-- 脚本支持python2和python3.7.5(MindStudio依赖python3.7.5)。
+- 脚本支持python2和python3。
 - 如未安装opencv-python，第一次使用，脚本会自动安装。
 
 ## 获取脚本
@@ -39,7 +39,7 @@ cd $HOME/AscendProjects/img2bin
 python2 img2bin.py -i ./images -w 416 -h 416 -f BGR -a NHWC -t uint8 -m [104,117,123] -c [1,1,1] -o ./out
 ```
 ```
-python3.7.5 img2bin.py -i ./images -w 416 -h 416 -f BGR -a NHWC -t uint8 -m [104,117,123] -c [1,1,1] -o ./out
+python3 img2bin.py -i ./images -w 416 -h 416 -f BGR -a NHWC -t uint8 -m [104,117,123] -c [1,1,1] -o ./out
 ```
 ### 第二类：
 - 第二类数据，需要新建一个文件，文件模板为test.txt，"input_node"为数据，"shape"为数据的shape信息。  
@@ -51,7 +51,7 @@ python3.7.5 img2bin.py -i ./images -w 416 -h 416 -f BGR -a NHWC -t uint8 -m [104
 python2 img2bin.py -i ./test.txt -t uint8 -o ./out
 ```
 ```
-python3.7.5 img2bin.py -i ./test.txt -t uint8 -o ./out
+python3 img2bin.py -i ./test.txt -t uint8 -o ./out
 ```
 
 ## 参数说明

img2bin/README_EN.md

@@ -9,7 +9,7 @@ Proportional resizing and zero-padding are performed on the images.
 ## Prerequisites
 
 - The script can be used in the CentOS and Ubuntu environments and supports only the x86 architecture.
-- The script supports Python 2 and Python 3.7.5 (MindStudio depends on python3.7.5).
+- The script supports Python 2 and Python 3.
 - If OpenCV-Python is not installed, the script will automatically install it in the initial use.
 
 ## Obtaining the Script
@@ -43,7 +43,7 @@ python2 img2bin.py -i ./images -w 416 -h 416 -f BGR -a NHWC -t uint8 -m [104,117
 ```
 
 ```
-python3.7.5 img2bin.py -i ./images -w 416 -h 416 -f BGR -a NHWC -t uint8 -m [104,117,123] -c [1,1,1] -o ./out
+python3 img2bin.py -i ./images -w 416 -h 416 -f BGR -a NHWC -t uint8 -m [104,117,123] -c [1,1,1] -o ./out
 ```
 
 ### Second type of data: text
@@ -58,7 +58,7 @@ python2 img2bin.py -i ./test.txt -t uint8 -o ./out
 ```
 
 ```
-python3.7.5 img2bin.py -i ./test.txt -t uint8 -o ./out
+python3 img2bin.py -i ./test.txt -t uint8 -o ./out
 ```
 
 ## Parameter Description

img2bin/img2bin.py

@@ -64,7 +64,7 @@ except:
                     print('[ERROR] install opencv-python failed,please check env.')
                     exit(0)
         else:
-            ret = os.system('sudo python3.7.5 -m pip install opencv-python')
+            ret = os.system('sudo python3 -m pip install opencv-python')
             if ret != 0:
                 print('[ERROR] install opencv-python failed,please check env.')
                 exit(0)
@@ -80,7 +80,7 @@ except:
     if sys.version_info.major == 2:
         os.system('pip2 install numpy')
     else:
-        os.system('python3.7.5 -m pip install numpy')
+        os.system('python3 -m pip install numpy')
     import numpy as np 
 
 def get_args():

makesd/for_1.7x.0.0/README.md

@@ -10,15 +10,15 @@
 
 - 开发者板操作系统镜像包：ubuntu-18.04.xxserver-arm64.iso，[点击跳转](http://cdimage.ubuntu.com/ubuntu/releases/18.04/release/)
 
-- 制卡软件包：Ascend310-firmware-xxx-minirc.run、Ascend310-aicpu_kernels-xxx-minirc.tar.gz、Ascend-acllib-xxx-ubuntu18.04.aarch64-minirc.run，[点击跳转](https://www.huaweicloud.com/ascend/resource/Software)
+- 制卡软件包：Ascend310-driver-xxx-ubuntu18.04.aarch64-minirc.tar.gz、Ascend310-aicpu_kernels-xxx-minirc.tar.gz、Ascend-acllib-xxx-ubuntu18.04.aarch64-minirc.run，[点击跳转](https://www.huaweicloud.com/ascend/resource/Software)
 
 ## 制卡步骤
 
 **制卡之前需要先将环境准备好**
 
 1. 如下图下载制卡需要的软件包  
-	下载地址：**https://www.huaweicloud.com/ascend/resource/Software。**  
-	![下载制卡包](https://images.gitee.com/uploads/images/2020/0729/140246_96db39de_5395865.png)
+	下载地址：**https://www.huaweicloud.com/ascend/resource/Software**  
+	![下载制卡包](https://images.gitee.com/uploads/images/2020/1026/114359_c5d3e9d6_5395865.png "屏幕截图.png")
 
 2. 下载制卡脚本和ubuntu18.04-arm.iso。  
 	在本地环境普通用户（以ascend用户为例）的$HOME目录下执行以下命令，下载tools仓代码。  

msame/src/model_process.cpp

@@ -440,137 +440,166 @@ void ModelProcess::OutputModelResult(std::string& s, std::string& modelName, siz
         }
         if (f_isTXT) {
             ofstream outstr(times + "/" + modelName + "_output_" + to_string(i) + ".txt", ios::out);
+	        int amount_onebatch = 1;
+	        for (int j = 1; j < dim->dimCount; j++) {
+                    amount_onebatch *= dim->dims[j];
+		    }
             switch (datatype) {
             case 0:
                 for (int i = 0; i < len / sizeof(float); i++) {
                     float out = *((float*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+		                if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+		            }
                 }
                 break;
             case 1:
                 for (int i = 0; i < len / sizeof(aclFloat16); i++) {
                     aclFloat16 out = *((aclFloat16*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+                        if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
                     }
-                }
+
+		        }
                 break;
             case 2:
                 for (int i = 0; i < len / sizeof(int8_t); i++) {
                     int8_t out = *((int8_t*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+                        if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
                     }
-                }
+		        }
                 break;
             case 3:
                 for (int i = 0; i < len / sizeof(int); i++) {
                     int out = *((int*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+			            if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+   		            }
                 }
                 break;
             case 4:
                 for (int i = 0; i < len / sizeof(uint8_t); i++) {
                     uint8_t out = *((uint8_t*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+		                if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+		            }
                 }
                 break;
             case 6:
                 for (int i = 0; i < len / sizeof(int16_t); i++) {
                     int16_t out = *((int16_t*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+		                if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+	                }
                 }
                 break;
             case 7:
                 for (int i = 0; i < len / sizeof(uint16_t); i++) {
                     uint16_t out = *((uint16_t*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+			            if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+		            }
                 }
                 break;
             case 8:
                 for (int i = 0; i < len / sizeof(uint32_t); i++) {
                     uint32_t out = *((uint32_t*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+		            if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+		            }
                 }
                 break;
             case 9:
                 for (int i = 0; i < len / sizeof(int64_t); i++) {
                     int64_t out = *((int64_t*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+			        if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+		            }
                 }
                 break;
             case 10:
                 for (int i = 0; i < len / sizeof(uint64_t); i++) {
                     uint64_t out = *((uint64_t*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+		                if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+		            }
                 }
                 break;
             case 11:
                 for (int i = 0; i < len / sizeof(double); i++) {
                     double out = *((double*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+		                if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+		            }
                 }
                 break;
             case 12:
                 for (int i = 0; i < len / sizeof(bool); i++) {
                     int out = *((bool*)outData + i);
                     outstr << out << " ";
-                    for (int j = 0; j < dim->dimCount; j++) {
-                        if (i != 0 && i % dim->dims[j] == 0 && dim->dims[j] > 10) {
+                    if (i != 0 && (i + 1) % amount_onebatch == 0 && i != len / sizeof(float)-1){
+                        outstr << "\n\n";
+                    } else{
+	                    if ((i + 1) % 100 == 0 && i != len / sizeof(float)-1){
                             outstr << "\n";
                         }
-                    }
+		            }
                 }
                 break;
             default: