FastDeploy簡(jiǎn)介

FastDeploy是一款全場(chǎng)景、易用靈活、極致高效的AI推理部署工具， 支持云邊端部署。提供超過 160+Text，Vision， Speech和跨模態(tài)模型開箱即用的部署體驗(yàn)，并實(shí)現(xiàn)端到端的推理性能優(yōu)化。包括 物體檢測(cè)、字符識(shí)別（OCR）、人臉、人像扣圖、多目標(biāo)跟蹤系統(tǒng)、NLP、Stable Difussion文圖生成、TTS 等幾十種任務(wù)場(chǎng)景，滿足開發(fā)者多場(chǎng)景、多硬件、多平臺(tái)的產(chǎn)業(yè)部署需求。

準(zhǔn)備工作本文的FastDeploy適配過程需要準(zhǔn)備如下：·凌蒙派-RK3568開發(fā)板（即需FastDeploy適配的設(shè)備終端）·Ubuntu（即建立于虛擬機(jī)的Linux編譯環(huán)境）目前，我已將FastDeploy適配到凌蒙派開發(fā)板上，可用于目標(biāo)檢測(cè)、人臉檢測(cè)、人臉識(shí)別、人臉對(duì)齊、圖像分割、OCR等領(lǐng)域，這將大大提高凌蒙派開發(fā)板在邊緣計(jì)算方面的能力。

編譯步驟

我們推薦在PC上進(jìn)行交叉編譯（即在Ubuntu進(jìn)行交叉編譯）。

git clone https://github.com/PaddlePaddle/FastDeploy.gitcd FastDeploy# 如果您使用的是develop分支輸入以下命令git checkout developmkdir build && cd buildcmake .. -DCMAKE_C_COMPILER=/home/zbc/opt/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc \-DCMAKE_CXX_COMPILER=/home/zbc/opt/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++ \-DCMAKE_TOOLCHAIN_FILE=./../cmake/toolchain.cmake \-DTARGET_ABI=arm64 \-DENABLE_ORT_BACKEND=OFF \-DENABLE_RKNPU2_BACKEND=ON \-DENABLE_VISION=ON \-DRKNN2_TARGET_SOC=RK356X \-DCMAKE_INSTALL_PREFIX=${PWD}/fastdeploy-0.0.0make -j12make install

目標(biāo)檢測(cè)模型速度表

為了方便大家選擇最適合自己的模型，我們選取了目前最流行的幾個(gè)模型，并整理了模型速度表供大家快速瀏覽。以下測(cè)試速度均為端到端的速度。

Demo演示

FastDeploy提供了統(tǒng)一的接口，可以快速的切換模型，這里以Yolov5為例子，展示如何在凌蒙派RK3568上進(jìn)行目標(biāo)檢測(cè)。

編寫代碼

// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.//// Licensed under the Apache License, Version 2.0 (the "License");// you may not use this file except in compliance with the License.// You may obtain a copy of the License at//// http://www.apache.org/licenses/LICENSE-2.0//// Unless required by applicable law or agreed to in writing, software// distributed under the License is distributed on an "AS IS" BASIS,// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.// See the License for the specific language governing permissions and// limitations under the License.#include "fastdeploy/vision.h"void RKNPU2Infer(const std::string& model_file, const std::string& image_file) {auto option = fastdeploy::RuntimeOption();option.UseRKNPU2();auto format = fastdeploy::RKNN;auto model = fastdeploy::RKYOLOV5(model_file, option,format);auto im = cv::imread(image_file);fastdeploy::DetectionResult res;fastdeploy::TimeCounter tc;tc.Start();if (!model.Predict(im, &res)) {std::cerr << "Failed to predict." << std::endl;return;}auto vis_im = fastdeploy::VisDetection(im, res,0.5);tc.End();tc.PrintInfo("RKYOLOV5 in RKNN");std::cout << res.Str() << std::endl;cv::imwrite("vis_result.jpg", vis_im);std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;}int main(int argc, char* argv[]) {if (argc < 3) {std::cout<< "Usage: infer_demo path/to/model_dir path/to/image run_option, ""e.g ./infer_model ./picodet_model_dir ./test.jpeg"<< std::endl;return -1;}RKNPU2Infer(argv[1], argv[2]);return 0;}

編譯代碼

# 編譯mkdir buildcd buildcmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-0.0.0make -j4# 下載圖片wgethttps://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg# 運(yùn)行./infer_rkyolov5 ../Model/yolov5-s-relu/yolov5s_relu_tk2_RK356X_i8.rknn./000000014439.jpg

展示結(jié)果

輸入圖片

輸出圖片