NCNN在MangoPi MQ Quad上的部署和测试

本来继上集（【芒果派MangoPi MQ Quad】利用MangoPi MQ Quad部署一个网络摄像头-电子发烧友网 (elecfans.com)）后，继续写公网视频传输，后来一直没找到合适的公网穿透工具，找了几个要么不免费，要么各种验证，还是放弃吧。

得益于四核A53的性能，NCNN跑起来应该问题不大，本文主要介绍NCNN在MangoPi MQ Quad上的部署和测试。

一、NCNN介绍

ncnn 是一个为手机端极致优化的高性能神经网络前向计算框架。 ncnn 从设计之初深刻考虑手机端的部署和使用。 无第三方依赖，跨平台，手机端 cpu 的速度快于目前所有已知的开源框架。 基于 ncnn，开发者能够将深度学习算法轻松移植到手机端高效执行， 开发出人工智能 APP，将 AI 带到你的指尖。 ncnn 目前已在腾讯多款应用中使用，如：QQ，Qzone，微信，天天 P 图等。

ncnn: ncnn ncnn 是腾讯优图实验室首个开源项目，是一个为手机端极致优化的高性能神经网络前向计算框架 (gitee.com)

二、源码下载&编译

（一）源码下载

NCNN的github（github.com/Tencent/ncnn）拉源码。

root@orangepizero2:~# git clone https://github.com/Tencent/ncnn.git
Cloning into 'ncnn'...
remote: Enumerating objects: 31640, done.
remote: Counting objects: 100% (5982/5982), done.
remote: Compressing objects: 100% (498/498), done.
remote: Total 31640 (delta 5754), reused 5489 (delta 5484), pack-reused 25658
Receiving objects: 100% (31640/31640), 22.19 MiB | 2.33 MiB/s, done.
Resolving deltas: 100% (26851/26851), done.
Updating files: 100% (3195/3195), done.

（二）源码编译

考虑到四核A53的能力，加上NCNN本身也支持板上直接编译，所以就不去PC上搞交叉编译了。

Debian包自带gcc，所以就不需要在MangoPi MQ Quad编译安装了，顶多更新下。

按照下面顺序：

cd <  ncnn-root-dir  >
mkdir -p build-aarch64-linux-gnu
cd build-aarch64-linux-gnu
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchains/aarch64-linux-gnu.toolchain.cmake ..
make -j$(nproc)

操作及log如下：

root@orangepizero2:~/ncnn/build-aarch64-linux-gnu# cmake -DCMAKE_TOOLCHAIN_FILE=../toolchains/aarch64-linux-gnu.toolchain.cmake ..
-- CMAKE_TOOLCHAIN_FILE = /root/ncnn/toolchains/aarch64-linux-gnu.toolchain.cmake
-- CMAKE_INSTALL_PREFIX = /root/ncnn/build-aarch64-linux-gnu/install
-- NCNN_VERSION_STRING = 1.0.20230721
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/aarch64-linux-gnu-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test NCNN_COMPILER_SUPPORT_GNU_INLINE_ASM
-- Performing Test NCNN_COMPILER_SUPPORT_GNU_INLINE_ASM - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM_VFPV4
-- Performing Test NCNN_COMPILER_SUPPORT_ARM_VFPV4 - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM82_FP16
-- Performing Test NCNN_COMPILER_SUPPORT_ARM82_FP16 - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM82_DOTPROD
-- Performing Test NCNN_COMPILER_SUPPORT_ARM82_DOTPROD - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM82_FP16FML
-- Performing Test NCNN_COMPILER_SUPPORT_ARM82_FP16FML - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM84_BF16
-- Performing Test NCNN_COMPILER_SUPPORT_ARM84_BF16 - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM84_I8MM
-- Performing Test NCNN_COMPILER_SUPPORT_ARM84_I8MM - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVE
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVE - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVE2
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVE2 - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVEBF16
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVEBF16 - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVEI8MM
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVEI8MM - Success
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVEF32MM
-- Performing Test NCNN_COMPILER_SUPPORT_ARM86_SVEF32MM - Success
-- Target arch: arm 64bit
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Configuring done
-- Generating done
-- Build files have been written to: /root/ncnn/build-aarch64-linux-gnu
root@orangepizero2:~/ncnn/build-aarch64-linux-gnu# make -j4
Scanning dependencies of target ncnn-generate-spirv
[  0%] Built target ncnn-generate-spirv
Scanning dependencies of target ncnn
[  1%] Building CXX object src/CMakeFiles/ncnn.dir/benchmark.cpp.o
[  1%] Building CXX object src/CMakeFiles/ncnn.dir/allocator.cpp.o
[  2%] Building CXX object src/CMakeFiles/ncnn.dir/blob.cpp.o
[  2%] Building CXX object src/CMakeFiles/ncnn.dir/c_api.cpp.o
[  3%] Building CXX object src/CMakeFiles/ncnn.dir/command.cpp.o
[  3%] Building CXX object src/CMakeFiles/ncnn.dir/cpu.cpp.o
[  4%] Building CXX object src/CMakeFiles/ncnn.dir/datareader.cpp.o
[  4%] Building CXX object src/CMakeFiles/ncnn.dir/gpu.cpp.o
[  5%] Building CXX object src/CMakeFiles/ncnn.dir/layer.cpp.o
[  5%] Building CXX object src/CMakeFiles/ncnn.dir/mat.cpp.o
....
[ 99%] Building CXX object src/CMakeFiles/ncnn.dir/layer/copyto.cpp.o
[100%] Linking CXX static library libncnn.a
[100%] Built target ncnn
Scanning dependencies of target benchncnn
[100%] Building CXX object benchmark/CMakeFiles/benchncnn.dir/benchncnn.cpp.o
[100%] Linking CXX executable benchncnn
[100%] Built target benchncnn
root@orangepizero2:~/ncnn/build-aarch64-linux-gnu# ll
total 56
drwxr-xr-x 3 root root  4096 Jul 21 15:36 benchmark
-rw-r--r-- 1 root root 28856 Jul 21 15:19 CMakeCache.txt
drwxr-xr-x 5 root root  4096 Jul 21 15:36 CMakeFiles
-rw-r--r-- 1 root root  1917 Jul 21 15:19 cmake_install.cmake
-rw-r--r-- 1 root root  7466 Jul 21 15:19 Makefile
drwxr-xr-x 3 root root  4096 Jul 21 15:35 src

等了大约10分钟，毕竟编译速度比不上X86，检查下生成的可执行文件，正常：

root@orangepizero2:~/ncnn/build-aarch64-linux-gnu# file benchmark/benchncnn
benchmark/benchncnn: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1,x 3.7.0, not stripped

三、测试

将生成的可执行文件拷贝到原目录的benchmark/文件夹中，因为这里面有测试需要的模型数据。

root@orangepizero2:~/ncnn/build-aarch64-linux-gnu/benchmark# cd ../../benchmark/
root@orangepizero2:~/ncnn/benchmark# ls
alexnet.param    efficientnet_b0.param    mnasnet.param             mobilenet_v2.param        proxylessnasnet.param  resnet50_int8.param    sq
benchncnn        efficientnetv2_b0.param  mobilenet_int8.param      mobilenetv2_yolov3.param  README.md              resnet50.param         sq
benchncnn.cpp    FastestDet.param         mobilenet.param           mobilenet_v3.param        regnety_400m.param     shufflenet.param       sq
blazeface.param  googlenet_int8.param     mobilenet_ssd_int8.param  mobilenet_yolo.param      resnet18_int8.param    shufflenet_v2.param    vg
CMakeLists.txt   googlenet.param          mobilenet_ssd.param       nanodet_m.param           resnet18.param         squeezenet_int8.param  vg

执行一下，几个模型测试都跑个遍：

责任编辑：彭菁