簡介

相信大部分同學們都已了解或接觸過OpenAtom OpenHarmony（以下簡稱“OpenHarmony”）了，但你一定沒在OpenHarmony上實現過人臉識別功能，跟著本文帶你快速在OpenHarmony標準設備上基于SeetaFace2和OpenCV實現人臉識別。

項目效果

本項目實現了導入人臉模型、人臉框選和人臉識別三大功能，操作流程如下：

1.錄入頁面點擊右下角按鈕，跳轉拍攝頁面進行拍照；

2.選擇一張或多張人臉作為訓練模型，并設置對應的名字；

3.選擇一張未錄入的人臉圖片，點擊框選按鈕實現人臉圖片框選功能；

4.最后點擊識別，應用會對當前圖片進行匹配，最終在界面中顯示識別結果。

快速上手

設備端開發

設備端通過OpenCV對圖像進行處理并通過Seetaface2對圖形數據進行人臉頭像的識別，最終輸出對應的NAPI接口提供給應用端調用。因此設備端開發主要涉及到OpenCV和Seetaface2的移植以及NAPI接口的開發。

OpenCV庫移植

OpenCV是一個功能非常強大的開源計算機視覺庫。此庫已由知識體系工作組移植到了OpenHarmony中，后期還會將此庫合入到主倉。在此庫上主倉之前，我們只需要以下幾個步驟就可以實現OpenCV的移植使用。

1.通過以下命令下載已經移植好的OpenCV

git clone git@gitee.com:zhong-luping/ohos_opencv.git

2.將OpenCV拷貝到OpenHarmony目錄的third_party下

cp -raf opencv ~/openharmony/third_party/

3.適當裁剪編譯選項

打開OpenCV目錄下的BUILD.gn，如下：

不需要video以及flann功能，將對應的模塊注釋即可。

import("http://build/ohos.gni")group("opencv") {    deps = [        "http://third_party/opencv/modules/core:opencv_core",      //  "http://third_party/opencv/modules/flann:opencv_flann",        "http://third_party/opencv/modules/imgproc:opencv_imgproc",        "http://third_party/opencv/modules/ml:opencv_ml",        "http://third_party/opencv/modules/photo:opencv_photo",        "http://third_party/opencv/modules/dnn:opencv_dnn",        "http://third_party/opencv/modules/features2d:opencv_features2d",        "http://third_party/opencv/modules/imgcodecs:opencv_imgcodecs",        "http://third_party/opencv/modules/videoio:opencv_videoio",        "http://third_party/opencv/modules/calib3d:opencv_calib3d",        "http://third_party/opencv/modules/highgui:opencv_highgui",        "http://third_party/opencv/modules/objdetect:opencv_objdetect",        "http://third_party/opencv/modules/stitching:opencv_stitching",        "http://third_party/opencv/modules/ts:opencv_ts",     //   "http://third_party/opencv/modules/video:opencv_video",       "http://third_party/opencv/modules/gapi:opencv_gapi",    ]}

4.添加依賴子系統的part_name，編譯框架子系統會將編譯出的庫拷貝到系統文件中。

此項目中我們新建了一個SeetaFaceApp的子系統，該子系統中命名part_name為SeetafaceApi，所以我們需要在對應模塊中的BUILD.gn中加上part_name="SeetafaceApi"

以module/core為例：

ohos_shared_library("opencv_core"){ sources = [ ... ]configs = [  ... ]deps = [ ... ]part_name = "SeetafaceApi"}

5. 編譯工程需要添加OpenCV的依賴。

在生成NAPI的BUILD.gn中添加以下依賴：

deps += [ "http://third_party/opencv:opencv" ]

至此，人臉識別中OpenCV的移植使用完成。

SeetaFace2庫移植

SeetaFace2是中科視拓開源的第二代人臉識別庫。包括了搭建一套全自動人臉識別系統所需的三個核心模塊，即：人臉檢測模塊FaceDetector、面部關鍵點定位模塊FaceLandmarker以及人臉特征提取與比對模塊 FaceRecognizer。

關于SeetaFace2的移植請參照文檔:SeetaFace2移植開發文檔。

NAPI接口開發

關于OpenHarmony中的NAPI開發，參考視頻：

OpenHarmony中napi的開發視頻教程。本文將重點講解NAPI接口如何實現OpenCV以及SeetaFace的調用。

1.人臉框獲取的NAPI接口的實現。

int GetRecognizePoints(const char *image_path)；

此接口主要是通過應用層輸入一張圖片，通過OpenCV的imread接口獲取到圖片數據，并通過人臉檢測模塊FaceDetector分析獲得圖片中所有的人臉矩形框（矩形框是以x,y,w,h的方式）并將人臉框矩形以數組的方式返回到應用層。

人臉框矩形獲取的主要代碼如下：

static int RecognizePoint(string image_path, FaceRect *rect, int num){    if (rect == nullptr) {        cerr << "NULL POINT!" << endl;        LOGE("NULL POINT! 
");        return -1;    }    seeta::Device device = seeta::CPU;    int id = 0;
    /* 設置人臉識別模型。*/    seeta::ModelSetting FD_model( "/system/usr/model/fd_2_00.dat", device, id );    seeta::ModelSetting FL_model( "/system/usr/model/pd_2_00_pts81.dat", device, id );
    seeta::FaceDetector FD(FD_model);    seeta::FaceLandmarker FL(FL_model);
    FD.set(seeta::PROPERTY_VIDEO_STABLE, 1);
    /* 讀取圖片數據 */    auto frame = imread(image_path);    seeta::ImageData simage = frame;    if (simage.empty()) {        cerr << "Can not open image: " << image_path << endl;        LOGE("Can not open image: %{public}s", image_path.c_str());        return -1;    }    /* 圖片數據進行人臉識別處理 ，獲取所有的人臉框數據對象*/    auto faces = FD.detect(simage);    if (faces.size <= 0) {        cerr << "detect " << image_path << "failed!" << endl;        LOGE("detect image: %s failed!", image_path.c_str());        return -1;    }    for (int i = 0; (i < faces.size && i < num); i++) {        /* 將所有人臉框對象數據以坐標形式輸出*/        auto &face = faces.data[i];        memcpy(&rect[i], &(face.pos), sizeof(FaceRect));    }    return faces.size;}

其中FD_model是人臉檢測模型，而FL_model是面部關鍵點定位模型(此模型分為5點定位和81點定位，本項目中使用的是81點定位模型)，這些模型從開源項目中免費獲取。

通過以上方式獲取到對應的人臉矩形框后，再將矩形框以數組的方式返回到應用端：

    string image = path;    p = (FaceRect *)malloc(sizeof(FaceRect) * MAX_FACE_RECT);    /* 根據圖片進行人臉識別并獲取人臉框坐標點 */    int retval = RecognizePoint(image, p, MAX_FACE_RECT);    if (retval <= napi_ok) {        LOGE("GetNapiValueString failed!");        free(p);        return result;    }      /*將所有坐標點以數組方式返回到應用端*/    for (int i = 0; i < retval; i++) {        int arry_int[4] = {p[i].x, p[i].y, p[i].w, p[i].h};        int arraySize = (sizeof(arry_int) / sizeof(arry_int[0]));        for (int j = 0; j < arraySize; j++) {            napi_value num_val;            if (napi_create_int32(env, arry_int[j], &num_val) != napi_ok) {                LOGE("napi_create_int32 failed!");                return result;            }            napi_set_element(env, array, i*arraySize + j, num_val);        }    }    if (napi_create_object(env, &result) != napi_ok) {        LOGE("napi_create_object failed!");        free(p);        return result;    }    if (napi_set_named_property(env, result, "recognizeFrame", array) != napi_ok) {        LOGE("napi_set_named_property failed!");        free(p);        return result;    }    LOGI("");    free(p);returnresult;

其中array是通過napi_create_array創建的一個NAPI數組對象，通過 napi_set_element將所有的矩形框數據保存到array對象中，最后通過 napi_set_named_property將array轉換成應用端可識別的對象類型result并將其返回。

2.人臉搜索識別初始化與逆初始化。

1.int FaceSearchInit()；

2.int FaceSearchDeinit()；

這2個接口主要是提供給人臉搜索以及識別調用的，初始化主要包含模型的注冊以及識別模塊的初始化：

static  int FaceSearchInit(FaceSearchInfo *info){    if (info == NULL) {        info = (FaceSearchInfo *)malloc(sizeof(FaceSearchInfo));        if (info == nullptr) {            cerr << "NULL POINT!" << endl;            return -1;        }    }
    seeta::Device device = seeta::CPU;    int id = 0;    seeta::ModelSetting FD_model( "/system/usr/model/fd_2_00.dat", device, id );    seeta::ModelSetting PD_model( "/system/usr//model/pd_2_00_pts5.dat", device, id );    seeta::ModelSetting FR_model( "/system/usr/model/fr_2_10.dat", device, id );
    info->engine = make_shared(FD_model, PD_model, FR_model, 2, 16);    info->engine->FD.set( seeta::PROPERTY_MIN_FACE_SIZE, 80);
    info->GalleryIndexMap.clear();
    return 0;}

而逆初始化就是做一些內存的釋放。

static void FaceSearchDeinit(FaceSearchInfo *info, int need_delete){    if (info != nullptr) {        if (info->engine != nullptr) {        }
        info->GalleryIndexMap.clear();        if (need_delete) {            free(info);            info = nullptr;        }    }}

3.人臉搜索識別注冊接口的實現。

int FaceSearchRegister(const char *value);

需要注意的是，該接口需要應用端傳入一個json數據的參數，主要包含注冊人臉的名字，圖片以及圖片個數，如{"name":"劉德華","sum":"2","image":{"11.jpg","12.jpg"}}。而解析參數的時候需要調用 napi_get_named_property對json數據的各個對象進行解析，具體代碼如下：

    napi_get_cb_info(env, info, &argc, &argv, &thisVar, &data);    napi_value object = argv;    napi_value value = nullptr;
    if (napi_get_named_property(env, object, (const char *)"name", &value) == napi_ok) {        char name[64] = {0};        if (GetNapiValueString(env, value, (char *)name, sizeof(name)) < 0) {            LOGE("GetNapiValueString failed!");            return result;        }        reg_info.name = name;    }    LOGI("name = %{public}s", reg_info.name.c_str());    if (napi_get_named_property(env, object, (const char *)"sum", &value) == napi_ok) {                if (napi_get_value_uint32(env, value, &sum) != napi_ok) {            LOGE("napi_get_value_uint32 failed!");            return result;        }    }    LOGI("sum = %{public}d", sum);    if (napi_get_named_property(env, object, (const char *)"image", &value) == napi_ok) {        bool res = false;        if (napi_is_array(env, value, &res) != napi_ok || res == false) {            LOGE("napi_is_array failed!");            return result;        }        for (int i = 0; i < sum; i++) {            char image[256] = {0};            napi_value imgPath = nullptr;            if (napi_get_element(env, value, i, &imgPath) != napi_ok) {                LOGE("napi_get_element failed!");                return result;            }            if (GetNapiValueString(env, imgPath, (char *)image, sizeof(image)) < 0) {                LOGE("GetNapiValueString failed!");                return result;            }            reg_info.path = image;            if (FaceSearchRegister(g_FaceSearch, reg_info) != napi_ok) {                retval = -1;                break;            }        }}

通過napi_get_cb_info獲取從應用端傳來的參數，并通過 napi_get_named_property獲取對應的name以及圖片個數，最后通過napi_get_element獲取圖片數組中的各個image，將name和image通過FaceSearchRegister接口將圖片和名字注冊到SeetaFace2模塊的識別引擎中。具體實現如下：

static int FaceSearchRegister(FaceSearchInfo &info, RegisterInfo &gegister){    if (info.engine == nullptr) {        cerr << "NULL POINT!" << endl;        return -1;    }
    seeta::ImageData image = cv::imread(gegister.path);    auto id = info.engine->Register(image);    if (id >= 0) {        info.GalleryIndexMap.insert(make_pair(id, gegister.name));    }
    return 0;}

注冊完數據后，后續可以通過該引擎來識別對應的圖片。

4.獲取人臉搜索識別結果接口的實現。

char *FaceSearchGetRecognize(const char *image_path);

該接口實現了通過傳入一張圖片，在識別引擎中進行搜索識別。如果識別引擎中有類似的人臉注冊，則返回對應人臉注冊時的名字，否則返回不識別(ignored)字樣。該方法是通過異步回調的方式實現的：

    // 創建async work，創建成功后通過最后一個參數(commandStrData->asyncWork)返回async work的handle    napi_value resourceName = nullptr;    napi_create_string_utf8(env, "FaceSearchGetPersonRecognizeMethod", NAPI_AUTO_LENGTH, &resourceName);    napi_create_async_work(env, nullptr, resourceName, FaceSearchRecognizeExecuteCB, FaceSearchRecognizeCompleteCB,            (void *)commandStrData, &commandStrData->asyncWork);
    // 將剛創建的async work加到隊列，由底層去調度執行napi_queue_async_work(env,commandStrData->asyncWork);

其中FaceSearchRecognizeExecuteCB實現了人臉識別：

static void FaceSearchRecognizeExecuteCB(napi_env env, void *data){    CommandStrData *commandStrData = dynamic_cast((CommandStrData *)data);    if (commandStrData == nullptr) {        HILOG_ERROR("nullptr point!", __FUNCTION__, __LINE__);        return;    }
    FaceSearchInfo faceSearch = *(commandStrData->mFaceSearch);    commandStrData->result = FaceSearchSearchRecognizer(faceSearch, commandStrData->filename);    LOGI("Recognize result : %s !", __FUNCTION__, __LINE__, commandStrData->result.c_str());}

FaceSearchRecognizeCompleteCB函數通過napi_resolve_deferred接口將識別結果返回到應用端。

static void FaceSearchRecognizeCompleteCB(napi_env env, napi_status status, void *data){    CommandStrData *commandStrData = dynamic_cast((CommandStrData *)data);    napi_value result;
    if (commandStrData == nullptr || commandStrData->deferred == nullptr) {        LOGE("nullptr", __FUNCTION__, __LINE__);        if (commandStrData != nullptr) {            napi_delete_async_work(env, commandStrData->asyncWork);            delete commandStrData;        }
        return;    }
    const char *result_str = (const char *)commandStrData->result.c_str();    if (napi_create_string_utf8(env, result_str, strlen(result_str), &result) != napi_ok) {        LOGE("napi_create_string_utf8 failed!", __FUNCTION__, __LINE__);        napi_delete_async_work(env, commandStrData->asyncWork);        delete commandStrData;        return;    }
    napi_resolve_deferred(env, commandStrData->deferred, result);    napi_delete_async_work(env, commandStrData->asyncWork);
    delete commandStrData;}

通過人臉特征提取與比對模塊，對傳入的數據與已注冊的數據進行對比，并通過返回對比的相似度來進行判斷當前人臉是否為可識別的，最后返回識別結果。具體實現代碼：

static string FaceSearchSearchRecognizer(FaceSearchInfo &info, string filename){    if (info.engine == nullptr) {        cerr << "NULL POINT!" << endl;        return "recognize error 0";    }    string name;    float threshold = 0.7f;    seeta::QualityAssessor QA;    auto frame = cv::imread(filename);    if (frame.empty()) {        LOGE("read image %{public}s failed!", filename.c_str());        return "recognize error 1!";    }    seeta::ImageData image = frame;    std::vector faces = info.engine->DetectFaces(image);
    for (SeetaFaceInfo &face : faces) {        int64_t index = 0;        float similarity = 0;
        auto points = info.engine->DetectPoints(image, face);
        auto score = QA.evaluate(image, face.pos, points.data());        if (score == 0) {            name = "ignored";        } else {            auto queried = info.engine->QueryTop(image, points.data(), 1, &index, &similarity);            // no face queried from database            if (queried < 1) continue;                // similarity greater than threshold, means recognized            if( similarity > threshold ) {                name = info.GalleryIndexMap[index];            }        }    }    LOGI("name : %{public}s 
", name.length() > 0 ? name.c_str() : "null");    return name.length() > 0 ? name : "recognize failed";}

至此，所有的NAPI接口已經開發完成。 5.NAPI庫編譯開發完NAPI接口后，我們需要將我們編寫的庫加入到系統中進行編譯，我們需要添加一個自己的子系統。

首先在庫目錄下新建一個ohos.build文件并添加以下代碼：

{    "subsystem": "SeetafaceApp",    "parts": {        "SeetafaceApi": {            "module_list": [               "http://seetaface:seetafaceapp_napi"            ],            "test_list": [ ]        }    }}

其次同一目錄新建一個BUILD.gn，將庫源文件以及對應的依賴加上，如下：

import("http://build/ohos.gni")
config("lib_config") {    cflags_cc = [        "-frtti",        "-fexceptions",        "-DCVAPI_EXPORTS",        "-DOPENCV_ALLOCATOR_STATS_COUNTER_TYPE=int",        "-D_USE_MATH_DEFINES",        "-D__OPENCV_BUILD=1",        "-D__STDC_CONSTANT_MACROS",        "-D__STDC_FORMAT_MACROS",        "-D__STDC_LIMIT_MACROS",        "-O2",        "-Wno-error=header-hygiene",    ]}
ohos_shared_library("seetafaceapp_napi") {    sources = [        "app.cpp",    ]
    include_dirs = [        "./",        "http://third_party/opencv/include",        "http://third_party/opencv/common",        "http://third_party/opencv/modules/core/include",        "http://third_party/opencv/modules/highgui/include",        "http://third_party/opencv/modules/imgcodecs/include",        "http://third_party/opencv/modules/imgproc/include",        "http://third_party/opencv/modules/calib3d/include",        "http://third_party/opencv/modules/dnn/include",        "http://third_party/opencv/modules/features2d/include",        "http://third_party/opencv/modules/flann/include",        "http://third_party/opencv/modules/ts/include",        "http://third_party/opencv/modules/video/include",        "http://third_party/opencv/modules/videoio/include",        "http://third_party/opencv/modules/ml/include",        "http://third_party/opencv/modules/objdetect/include",        "http://third_party/opencv/modules/photo/include",        "http://third_party/opencv/modules/stitching/include",        "http://third_party/SeetaFace2/FaceDetector/include",        "http://third_party/SeetaFace2/FaceLandmarker/include",        "http://third_party/SeetaFace2/FaceRecognizer/include",        "http://third_party/SeetaFace2/QualityAssessor/include",        "http://base/accessibility/common/log/include",        "http://base/hiviewdfx/hilog_lite/interfaces/native/innerkits"    ]
    deps = [ "http://foundation/ace/napi:ace_napi" ]    deps += [ "http://third_party/opencv:opencv" ]    deps += [ "http://third_party/SeetaFace2:SeetaFace2" ]
    external_deps = [        "hiviewdfx_hilog_native:libhilog",    ]
    configs = [       ":lib_config"    ]
    # 指定庫生成的路徑    relative_install_dir = "module"    # 子系統及其組件，后面會引用    subsystem_name = "SeetafaceApp"    part_name = "SeetafaceApi"}

添加完對應的文件后，我們需要將我們的子系統添加到系統中進行編譯，打開build/subsystem_config.json并在最后添加以下代碼：

  "SeetafaceApp": {    "path": "seetaface",    "name": "SeetafaceApp"}

添加完子系統再修改產對應的品配置打開productdefine/common/products/rk3568.json并在最后添加以下代碼：

"SeetafaceApp:SeetafaceApi":{}

做完以上修改后我們就可以通過以下命令直接編譯NAPI的庫文件了：

./build.sh --product-name rk3568 --ccache

參考RK3568快速上手-鏡像燒錄完成燒錄即可。

應用端開發

在完成設備NAPI功能開發后，應用端通過調用NAPI組件中暴露給應用的人臉識別接口，即可實現對應功能。接下來就帶著大家使用NAPI實現人臉識別功能。

開發準備

1.下載DevEco Studio 3.0 Beta4；

2.搭建開發環境，參考開發準備；

3.了解屬性eTS開發，參考eTS語言快速入門；

SeetaFace2初始化

1.首先將SeetaFace2 NAPI接口聲明文件放置于SDK目錄/api下；

2.然后導入SeetaFace2 NAPI模塊；ck-start/star

3.調用初始化接口；

// 首頁實例創建后async aboutToAppear() {  await StorageUtils.clearModel();  CommonLog.info(TAG,'aboutToAppear')  // 初始化人臉識別  let res = SeetafaceApp.FaceSearchInit()  CommonLog.info(TAG,`FaceSearchInit res=${res}`)  this.requestPermissions()}
// 請求權限requestPermissions(){  CommonLog.info(TAG,'requestPermissions')  let context = featureAbility.getContext()  context.requestPermissionsFromUser(PERMISSIONS, 666,(res)=>{    this.getMediaImage()  })}

獲取所有人臉圖片

通過文件管理模塊fileio和媒體庫管理mediaLibrary，獲取指定應用數據目錄下所有的圖片信息，并將路徑賦值給faceList，faceList數據用于Image組件提供url進行加載圖片。

// 獲取所有圖片async getMediaImage(){  let context = featureAbility.getContext();  // 獲取本地應用沙箱路徑  let localPath = await context.getOrCreateLocalDir()  CommonLog.info(TAG, `localPath:${localPath}`)  let facePath = localPath + "/files"  // 獲取所有照片  this.faceList = await FileUtil.getImagePath(facePath)}

設置人臉模型

獲取選中的人臉圖片地址和輸入的名字，調用SeetafaceApp.FaceSearchRegister(params)進行設置人臉模型。其中參數params由name名字、image圖片地址集合和sum圖片數量組成。

async submit(name) {    if (!name || name.length == 0) {        CommonLog.info(TAG, 'name is empty')        return    }    let selectArr = this.faceList.filter(item => item.isSelect)    if (selectArr.length == 0) {        CommonLog.info(TAG, 'faceList is empty')        return    }    // 關閉彈窗    this.dialogController.close()    try {        let urls = []        let files = []        selectArr.forEach(item => {            let source = item.url.replace('file://', '')            CommonLog.info(TAG, `source:${source}`)            urls.push(item.url)            files.push(source)        })
        // 設置人臉識別模型參數        let params = {            name: name,            image: files,            sum: files.length        }        CommonLog.info(TAG, 'FaceSearchRegister' + JSON.stringify(params))        let res = SeetafaceApp.FaceSearchRegister(params)        CommonLog.info(TAG, 'FaceSearchRegister res ' + res)        // 保存已設置的人臉模型到輕量存儲        let data = {            name:name,            urls:urls        }        let modelStr = await StorageUtils.getModel()        let modelList = JSON.parse(modelStr)        modelList.push(data)        StorageUtils.setModel(modelList)        router.back()    } catch (err) {        CommonLog.error(TAG, 'submit fail ' + err)    }}

實現框選人臉

調用SeetafaceApp.GetRecognizePoints傳入當前圖片地址，獲取到人臉左上和右下坐標，再通過CanvasRenderingContext2D對象繪畫出人臉框。

// 框選當前人臉select(){    try{        CommonLog.info(TAG,`select start`)        // 獲取人臉左上和右下坐標        let res = SeetafaceApp.GetRecognizePoints(this.url)        CommonLog.info(TAG,`select start1` + JSON.stringify(res))        let points = res.recognizeFrame as Array        CommonLog.info(TAG,`select success ${JSON.stringify(res)}`)        let faceNumber = Math.floor(points.length/4)        // 框選出所有人臉        this.context.lineWidth = 5        this.context.strokeStyle = Color.Yellow.toString()        for (let index = 0; index < faceNumber; index++) {            let baseNumber = 4 * index            this.context.strokeRect(points[baseNumber],points[1 + baseNumber],points[2 + baseNumber],points[3 + baseNumber])            this.context.save()        }        CommonLog.info(TAG,`strokeRect success `)    }catch(err){        CommonLog.error(TAG,`select fail ${JSON.stringify(err)}`)    }}

實現人臉識別

調用SeetafaceApp.FaceSearchGetRecognize(url)，傳入圖片地址對人臉進行識別并返回對應識別出來的名字。

// 人臉識別recognize(){    SeetafaceApp.FaceSearchGetRecognize(this.url).then(res=>{        CommonLog.info(TAG,'recognize suceess' + JSON.stringify(res))        if(res && res != 'ignored' && res != "recognize failed" && res != 'recognize error 1!'){            // 賦值識別到的人物模型            this.name = res        }else{            this.name = '未識別到該模型'        }    }).catch(err=>{        CommonLog.error(TAG,'recognize' + err)        this.name = '未識別到該模型'    })}

審核編輯：湯梓紅