Load two models nef into KL720

Tri

My platform:

++ USB dongle

++ plugin to my board.

1. I try to load two models into KL720 but the result are fails.

Do we load two models on USB KL720 ?

2. I also try to combine two nef model into one models by command:

./kneron_nef_utils --combine_nef path/to/file_A.nef path/to/file_B.nef

Although the result is finish, but can see file_A.nef , i use command:

./kneron_nef_utils --info path/to/file.nef

Could you help us how to combine two nef models ?

Ethon Lin

Hello,

Please refer to the following description for your question.

1. It's recommended to combine different models into an NEF file before inference. You can then select the inference model by its ID.
2. The issue with NEF combining occurred previously due to a toolchain version conflict. You can try using the latest version of the toolchain to re-generate the NEF models and combine them again. If the issue persists, please provide both NEF files for debugging.

Tri

models.tgz

Thank for your support!

//The issue with NEF combining occurred previously due to a toolchain version conflict. 

//You can try using the latest version of the toolchain to re-generate the NEF models and combine them again. If the issue persists, please provide both NEF files for debugging.

Yes, i also try to re-generate two NEF models, then combine them again. But only see the first NEF model.

Could you help to review the issue ?

I attached two nef models.

Thank very much.

Maria Chen

Hi Tri,

The NEF files can't be combined because the versions of the NEF files are different: one is v0, and the other one is v1. You could check the versions of the NEF files with:

./kneron_nef_utils --info path/to/file.nef

To resolve this, you could edit the flatbuffer to False in batch compile to disable flatbuffer mode so that it compiles v0 NEF files.

(You may use either ktc.compile or ktc.encrypt_compile)

Then, you should be able to combine both v0 NEF files.

Reference: 5. Compilation - Document Center (kneron.com)

Tri

Thank for your support.

We are using two nef modesl to combine, and if we use onnx to combine model, We do not have onnx of "yolov5_pd/models_720.nef"

Could you help us where do we get the onnx of the model below ?

+ kneron_plus_v2.1.1/kneron_plus/res/models/KL720/yolov5_pd/models_720.nef

Thank you very much.

Ethon Lin

Hi Tri,

The model "yolov5_pd/models_720.nef" is trained by Kneron. We provided the NEF file as an example for users, but we will not release the original ONNX model.

If you want to train the yolov5/v7 model yourself, you can refer to the modifications discussed in the following link: https://www.kneron.com/forum/discussion/comment/1421/#Comment_1421

Tri

NEF Header:

platform: 

target: 720

crc: 0xE2D6AFDB

kn_number: 0x0

encryption_mode: NONE

compiler_version: v0.9.1(befccbd0f)

toolchain_version: kneron/toolchain:v0.23.1

schema_version: v0.0.1

uuid: 0xE2D6AFDB

solution_id: 

solution_tag: 

---------------------------

NEF Model Info:

++++++++++++++++++++++

Model_ID: 2

Model_Name: input.kdp720.scaled.bie

Model_Version: 0x2

++++++++++++++++++++++

Model_ID: 1

Model_Name: input.kdp720.scaled.bie

Model_Version: 0x1

===========================

This is model i combine:

I try to use kp_generic_raw_inference_send successful only "Model_ID: 2",

If i change "_input_desc.model_id = 1", kp_generic_raw_inference_send is fail. this is parameter i use:

  _input_desc.model_id = 1;//_model_desc.models[0].id;  // first model ID

      _input_desc.resize_mode = KP_RESIZE_ENABLE;    // enable resize in pre-process

      _input_desc.padding_mode = KP_PADDING_CORNER;   // enable corner padding in pre-process

      _input_desc.normalize_mode = KP_NORMALIZE_KNERON; // this depends on models

      _input_desc.image_format = KP_IMAGE_FORMAT_RGBA8888; // image format

      _input_desc.width = pFrameBuffer->width;      // image width

      _input_desc.height = pFrameBuffer->height;     // image height

      _input_desc.inference_number = 0;         // inference number, used to verify with output result

      _input_desc.crop_count = 0;            // number of crop area, 0 means no cropping

Do we have any wrong ?

Thank you very much.

Maria Chen

Hi Tri,

It might be hard to debug with only the information above. Could you provide us with your NEF files and the program file you're running, so we could look into them? Thanks!