Same deep learning but different results between RGB565 and RGBA8888
Hi,
We have a Mobilenet backbone based deep learning model. At first, we tested it by using Python (onnx runtime) and the result is correct ("0").
We convert it to Kneron quantized model by using following commands:
mkdir /data1;
cp ./input_params.json /data1/;
python /workspace/scripts/fpAnalyserCompilerIpevaluator_520.py -t 8;
cp ./batch_input_params.json /data1/;
python /workspace/scripts/batchCompile_520.py;
Related json files are attached.
Our expected pre-process is "tensorflow": (x/127.5)-1.0
But we can't use the default "tensorflow" pre-process because our want to use RBGA8888 as our input.
So we run these two tests but got totally different results:
1. Input RGB565
#define IMG_SOURCE_W 100
#define IMG_SOURCE_H 100
#define IMG_SIZE (IMG_SOURCE_W * IMG_SOURCE_H * 2)
dme_cfg.model_id = DME_MODEL_ID;
dme_cfg.output_num = 1;
dme_cfg.image_col = IMG_SOURCE_W;
dme_cfg.image_row = IMG_SOURCE_H;
dme_cfg.image_ch = 3;
dme_cfg.image_format = IMAGE_FORMAT_SUB128 | NPU_FORMAT_RGB565 | IMAGE_FORMAT_RAW_OUTPUT;
The output is:
9.65505123, -2.9817071, -7.66724682
2. Input RGBA8888
#define IMG_SOURCE_W 100
#define IMG_SOURCE_H 100
#define IMG_SIZE (IMG_SOURCE_W * IMG_SOURCE_H * 4
dme_cfg.model_id = DME_MODEL_ID;
dme_cfg.output_num = 1;
dme_cfg.image_col = IMG_SOURCE_W;
dme_cfg.image_row = IMG_SOURCE_H;
dme_cfg.image_ch = 4;
dme_cfg.image_format = IMAGE_FORMAT_BYPASS_PRE | IMAGE_FORMAT_RAW_OUTPUT;
...
cv::cvtColor(img, img, cv::COLOR_BGR2RGBA);
img.convertTo(img, CV_8SC4, 1.0, -128.0);
...
The output is:
-1.56184661, -0.993902326, 2.55574894
Anything wrong? Thanks
Comments
Thanks for the question.
I see that your input image size is 100x100. What is your model input size? Because if you select IMAGE_FORMAT_BYPASS_PRE , then it won't do any resize or padding. You have to provide the exact RGBA with the input size of the model.
Also, you mentioned that you suppose to get a result of 0, but there are 3 outputs to your models, how do you do the conversion?
Hi,
Our model input size is 100x100x3 and output size is 1x3.
1st test, the output is:
9.65505123, -2.9817071, -7.66724682
Since first number is largest among the three, the correct result is "0".
2nd test, the output is
-1.56184661, -0.993902326, 2.55574894
Since third number is largest among the three, the in-correct result is "2", out expected result should be "0".
Hello,
As you mentioned the result of RGB565 was right but RGBA8888 (you bypass preprocess and implement by yourself) was wrong. I think the issue could be related to image format.
Here is the format of RGB565, RGBA8888 and bypass preprocess
As the above picture showed, the sequence of bypass preprocess is opposite to preprocess setting RGB565 and RGBA8888.
You can try preprocess with setting "NPU_FORMAT_RGBA8888" to check it.