preprocess and quantization error
Hi,
I am trying to deploy my facial landmark detection model into the KL520 device.
My model takes input value between [0,1], so I set img_preprocess_method as "yolo" in my input_params.json as following.
{
"model_info": {
"input_onnx_file": "/data1/tf_test/lm_pad_mask_64_0921.tflite.opt.onnx",
"model_inputs": [{
"model_input_name": "input" ,
"input_image_folder": "/data1/tf_test/img_rgb"
}]
},
"preprocess": {
"img_preprocess_method": "yolo",
"img_channel": "RGB",
"radix": 7,
"keep_aspect_ratio": true,
"pad_mode": 1,
"p_crop": {
"crop_x": 0,
"crop_y": 0,
"crop_w": 0,
"crop_h": 0
}
},
"simulator_img_files": [{
"model_input_name": "input" ,
"input_image": "/data1/tf_test/img_rgb/1_112_112.jpg"
}]
}
However, when I run the generated all_model.bin and fw_info.bin using dme_keras.py, I found the inference result is quite different from my original model.
Kneron Device: [0.35792017 0.27688164 0.3174009 0.8576578 0.46597153 0.45921832 0.66181463 0.27012843 0.6348018 0.77661926] TFlite: [0.26855683 0.23025376 0.2584893 0.6576536 0.4441332 0.36857533 0.62811494 0.28813866 0.62961656 0.6005157 ]
I use the same dme cfg setting as kdp_dme_load_yolo_model.
# dme configuration model_id = 1000 # model id when compiling in toolchain output_num = 1 # number of output node for the model image_col = 64 image_row = 64 image_ch = 3 image_format = (constants.IMAGE_FORMAT_SUB128 | constants.NPU_FORMAT_RGB565 | constants.IMAGE_FORMAT_RAW_OUTPUT)
I am wondering whether my image_format setting is wrong, so my model takes the wrong value range as input or the difference of inference result is just quantization error.
The discussion has been closed due to inactivity. To continue with the topic, please feel free to post a new discussion.
Comments
I think because you select yolo [0-1] as your input pre process, so you have to modify the dme configuration. You should modify the image_format to the following:
image_format = (constants.IMAGE_FORMAT_RIGHT_SHIFT_ONE_BIT |
constants.NPU_FORMAT_RGB565 |
constants.IMAGE_FORMAT_RAW_OUTPUT)
it means you right shift 0-255 by 1 bit, so it would means 0-1 in 8 bit fixed point. Remember the MSB is sign bit. Please give it a try and let us know if it leads to better result. Thanks,
Hi,
Thank you for your reply and help.
The inference result became more reasonable after I use "constants.IMAGE_FORMAT_RIGHT_SHIFT_ONE_BIT".
But, I still met some problems with the preprocessing, when I am trying to convert other models using the toolchain 520.
I am not sure what those constants and preprocess functions actually do on the device.
Is there any documentation writing about it?
If I use IMAGE_FORMAT_RIGHT_SHIFT_ONE_BIT, the value of input will be divide by 2, so the value range is [0,128].
Supposedly, it will then divided by 2^7 to make it [0,1] on the device. But, where to set this 2^7 value?
Also, I am trying to convert a model that takes input range as [0,255] directly.
I made a customized preprocess method in img_preprocess.py to return x without doing any normalization method.
but it failed when executing fpAnalyserCompilerIpevaluator_520.py.
Can the quantization tool and demo API handle the model with input range [0,255]?
For your question:
Supposedly, it will then divided by 2^7 to make it [0,1] on the device. But, where to set this 2^7 value?
When you set the input_params.json, you already tell the hardware radix at 7, this means the decimal point is set to before the 7th bit of the 8 bits value.
for example, the white pixel in 0-255 is 255, and after you do the right shift to 1 is 127, which is 0111_1111 in 8 bits. And since you have let the hardware know the input radix is 7, this means the decimal point at 0.111_1111=1-1/127= 0.992 -> the input range is [0-1]
If you want to set the input range to [0-255], you still need to do right shift because the 8th bit is a sign bit, so it cannot be 1, because that would mean the number is a negative number. for 0-255, you need to set the radix to be -1 in input_params.json in order to make it work. radix -1 means where the decimal point at. -1 means the bit 0 actually stands for 2.