Error in generating the nef file
Hi,
I am trying to generate nef file and I have been getting this error:
input bit width [8] of node [general/pool3/MaxPool] is invalid, should be [16]
I was able to generate the onnx file and also, the E2E simulation generated a success message.
What could be the issue?
The discussion has been closed due to inactivity. To continue with the topic, please feel free to post a new discussion.
Comments
Hi Anoop,
Could you run "eval_result = km.evaluate()"? If so, what does the result say?
There are some possible reasons, such as the input shape is incorrect, or there is an unsupported operator, but we would need more information to find out the actual issue. If possible, could you provide your model structure or onnx file?
Thanks for your response. Please find the onnx file attached.
Hi Anoop,
It seems like this model hasn't gone through ONNX to ONNX. Please refer to Toolchain Manual - Document Center (kneron.com), sections 3.1.2 Pytorch to ONNX and 3.1.5 ONNX Optimization. After using the Python API to go through these 2 steps, you will be able to generate the NEF file:
The onnx file that was shared was already optimized as per the toolchain documentation. I could not get the oonx file compiled after running through the workflow.
Hi Anoop,
We ran your onnx model and got the same error as you when we didn't go through ktc.onnx_optimizer.onnx2onnx_flow.
We optimized your onnx file and successfully generated .nef files for both KL520 and KL720. The code below worked for us:
After that process, you could run ktc.ModelConfig and ktc.kneron_inference using the new optimized_cnn_1_test.onnx rather than the original onnx file.
If that still didn't work, could you provide your .onnx file (again), Python script, and a few images for your quantization so we could check if your script was written correctly?
Hi Maria,
Thanks for your response. I was able to generate the nef file. However, I would like to understand how to get the inference from device, same inference as the docker version . I followed the documentation and was able to get this result from device :
The result after inference on docker was: [array([[[[ 3.5251236, -3.8188839]]]])]
The two results look different.
Hi Anoop,
Could you provide us the inference program and the image used, along with your NEF file? Thanks for your help.
By the way, if you set to_rgb = True in your configuration file when you trained your model, please make sure you aren't converting your image to RGB again in your inference program. In other words, you could use
img_input = img
instead of
img_input = cv2.cvtColor(src=img, code=cv2.COLOR_BGR2RGB)
when you are inferencing.
I have attached the files here. The image is not converted to RGB again.
Also,
Shoud we consider the ndarray generated from the raw inference as the ouput from final layer of the model or is it something else?
I would like to understand how to interpret the above result. I see the same result for all inputs/images.
Thanks!
Hi Anoop,
Thank you for providing the files. Here are a few things you could check:
-When you generate the .nef file, please check if your normalization settings match the settings used in device inferencing. For example, if your configuration for converting the model is RGB/256 - 0.5, please use KP_NORMALIZE_KNERON when you are inferencing
Normalize mode reference: kp enum - Document Center (kneron.com)
-If you were using the exact same image and inference program, please make sure that your preprocessing (processing your image before inferencing, e.g. image format, resize mode) is done in the same way for both the docker inference and the device inference. There are a lot of times the results turned out differently because they were different
And yes, the raw result (ndarray) is the output from the final layer of the model. You could also test your model by using the E2E simulator: Toolchain Manual - Document Center (kneron.com)
Hi Maria,
Despite using the same preprocessing function and normalize mode, I have not seen any change in the accuracy of results on the device.
There is a lot of difference in the results between docker and the inference from device. I am testing a custom binary image classification model. The results are accurate only for one of the classifications.
Hi Anoop,
Where is your model from, and how did you train your model? If you didn't follow Kneron mmlab for model training and converted a self-trained model to .nef, you would need to inference using Kneron PLUS 2.0.1 and write your own generic inference scripts referring to the demo programs in it.
Also, how did you set your configs when converting the bie into an NEF? Incorrect configurations could also lead to incorrect results in inferencing in PLUS.
Hi Maria,
Our pre-processing function for getting the inference :
def preprocess(img_file_path):
image = cv2.imread(img_file_path)
image = cv2.cvtColor(image, cv2.COLOR_RGB2RGBA)
image = cv2.resize(image , (224, 224), interpolation=cv2.INTER_LINEAR)
return image
parameters of the inference function:
generic_inference_input_descriptor = kp.GenericImageInferenceDescriptor(
model_id=model_nef_descriptor.models[0].id,
inference_number=0,
input_node_image_list=[
kp.GenericInputNodeImage(
image=img,
image_format=kp.ImageFormat.KP_IMAGE_FORMAT_RGBA8888,
resize_mode=kp.ResizeMode.KP_RESIZE_DISABLE,
padding_mode=kp.PaddingMode.KP_PADDING_SYMMETRIC,
normalize_mode=kp.NormalizeMode.KP_NORMALIZE_CUSTOMIZED_DEFAULT
)
]
)
Our input parameters:
Image format of the model: RGBA
Image size: 224*224
Normalization: Image/255
The above configuration works on docker but not with the device. I have also attached the docker workflow configuration.
We want to know the correct input configurations for getting the inference with our parameters. How to apply custom normalization for RGBA8888 input ?
We get the following error:
Error: inference failed, error = Error raised in function: generic_image_inference_receive. Error code: 108. Description: ApiReturnCode.KP_FW_NOT_SUPPORT_PREPROCESSING_108
We have tried all the combinations from our earlier conversations and also from the documentation. We could not find any documentation specific to a customized model other than the YOLO model.
If you need any other information, we can send a detailed email as we can't share every detail on the forum due to our research protocol.
Thanks.
@Anoop Mishra
Hi Anoop,
I would like to know which result is close to your prediction for the inference of on docker E2E inference and the KL520 device.
I guess the docker inference is close to your inference result. If you need to perform inference on a KL520 device the same as docker, you need to have the same inference conditions as on docker (e.g., same data format, radix = 7, scale = 1.0).
You can refer to the following:
https://doc.kneron.com/docs/#toolchain/python_app/app_flow_manual/ (Python API Inference)
https://doc.kneron.com/docs/#plus_python/tutorial/chapter/model_inference_with_data_inference/ (Inference without Built-In Hardware Image Pre-Processing)
As mentioned in the link, before you process your data into dongle KL520 (NPU) inference, you must re-layout the data to fit the NPU data layout format.
I have modified your scripts here so that the docker e2e nef inference result has the same result as the KL520 inference result, you can send me a private message, and I can organize it and send it to you.