[FAQ] 用NEF推論不如預期 / NEF inference results are different from anticipated
當轉換後的nef模型在Kneron PLUS推論的結果不如預期，可以依照下列的步驟來debug:
1. 用E2E Simulator檢查optimized onnx的結果
在Kneron toolchain上，可以使用optimized onnx模型，加上您的前處理與後處理來跑E2E Simulator (ktc.kneron_inference())，檢查optimized onnx推論的結果是否與原始onnx模型的結果相似。如果有剪掉原始onnx模型的nodes，會需要將這些nodes加回去postprocess裡
範例:
E2E Simulator (floating point)文檔: 3. Floating-Point Model Preparation - Document Center
ktc.kneron_inference()文檔: End to End Simulator - Document Center
2. 用E2E Simulator檢查nef的結果
若上面步驟的推論結果可以接受，接下來可以用nef模型和上面步驟的前處理與後處理來跑E2E Simulator，檢查nef的結果是否與上面optimized onnx的結果相似
範例:
E2E Simulator (hardware)文檔: 5. Compilation - Document Center
如果optimized onnx和nef的E2E Simulator結果不相似，請確認這兩個用的前處理、後處理、輸入資料(例:圖片)、input_data等是否一致
3. 比較E2E Simulator的nef與Kneron PLUS的nef的結果
如果optimized onnx和nef的E2E Simulator結果相似，但實際在Kneron PLUS上跑的結果跟E2E Simulator的結果不一樣，請檢查PLUS端的前處理設定(例:正規化)、後處理nodes的排列、輸入資料(例:圖片)等
想要提升模型的精準度時，請確認:
-量化的資料(例:圖片)。我們推薦使用100筆資料，需要有多樣性，且要與當初訓練時用的資料有關聯
-km.analysis的參數，像是將datapath_bitwidth_mode和weight_bitwidth_mode從int8調整成int16
---
If the converted NEF model’s inference result in Kneron PLUS isn’t as expected, please follow the below steps to debug:
1. Use E2E Simulator to check the optimize onnx result
On Kneron Toolchain, check if the optimized onnx model can produce a similar result as the original onnx model by using E2E Simulator (ktc.kneron_inference()). If you cut off nodes in the original onnx model, please remember to add the nodes back in your postprocess function
Example:
E3E Simulator (floating point) documentation: 3. Floating-Point Model Preparation - Document Center
ktc.kneron_inference() documentation: End to End Simulator - Document Center
2. Use E2E Simulator to check the NEF result
If the result in the above step is acceptable, run E2E Simulator for nef and compare the result to the above optimized onnx result to see if they are similar
Example:
E2E Simulator (hardware) documentation: 5. Compilation - Document Center
If the E2E Simulator results for optimized onnx and NEF are completely different, please check if their preprocess functions, postprocess functions, inputs (e.g. images), and input_data are the same
3. Compare the NEF results for E2E Simulator and Kneron PLUS
If the results are similar in the above steps, but the Kneron PLUS result is different from them, please check your preprocess settings (e.g. normalization), output node order in postprocess, inputs (e.g. images), etc.
To increase model accuracy, please try checking these:
-Input data used for quantization (e.g. images). We recommend you use 100 inputs, and they need to be diverse and relevant to the data you used to train the model
-Adjust the parameters in km.analysis, such as setting datapath_bitwidth_mode and weight_bitwidth_mode from int8 to int16