kl720 compile and quantization issuse

hyun

What kind of quantization do you use? Is nef in uint8 format?

I know km.analysis() is a function for quantization, but I don't know how it works internally.

I'm afraid the model-compilation will crash due to My custom quantization.

Can kl720 receive the uint8 type model as input and convert it to nef?

I look forward to your reply,

BR

Ethon Lin

Hello,

Kneron's quantization is used for floating to INT8 format, you can get more detail from following link.

https://doc.kneron.com/docs/#toolchain/manual_4_bie/#41-quantization

It's possible to get extra loss if you are using quantizing model to run on KL720. If you want to check the effect before implement, you can evaluate your model with our toolchain simulator.

hyun

in https://doc.kneron.com/docs/#toolchain/manual_4_bie/#41-quantization, there is too less information about quantization.

I have to do QAT, and I wonder if your toolchain supports QAT. If not supported, I wonder if the model quantized in the form of uint8 can be used without going through the toolchain or Can kl720 receive the uint8 type model as input and convert it to nef?

Bike Xie

Hi Hyun,

The toolchain will process the quantization based on the given model and the typical data set. Even if the model weights are UINT8, toolchain will still convert all weights into INT8 and the quantized model will be different from the original UINT8 model.

Our toolchain does not support QAT yet. QAT feature will be released late this year.