Implement GRADCAM

Anoop Mishra

Hi,

I want to add a GRADCAM module [ this code https://keras.io/examples/vision/grad_cam/] to KL520. My understanding is that I will have to retrieve the weights of the intermediate weights of the .nef model (file) to calculate the gradients of the feature maps. Is that feasible? If so, please point me in the right direction.

Ethon Lin

Hello,

To port your own model to KL520, you should check whether the operators in your model are supported or not first. Please refer to the chapter 2.3 Supported operators in following link.

http://doc.kneron.com/docs/#toolchain/manual/#2-toolchain-docker-overview

Then follow the steps in document to convert, quantize, and compiler your model to NEF format.

After you get your NEF model, you can run it with our PLUS SDK on KL520.

Anoop Mishra

Hi Lin,

I have checked if the model is supported and it does. However, the unknown here is if I can retrieve intermediate model feature maps (from CNNs) which can be used for post-processing from the NEF model. A workaround I could think of is having multiple models if it is not possible to retrieve intermediate outputs. As a follow-up question, in addition to inferencing, can we do post-processing such as python matrix operations or OpenCV on KL520?

Ethon Lin

Hi Mishra,

KL520 only can output the result of last layer in model.

And due to the memory limitation, python or opencv is too large to process on KL520 (firmware). But the SDK "PLUS" we provided can do that easily. PLUS is a software SDK for user to access KL520. You can process the image on your PC and send it to KL520, then get the inference result through usb. You can do any processing after you got the result on PC.

http://doc.kneron.com/docs/#plus_c/getting_started/#_top