KL720 time efficiency

in kl720_cam_isi_center_app/kl720_cam_isi_center_app.cpp, which has a kdp_isi_inference api. We use the video to test the efficiency of the api.

86.93ms at 1920x1080,

39.47ms at 1280x720.

So there are the following two questions,

  1. In addition to model inference, what other functions does this api have?
  2. How to know the time efficiency of the model inference process?


Comments

    1. This kdp_isi_inference API will send a command and image to KL720 and wait for an ack. After sending the ack, KL720 will do pre-processing for the image, model inference, and post processing.
    2. If you can get the time (time_1) of capturing one video image and get the total time (time_2) of capturing video image and doing kdp_isi_inference() and kdp_isi_retrieve_res(), subtract time_1 from time_2 will tell the time of (pre-processing + model inference + post processing).

    If the time (like 39.47ms at 1280x720) is what printed out along with FPS info, that time is time_2.

  • We think the time (86.93ms at 1920x1080) is longer than we expected, so we want to know the individual time of pre-processing, model inference and post processing.

  • do you have an uart connect to your development board? What's the command you pass to the KL720? do you run it in parallel mode or non-parallel mode?

  • We did not change any line of code. Execution is standard process to the KL720. So kl720_cam_isi_center_app run parallel mode or non-parallel mode ?

    we want to know the time of  model inference ?

  • Could you let us know which model you are running? And what's the command you run in host? Could you post it here?

    The timing breakdown is in uart log. If you don't connect the uart, then it is hard to see the time of inference. We could run 100 inferences in parallel mode, then it should pipeline and hide most of the data transfer time.

    1. What command should we pass to KL720? Will the command decide it run in parallel mode or non-parallel mode?
    2. Do you know which function can run model inference?

  • you can connect the microUSB port on the left-up side to get UART log.

    1. Parallel mode is in kl720_isi_center_app/kl720_isi_center_app.cpp example. Using 1 in its first argument for parallel mode vs 0 for serial mode. E.g. "kl720_isi_center_app.exe 1 100 211".
    2. Model inference is initiated when kdp_isi_inference is called.

    After UART is connected to a terminal like Putty, logs can be seen in the terminal. Timing information looks like this:

    Time[100] Loop 105 (img read 8): Trip 70: npu 40 pre 1 post 28
    
    where :
    Time[#]: Timing log message with image sequence number
    Loop #: Loop time for KL720 to finish an image inference
    (img read #): Time to read in image from host to KL720
    Trip #: Round trip time from triggering DSP/NPU and back
    npu #: NPU time to run model
    pre #: Pre-processing time by DSP
    post #: Post-processing time by DSP
    
The discussion has been closed due to inactivity. To continue with the topic, please feel free to post a new discussion.