Simulator inference fails for a Transformer

Comments

• Andy Hsieh

  June 26

  @Chih-Hsuan Yen

  Hi Chih Hsuan,

  Thank you for reporting this issue.

  To help us reproduce and further investigate the memory consumption issue during BIE simulator inference, could you please share the exact script or command you used, including the arguments and configuration files, if any? Since you mentioned you are using python_api_workflow.py from the examples, it would be very helpful if you could also share any modifications you made to the script.

  Regarding the NEF inference returning all zeros, this may indicate that something went wrong during the ONNX-to-BIE conversion or quantization process. To better understand the issue, could you please provide the following information?

  1. The original model used for conversion
  2. The calibration / quantization images used during ONNX-to-BIE conversion
  3. The number of images used during inference
  4. The input image(s) used for inference
  5. The generated ONNX and BIE files, if possible
  6. The NEF file, if available
  7. The full conversion and inference logs

  With these materials, we should be able to reproduce the behavior on our side and check whether the all-zero output is caused by quantization, model conversion, input preprocessing, or simulator/runtime configuration.

  Thank you.

  
  

  Best regards,

  Andy Hsieh
• Chih-Hsuan Yen

  June 29 edited June 29

  Thanks for the help. For the used script, see the python file in the attached zip. Here are other information:

  
  

  1. git clone https://huggingface.co/ainekko/smolvla_base_onnx, where smolvlm_vision.onnx in it is tested

  2. two images in the npy file in attached zip

  3/4. 1 image - the first out of two in npy

  5. ONNX was not generated by me. It's taken directly from the aforementioned project. For BIE, it is too large to upload (441MB)

  6. NEF file is too large to upload (167MB)

  7. See the log file in attached zip

  workspace.zip