Kneron AI-KL730

tstctstc
in AI Model migration

您好：

我在kl730模型开发的时候遇到了问题，就是我的.onnx转.nef模型的时候，工具链v.30.0版本是modelconfig增加了一个可选参数 input_fmt用于指定模型输入格式，但我填入参数的 input_fmt="4W4C8B"的时候去进行转换，转换代码伪代码km = ktc.ModelConfig(211, "0001", "730", onnx_model=m, input_fmt="4W4C8B")。

总是有如下报错。

[common][error][exceptions.cc:41][KneronException] HardwareNotSupport: Unsupported ReFormat fmt [4W4C8B]

[common][info][utils.cc:271][DumpBacktrace] dump backtrace

Stack trace (most recent call last):

#10  Object "", at 0xffffffffffffffff, in

#9  Object "/workspace/libs_V2/compiler/batch_compile", at 0x5fecd2a925bd, in _start

BFD: DWARF error: section .debug_info is larger than its filesize! (0x93f189 vs 0x530e70)

#8  Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x70cc17576082, in __libc_start_main

#7  Source "/projects_src/kneron-piano_v2/compiler/bin/batch_compile/batch_compile.cc", line 375, in main [0x5fecd2a8da4f]

#6 | Source "/projects_src/kneron-piano_v2/compiler/bin/batch_compile/batch_compile.cc", line 174, in BatchCompile

  | Source "/projects_src/kneron-piano_v2/compiler/bin/batch_compile/batch_compile.cc", line 149, in RunCompile

   Source "/projects_src/kneron-piano_v2/compiler/lib/common/public/compile_impl.h", line 62, in Compile(char const*, char const*, Json::Value const&, FirmInfo*) [0x5fecd2a93197]

#5  Source "/projects_src/kneron-piano_v2/compiler/lib/wagner/backend/compile.cc", line 313, in CompileImpl [0x70cc167b079c]

#4  Source "/projects_src/kneron-piano_v2/compiler/lib/common/backend/compile_utils.cc", line 606, in GraphCompiler::OptimizeGraph() [0x70cc1685ca0b]

#3  Source "/projects_src/kneron-piano_v2/compiler/lib/wagner/frontend/tgt_graph_gen.cc", line 79, in GraphOptBE(Graph&, ShapeInferencer&) [0x70cc1681917e]

#2 | Source "/projects_src/kneron-piano_v2/compiler/lib/wagner/frontend/tgt_post_process.cc", line 163, in WagnerPostProcess(Graph&)

  | Source "/projects_src/kneron-piano_v2/compiler/lib/wagner/frontend/tgt_post_process.cc", line 138, in PostCheck

  | Source "/projects_src/kneron-piano_v2/compiler/lib/wagner/frontend/tgt_post_process.cc", line 87, in CheckReFormatFmt

   Source "/projects_src/kneron-piano_v2/piano/lib/piano/common/exceptions.h", line 50, in HardwareNotSupport::HardwareNotSupport(std::string const&) [0x70cc166bb825]

#1  Source "/projects_src/kneron-piano_v2/piano/lib/piano/common/exceptions.cc", line 44, in KneronException::KneronException(ErrorCode, std::string const&) [0x70cc17c9fd76]

#0  Source "/projects_src/kneron-piano_v2/piano/lib/piano/common/utils.cc", line 273, in DumpBacktrace() [0x70cc17cb79e8]

BFD: DWARF error: section .debug_info is larger than its filesize! (0x93f189 vs 0x530e70)

[tool][error][batch_compile.cc:175][BatchCompile] Failed to compile input.onnx

Traceback (most recent call last):

 File "/docker_mount/test_data/25_8_8.py", line 60, in <module>

  main()

 File "/docker_mount/test_data/25_8_8.py", line 56, in main

  nef_model_path = ktc.compile([km])

 File "/workspace/miniconda/envs/onnx1.13/lib/python3.9/site-packages/ktc/toolchain.py", line 519, in compile

  return encrypt_compile(

 File "/workspace/miniconda/envs/onnx1.13/lib/python3.9/site-packages/ktc/toolchain.py", line 661, in encrypt_compile

  subprocess.run(commands, check=True)

 File "/workspace/miniconda/envs/onnx1.13/lib/python3.9/subprocess.py", line 528, in run

  raise CalledProcessError(retcode, process.args,

subprocess.CalledProcessError: Command '['/workspace/libs_V2/compiler/batch_compile', '-T', '730', '/data1/kneron_flow/batch_compile_bconfig.json', '-t', 'kneron/toolchain:v0.30.0\n', '-o']' returned non-zero exit status 4.

Tagged:

Comments

  • Ethon LinEthon Lin
    edited August 18

    Hello,

    KL730 如果要指定 input 是 4W4C8B，格式上需要滿足 conv 是 first layer， 且 input channel <=4 && column stride <= 2 && column dilation == 1

    onnx conv stride/dilation 定義可以參考 https://github.com/onnx/onnx/blob/rel-1.15.0/docs/Operators.md#conv


    若轉換4W4C8B會失敗且模型無法修改的話，可以再試試看下面兩種format

    https://doc.kneron.com/docs/#toolchain/manual_3_onnx/#32-ip-evaluation

    • 11: ("HW4C8B_DROP_A", 8),
    • 8: ("1W16C8B", 8),

    這兩種在FW上有特別處理，理論上也可以配合KL730的硬體使用

  • tstctstc
    • 11: ("HW4C8B_DROP_A", 8)也是转换失败，8: ("1W16C8B", 8)能转出models_730.nef模型成功，但是我放进plus v3.1.2进行模型推理，kp_customized_inference_send推理部分都是返回失败，inference failed, error = -1 (USB: Input/output error)。kp_generic_image_inference_send和kp_generic_data_inference_send虽然推理成功，但是并不能和models_730.nef转换前身.pth文件有一样的效果。我还需要哪些处理。
  • tstctstc

    如果我转换4W4C8B输入类型成功，是否都能适配plus接口和FW上的处理，无需做过多的修改，就能进行模型替换推理？

  • Ethon LinEthon Lin

    要比較models_730.nef與pth，請先在toolchain上透過ktc.kneron_inference()分別對ONNX與NEF模型推論，並確認兩者使用完全相同的流程(preprocess, postprocess. etc..)，僅有模型的差異，來比較兩者答案是否都有符合您的預期

    在因為PLUS上容易因為前後處理與原本有差異而導致結果不同，在toolchain上透過ktc.kneron_inference()能夠針對確認模型效果，使用方法請參考下面toolchain文件

    https://doc.kneron.com/docs/#toolchain/manual_1_overview/#12-workflow-overview


    另外，要使用kp_customized_inference_send來進行推論的話，必須事先透過SDK在FW中加入您的模型資訊與流程(model_id, job_id, preprocess, postprocess function, etc..)，4W4C8B只是確保您的模型與KL730前處理IP的格式相符，對應的周邊流程還是要由您這邊自行開發

    相關使用方法可以參考文件

    https://doc.kneron.com/docs/#plus_c/feature_guide/customized_api/create_kl730_single_model_example/

  • tstctstc
    https://www.kneron.com/forum/discussion/comment/2332#Comment_2332

    感谢您的回答，我大致明白了

Sign In or Register to comment.

© Kneron, Inc.