• atc转换错误
    按照流程装了运行环境,报错如图
  • [其他问题] npu-smi无法运行
    Ascend310-driver-6.0.0-ubuntu18.04.aarch64-minirc.tar.gzAscend-cann-nnrt_6.0.1_linux-aarch64.runAscend-cann-toolkit_6.0.1_linux-aarch64.runroot@davinci-mini:~/Ascend/ascend-toolkit# npu-smi info dcmi module initialize failed. ret is -8005 root@davinci-mini:~/Ascend/ascend-toolkit# dmesg root@davinci-mini:~/Ascend/ascend-toolkit# 
  • [问题求助] MindX SDK2.0.4怎么下载,社区版有没有,下载不了
    MindX SDK2.0.4怎么下载,社区版有没有,下载不了
  • [硬件整机] 从SSH登录不上MDC 210
    已经将 PC 和 MDC 设备在同一网段(192.168.30.42),MTB 310和MDC已经连接,12V供电。电脑和MTB通过网口相连。192.168.30.42可以ping通,但是SSH登录不上MDC 210,截图如下。请问是哪里出的问题
  • [其他问题] 【CANN】安装toolkit安装包时,出现No space left on device的error
    安装toolkit安装包时,出现No space left on device的error,该怎么解决?
  • [应用开发] 如何处理OM模型的输入
    pb模型的输入[1,1,9] 输出[1,4],1、如何设置ATC命令的参数,转成OM模型?2、AscendCL如何构造输入(c++代码),调用OM模型?
  • [应用开发] MDC300F设备onnx转om报错
    1.用的设备是mdc300f平台2.cann版本是5.0mdc3003.mindstudio版本是2.0.0(bete3)4.onnx模型链接:cid:link_05.在cann6.0可以成功转出om模型,但是最终需要部署到mdc300平台上,必须要用自带的cann转换。自带的5.0mdc300转换报错flatten算子axis必须改为1,于是将flatten算子的axis=3强制转为1,出现了如下图的错误,经过初步分析是flatten算子后gather算子没有起作用。想请教一下如何在改了flatten算子axis值的情况下,还能保持给gather算子正确的输入。
  • [AI类] 【求助】MindSpore 自定义算子运行出错
    参考Mindpore 官方自定义算子文档,修改算子定义,构造正向及反向函数,执行报错,有哪位大神帮忙看下咩靴靴!Mindpore版本1.9.0
  • [其他问题] 鲲鹏社区和昇腾社区为什么打不开网页了?
    鲲鹏和昇腾社区打不开网页了,浏览器一直再转。(如下图)
  • [常见问题汇总帖] 【MindX 3.0.T700 mxindex 特征检索】AscendIndexIVFSQT 运行update出错
    同一个基于AscendIndexIVFSQT检索方法的demo,在100000底库小数据集上可成功运行:Load data set successfull. index start index init train start Training IVF quantizer on 100000 vectors in 256D to 1024 cluster training computeCompressIndex on correlation matrix Clustering 100000 points in 256D to 1024 clusters, redo 1 times, 16 iterations Preprocessing in 0.03 s training computeCompressValue on 100000 vectors in 256D Iteration 15 (2.73 s, search 2.59 s): objective=87058.3 imbalance=1.155 nsplit=0 training scalar quantizer on 100000 vectors in 64D add start AscendIndex::add: adding 0:100000 / 100000 update start search start Search finished successfully Write result successfully但是在9000000底库规模的数据集上,执行到update时报错:Load data set successfull. index start index init train start Training IVF quantizer on 9147767 vectors in 256D to 1024 cluster training computeCompressIndex on correlation matrix Sampling a subset of 524288 / 9147767 for training Clustering 524288 points in 256D to 1024 clusters, redo 1 times, 16 iterations Preprocessing in 3.41 s training computeCompressValue on 9147767 vectors in 256D Iteration 15 (114.84 s, search 114.24 s): objective=472094 imbalance=1.097 nsplit=0 training scalar quantizer on 9147767 vectors in 64D add start AscendIndex::add: adding 0:882984 / 9147767 AscendIndex::add: adding 882984:1765968 / 9147767 AscendIndex::add: adding 1765968:2648952 / 9147767 AscendIndex::add: adding 2648952:3531936 / 9147767 AscendIndex::add: adding 3531936:4414920 / 9147767 AscendIndex::add: adding 4414920:5297904 / 9147767 AscendIndex::add: adding 5297904:6180888 / 9147767 AscendIndex::add: adding 6180888:7063872 / 9147767 AscendIndex::add: adding 7063872:7946856 / 9147767 AscendIndex::add: adding 7946856:8829840 / 9147767 AscendIndex::add: adding 8829840:9147767 / 9147767 update start terminate called after throwing an instance of 'faiss::FaissException' what(): Error in void faiss::ascend::AscendClusteringImpl::subClusExecInt8(int, uint16_t*, float*, int, size_t) at ascend/custom/impl/AscendClusteringImpl.cpp:117: Error: 'ret == RPC_ERROR_NONE' failed: RpcIndexClusteringExecInt8 failed(1).以下是一些参数配置: const size_t dim = 256; const size_t dimOut = 64; const size_t nlist = 1024; const size_t k = 10; const size_t nprobe = 24; const size_t lowerBound = 1000; const size_t mergeThres = 1000; const float memoryLimit = 100000000; const size_t addTotal = 100000000;求大佬帮忙解答下~~
  • [其他问题] ATLAS 200DK 安装pytorch报错
    参照https://gitee.com/ascend/modelzoo-GPL/tree/master/built-in/ACL_Pytorch/Yolov5_for_Pytorch进行环境配置,执行pip3 install -r requirements.txt显示源也换了好几次,但是还是报相同错误,执行sudo python3 -m pip install torch===1.8.1 torchvision===0.9.1 -f https://download.pytorch.org/whl/torch_stable.html -i https://pypi.douban.com/simple依旧报相同错误。还请专家帮忙指点!
  • [MindX SDK] 200dk上使用mind开发,使用几天后,200dk登录不上了
    根据文档制作dd镜像https://bbs.huaweicloud.com/forum/thread-139685-1-1.html,使用几天后,再给200dk上电,就不能启动了。无法ssh连接到200dk。重新制作卡,设备又可以启动。无法查明问题。
  • [问题求助] 华为云服务器镜像推理报错
    华为云主机驱动从20.1.rc1升级到6.0.rc1,固件未升级(虚拟机无法安装固件),nnae、toolkit未安装,在22.0.RC3版本ascend-tensorflow镜像中进行推理,推理脚本来自于华为官方文档样例:https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/60RC1alpha003/moddevg/onlineinfer1/atlasoiug_26_0004.html,推理执行用户HwHiAiUser报错信息如下:WARNING:tensorflow:From infer_from_pb.py:195: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.########NOW Start Preprocess!!!#########WARNING:tensorflow:From infer_from_pb.py:75: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.2023-01-16 08:59:08.096267: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency2023-01-16 08:59:08.097916: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xaaaad1c3f4f0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:2023-01-16 08:59:08.097962: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default VersionWARNING:tensorflow:From infer_from_pb.py:79: FastGFile.__init__ (from tensorflow.python.platform.gfile) is deprecated and will be removed in a future version.Instructions for updating:Use tf.gfile.GFile.WARNING:tensorflow:From infer_from_pb.py:82: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.Instructions for updating:box_ind is deprecated, use box_indices instead########NOW Start Batch!!!#########WARNING:tensorflow:From infer_from_pb.py:106: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.WARNING:tensorflow:From infer_from_pb.py:133: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.WARNING:tensorflow:From infer_from_pb.py:134: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.########NOW Start inference!!!#########2023-01-16 08:59:09.008236: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.008535: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.008665: W tf_adapter/util/npu_attrs.cc:1077] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.008685: W tf_adapter/util/npu_attrs.cc:1207] The op_select_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-16 08:59:09.008694: W tf_adapter/util/npu_attrs.cc:1211] The optypelist_for_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-16 08:59:09.060358: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.060464: W tf_adapter/util/npu_attrs.cc:1077] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.060487: W tf_adapter/util/npu_attrs.cc:1207] The op_select_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-16 08:59:09.060503: W tf_adapter/util/npu_attrs.cc:1211] The optypelist_for_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-16 08:59:09.064144: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.064227: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.064315: W tf_adapter/util/npu_attrs.cc:1077] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.064335: W tf_adapter/util/npu_attrs.cc:1207] The op_select_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-16 08:59:09.064348: W tf_adapter/util/npu_attrs.cc:1211] The optypelist_for_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-16 08:59:09.064495: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.064573: W tf_adapter/util/npu_attrs.cc:1077] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.064590: W tf_adapter/util/npu_attrs.cc:1207] The op_select_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-16 08:59:09.064602: W tf_adapter/util/npu_attrs.cc:1211] The optypelist_for_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-16 08:59:09.064669: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-16 08:59:09.065074: W tf_adapter/util/npu_attrs.cc:148] [GePlugin] DEVICE_ID and ASCEND_DEVICE_ID is none, use default device id : 0, if set session_device_id, session_device_id has a higher priority2023-01-16 08:59:14.059804: I tf_adapter/kernels/geop_npu.cc:825] The model has been compiled on the Ascend AI processor, current graph id is: 1Traceback (most recent call last): File "/usr/local/python37/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call return fn(*args) File "/usr/local/python37/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn target_list, run_metadata) File "/usr/local/python37/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun run_metadata)tensorflow.python.framework.errors_impl.InternalError: GeOp3_0GEOP::::DoRunAsync FailedError Message is : E49999: Inner Error!E49999 Compile op[trans_Cast_169,] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/cast.py], optype[Cast], taskID[5]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/dense/MatMul,[fp32_vars/dense/bias,fp32_vars/dense/bias,fp32_vars/dense/BiasAdd,fp32_vars/dense/MatMul]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/mat_mul.py], optype[MatMulV2], taskID[7]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[trans_TransData_163,] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/trans_data.py], optype[TransData], taskID[132]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_14/Conv2D,[fp32_vars/BatchNorm_14/FusedBatchNormV3,fp32_vars/BatchNorm_14/FusedBatchNormV3,fp32_vars/conv2d_14/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[136]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[138]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Mean,[fp32_vars/Mean/reduction_indices,fp32_vars/Mean/reduction_indices,fp32_vars/Mean]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/reduce_mean_d.py], optype[ReduceMeanD], taskID[133]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_1/Conv2D,[fp32_vars/BatchNorm_1/FusedBatchNormV3,fp32_vars/BatchNorm_1/FusedBatchNormV3,fp32_vars/conv2d_1/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[137]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_46/Conv2D,[fp32_vars/BatchNorm_46/FusedBatchNormV3,fp32_vars/BatchNorm_46/FusedBatchNormV3,fp32_vars/conv2d_46/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[134]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_27/Conv2D,[fp32_vars/BatchNorm_27/FusedBatchNormV3,fp32_vars/BatchNorm_27/FusedBatchNormV3,fp32_vars/conv2d_27/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[135]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[trans_TransData_168,] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/trans_data.py], optype[TransData], taskID[6]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[trans_Cast_0,] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/cast.py], optype[Cast], taskID[142]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[trans_TransData_164,] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/trans_data.py], optype[TransData], taskID[8]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464415437296] recompile single op[fp32_vars/dense/MatMul] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464415437296] recompile single op[trans_Cast_169] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464415437296] recompile single op[trans_TransData_168] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464415437296] recompile single op[trans_TransData_164] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ParalCompOp] Thread[281464415437296] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] [SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:994] [GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1467] Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank3_new_sub_graph110[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] Compile op[fp32_vars/conv2d/Conv2D,[fp32_vars/BatchNorm/FusedBatchNormV3,fp32_vars/BatchNorm/FusedBatchNormV3,fp32_vars/conv2d/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[140]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[trans_TransData_1,] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/trans_data.py], optype[TransData], taskID[141]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/MaxPoolWithArgmax] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/max_pool_with_argmax.py], optype[MaxPoolWithArgmax], taskID[139]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_46] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[202]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/add_15] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/add.py], optype[Add], taskID[196]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/add_12] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/add.py], optype[Add], taskID[212]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_48] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[197]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_39] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[213]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_50/Conv2D,[fp32_vars/BatchNorm_50/FusedBatchNormV3,fp32_vars/BatchNorm_50/FusedBatchNormV3,fp32_vars/conv2d_50/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[201]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_45/Conv2D,[fp32_vars/BatchNorm_45/FusedBatchNormV3,fp32_vars/BatchNorm_45/FusedBatchNormV3,fp32_vars/conv2d_45/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[205]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_22] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[227]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_40] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[229]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_38] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[231]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_25/Conv2D,[fp32_vars/BatchNorm_25/FusedBatchNormV3,fp32_vars/BatchNorm_25/FusedBatchNormV3,fp32_vars/conv2d_25/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[226]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_44/Conv2D,[fp32_vars/BatchNorm_44/FusedBatchNormV3,fp32_vars/BatchNorm_44/FusedBatchNormV3,fp32_vars/conv2d_44/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[228]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/add] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/add.py], optype[Add], taskID[248]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_3] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[249]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/add_13] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/add.py], optype[Add], taskID[252]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_40/Conv2D,[fp32_vars/BatchNorm_40/FusedBatchNormV3,fp32_vars/BatchNorm_40/FusedBatchNormV3,fp32_vars/conv2d_40/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[243]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_51/Conv2D,[fp32_vars/BatchNorm_51/FusedBatchNormV3,fp32_vars/BatchNorm_51/FusedBatchNormV3,fp32_vars/conv2d_51/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[207]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_41/Conv2D,[fp32_vars/BatchNorm_41/FusedBatchNormV3,fp32_vars/BatchNorm_41/FusedBatchNormV3,fp32_vars/conv2d_41/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[230]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_43/Conv2D,[fp32_vars/BatchNorm_43/FusedBatchNormV3,fp32_vars/BatchNorm_43/FusedBatchNormV3,fp32_vars/conv2d_43/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[251]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_26/Conv2D,[fp32_vars/BatchNorm_26/FusedBatchNormV3,fp32_vars/BatchNorm_26/FusedBatchNormV3,fp32_vars/conv2d_26/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[241]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/add_3] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/add.py], optype[Add], taskID[257]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/add_7] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/add.py], optype[Add], taskID[260]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_12] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[258]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_2/Conv2D,[fp32_vars/BatchNorm_2/FusedBatchNormV3,fp32_vars/BatchNorm_2/FusedBatchNormV3,fp32_vars/conv2d_2/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[254]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_10] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[279]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/add_1] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/add.py], optype[Add], taskID[282]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_24/Conv2D,[fp32_vars/BatchNorm_24/FusedBatchNormV3,fp32_vars/BatchNorm_24/FusedBatchNormV3,fp32_vars/conv2d_24/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[259]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_12/Conv2D,[fp32_vars/BatchNorm_12/FusedBatchNormV3,fp32_vars/BatchNorm_12/FusedBatchNormV3,fp32_vars/conv2d_12/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[278]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_11/Conv2D,[fp32_vars/BatchNorm_11/FusedBatchNormV3,fp32_vars/BatchNorm_11/FusedBatchNormV3,fp32_vars/conv2d_11/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[256]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/Relu_13] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/relu.py], optype[Relu], taskID[294]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_8/Conv2D,[fp32_vars/BatchNorm_8/FusedBatchNormV3,fp32_vars/BatchNorm_8/FusedBatchNormV3,fp32_vars/conv2d_8/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[263]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/add_4] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/add.py], optype[Add], taskID[308]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_15/Conv2D,[fp32_vars/BatchNorm_15/FusedBatchNormV3,fp32_vars/BatchNorm_15/FusedBatchNormV3,fp32_vars/conv2d_15/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[293]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_3/Conv2D,[fp32_vars/BatchNorm_3/FusedBatchNormV3,fp32_vars/BatchNorm_3/FusedBatchNormV3,fp32_vars/conv2d_3/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[287]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_13/Conv2D,[fp32_vars/BatchNorm_13/FusedBatchNormV3,fp32_vars/BatchNorm_13/FusedBatchNormV3,fp32_vars/conv2d_13/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[299]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] Compile op[fp32_vars/conv2d_16/Conv2D,[fp32_vars/BatchNorm_16/FusedBatchNormV3,fp32_vars/BatchNorm_16/FusedBatchNormV3,fp32_vars/conv2d_16/Conv2D]] failed, oppath[/usr/local/Ascend/nnae/6.0.RC1/opp/op_impl/built-in/ai_core/tbe/impl/conv2d.py], optype[Conv2D], taskID[301]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:856] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_22/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_19/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_16/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_13/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_9/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_6/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_3/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_21/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_18/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_15/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_6] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_5] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_4] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_5/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_8/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_20] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_17] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_14] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_11] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_19] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_16] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_13] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_11/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_12/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_24/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_10] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_2/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_21] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_18] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_15] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_12] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_7] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_3] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_26/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_43/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_29/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_32/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_35/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_38/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_41/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_48/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_51/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_28/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_52/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_31] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_49/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_28] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_42/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_5] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_39/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_36/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_33/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_30/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/MaxPoolWithArgmax] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_4/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[trans_Cast_0] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_7] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_27/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_14/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_4] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[trans_TransData_163] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Mean] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_1/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_46/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_15] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[trans_TransData_1] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_50/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_7/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_46] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_10/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_45/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_8] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_39] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_17/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_44/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_20/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_38] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_23/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_43] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_6] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_41] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_47] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_44] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_14] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_12] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_11] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_10] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_9] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_8] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_48] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_31/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_45] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_42] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_36] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_33] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_30] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_27] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_24] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_47/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_22] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_40] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_35] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_32] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_29] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_26] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_23] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_37] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_34] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_25] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_25/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_3] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/Relu_9] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/add_13] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_40/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_37/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281464432222704] recompile single op[fp32_vars/conv2d_34/Conv2D] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:827] [SubGraphOpt][Compile][ParalCompOp] Thread[281464432222704] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:867] Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank1_new_sub_graph1[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:130] subgraph 0 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:714] Solution: Please contact support engineer. [[{{node GeOp3_0}}]]使用root用户进行推理,无报错
  • [问题求助] AscendHub下载的TF镜像,在线推理报错
    一、问题现象(附报错日志上下文):[EVENT] FMK(2277,python3):2023-01-14-03:04:11.647.742 [tf_adapter/common/adp_logger.cc:27][TF_ADAPTER] [tf_adapter/optimizers/om_partition_subgraphs_pass.cc:2136] OMPartition subgraph_3 begin. mix_compile_mode is False, iterations_per_loop is 1, input_shape: , dynamic_dims:[EVENT] FMK(2277,python3):2023-01-14-03:04:11.691.310 [tf_adapter/common/adp_logger.cc:27][TF_ADAPTER] [tf_adapter/optimizers/om_partition_subgraphs_pass.cc:2247] OMPartition subgraph_3 markForPartition success.[EVENT] FMK(2277,python3):2023-01-14-03:04:11.719.649 [tf_adapter/common/adp_logger.cc:27][TF_ADAPTER] [tf_adapter/optimizers/om_partition_subgraphs_pass.cc:2260] OMPartition subgraph_3 SubgraphsInFunctions succeed.[EVENT] FMK(2277,python3):2023-01-14-03:04:11.719.746 [tf_adapter/common/adp_logger.cc:27][TF_ADAPTER] [tf_adapter/optimizers/om_partition_subgraphs_pass.cc:2271] OMPartition subgraph_3 succeed. [72 ms]2023-01-14 03:04:11.719841: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-14 03:04:11.719925: W tf_adapter/util/npu_attrs.cc:1077] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-14 03:04:11.719943: W tf_adapter/util/npu_attrs.cc:1207] The op_select_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-14 03:04:11.719955: W tf_adapter/util/npu_attrs.cc:1211] The optypelist_for_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-14 03:04:11.723206: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-14 03:04:11.723289: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-14 03:04:11.723375: W tf_adapter/util/npu_attrs.cc:1077] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-14 03:04:11.723392: W tf_adapter/util/npu_attrs.cc:1207] The op_select_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-14 03:04:11.723406: W tf_adapter/util/npu_attrs.cc:1211] The optypelist_for_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-14 03:04:11.723567: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-14 03:04:11.723643: W tf_adapter/util/npu_attrs.cc:1077] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-14 03:04:11.723657: W tf_adapter/util/npu_attrs.cc:1207] The op_select_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-14 03:04:11.723668: W tf_adapter/util/npu_attrs.cc:1211] The optypelist_for_implmode option IS DEPRECATED. It will be removed in a future version. Please use op_precision_mode instead2023-01-14 03:04:11.723742: W tf_adapter/util/npu_attrs.cc:861] The enable_data_pre_proc option IS DEPRECATED. It will be removed in a future version.2023-01-14 03:04:11.724170: W tf_adapter/util/npu_attrs.cc:148] [GePlugin] DEVICE_ID and ASCEND_DEVICE_ID is none, use default device id : 0, if set session_device_id, session_device_id has a higher priority[ERROR] TDT(2277,python3):2023-01-14-03:06:11.740.978 [log.cpp:143]deviceId=0: drvtype=8 drv hdc server create failed, ret = 31,[hdc_server.cpp:136:Init]2277 Msg: hdc server create error[ERROR] TDT(2277,python3):2023-01-14-03:06:11.741.025 [log.cpp:143]TuningDataReceive hdc server init failed, deviceId=0,[tuning_data_receive.cpp:81:InitDirectly]2277 Msg: call member function before instance initialed[ERROR] TDT(2277,python3):2023-01-14-03:06:11.741.051 [log.cpp:143]TuningDataReceive InitDirectly failed, logicId=0,[tuning_data_receive.cpp:102:Init]2277 Msg: hdc server create error[ERROR] TDT(2277,python3):2023-01-14-03:06:11.741.073 [log.cpp:143]log summary receive init failed,[tdt_host.cpp:683:TdtOutFeedInit]2277 Msg: hdc server create error[TRACE] GE(2277,python3):2023-01-14-03:06:11.742.422 [status:INIT] [ge_api.cc:194]2277 GEInitializeImpl:GEInitialize start[EVENT] PROFILING(2277,python3):2023-01-14-03:06:11.770.387 [msprof_callback_impl.cpp:78] >>> (tid:2277) MsprofCtrlCallback called, type: 255[EVENT] PROFILING(2277,python3):2023-01-14-03:06:11.770.491 [prof_acl_mgr.cpp:1190] >>> (tid:2277) Init profiling for dynamic profiling[TRACE] GE(2277,python3):2023-01-14-03:06:11.803.836 [status:RUNNING] [ge_api.cc:248]2277 GEInitializeImpl:Initializing environment[EVENT] GE(2277,python3):2023-01-14-03:06:11.803.914 [gelib.cc:55]2277 Initialize:[GEPERFTRACE] GE Init Start[ERROR] RUNTIME(2277,python3):2023-01-14-03:06:11.804.931 [runtime.cc:1051]2277 CheckHaveDevice:[INIT][SYS]Call halGetDeviceInfo failed: drvRet=4, module type=0, info type=1.[ERROR] RUNTIME(2277,python3):2023-01-14-03:06:11.805.800 [runtime.cc:1003]2277 InitSocType:[INIT][SYS]Call halGetDeviceInfo failed: drvRetCode=4, module type=0, info type=1.[ERROR] RUNTIME(2277,python3):2023-01-14-03:06:11.805.835 [runtime.cc:1075]2277 Init:[INIT][SYS]Init SocType failed.[ERROR] RUNTIME(2277,python3):2023-01-14-03:06:11.805.892 [api_c.cc:2967]2277 rtGetSocVersion:[INIT][SYS]ErrCode=507000, desc=[instance pointer null], InnerCode=0x70b0001[ERROR] GE(2277,python3):2023-01-14-03:06:11.805.925 [gelib.cc:276]2277 SetRTSocVersion: ErrorNo: 507000() [INIT][SYS][Get][SocVersion]rtGetSocVersion failed[ERROR] GE(2277,python3):2023-01-14-03:06:11.805.947 [gelib.cc:68]2277 Initialize: ErrorNo: 4294967295(failed) [INIT][SYS][Set][RTSocVersion]GeLib initial: SetRTSocVersion failed.[ERROR] GE(2277,python3):2023-01-14-03:06:11.805.985 [ge_api.cc:254]2277 GEInitializeImpl: ErrorNo: 1343229953(GEInitialize Failed.) [INIT][SYS][Init][GELib]Failed, error code = 4294967295[ERROR] FMK(2277,python3):2023-01-14-03:06:14.806.114 [tf_adapter/common/adp_logger.cc:24][TF_ADAPTER] [tf_adapter/util/ge_plugin.cc:251] [GePlugin] Initialize ge failed, ret : failed2023-01-14 03:06:14.807266: F tf_adapter/util/ge_plugin.cc:253] [GePlugin] Initialize ge failed, ret : failedError Message is :EL0003: The argument is invalid.Solution: Try again with a valid argument.TraceBack (most recent call last):Check param failed, rtInstance can not be NULL![FUNC:rtGetSocVersion][FILE:api_c.cc][LINE:2967]rtGetSocVersion failed.[FUNC:SetRTSocVersion][FILE:gelib.cc][LINE:276]SetRTSocVersion failed.[FUNC:Initialize][FILE:gelib.cc][LINE:69]Aborted (core dumped)二、软件版本:-- CANN 版本 (e.g., CANN 3.0.x,5.x.x): CANN 6.0.rc1--Tensorflow/Pytorch/MindSpore 版本: Tensorflow 1.15.0--Python 版本 (e.g., Python 3.7.5): Python 3.7.5--操作系统版本 (e.g., Ubuntu 18.04): Ubuntu 18.04使用ascend-tensorflow 22.0.RC3镜像请问有人遇到过类似问题吗?
  • [问题求助] Pytorch推理脚本和模型如何在昇腾服务器上正常推理
    网上查到的资料显示要想在昇腾服务器上使用cuda训练的pytorch模型,需要将模型转为om格式,但是没找到推理脚本改造的文档,请问在昇腾服务器上用pytorch推理有没有官方资料可以学习?另外还有TensorFlow的,请大神们指点,谢谢