-
【操作步骤&问题现象】1、安装MindSpore后运行失败【截图信息】【日志信息】(vgg_train4) user1@ascend:~$ python -c "import mindspore;mindspore.run_check()"MindSpore version: 1.6.1[ERROR] KERNEL(58768,ffffb6d865c0,python):2022-04-26-14:11:00.794.875 [mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_kernel_compile.cc:143] PrintInfo] Job id:2, name :mul_1256030608483155855_0, message:2022-04-26 14:11:00.791086: Process Job Failed[ERROR] KERNEL(58768,ffffb6d865c0,python):2022-04-26-14:11:00.794.953 [mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_kernel_compile.cc:143] PrintInfo] Job id:2, name :mul_1256030608483155855_0, message:2022-04-26 14:11:00.791196: Job json string:{"job_content":{"SocInfo":{"autoTilingMode":"NO_TUNE","coreNum":"","coreType":"","deviceId":"0","l1Fusion":"false","l2Fusion":"false","l2Mode":"2","mdl_bank_path":"","offlineTune":false,"op_bank_path":"","op_bank_update":false,"op_debug_dir":"./rank_0/","op_debug_level":"3","op_impl_mode":"","op_impl_mode_list":[],"socVersion":"Ascend910B","vector_fp_ceiling":""},"full_name":"Default/Mul-op0","fusion_op_name":"mul_1256030608483155855_0","graph_name":"","l1_size":-1,"op_list":[{"name":"x_0","output_desc":[{"L1_addr_offset":0,"L1_fusion_type":-1,"L1_workspace_size":-1,"addr_type":0,"data_type":"float32","dtype":"float32","format":"ND","name":"x_0","ori_format":"NCHW","ori_shape":[3],"param_type":"required","range":[[3,3]],"shape":[3],"slice_offset":[],"split_index":0,"total_shape":[],"valid":true,"valid_shape":[]}],"type":"Data"},{"name":"y_0","output_desc":[{"L1_addr_offset":0,"L1_fusion_type":-1,"L1_workspace_size":-1,"addr_type":0,"data_type":"float32","dtype":"float32","format":"ND","name":"y_0","ori_format":"NCHW","ori_shape":[3],"param_type":"required","range":[[3,3]],"shape":[3],"slice_offset":[],"split_index":0,"total_shape":[],"valid":true,"valid_shape":[]}],"type":"Data"},{"build_type":"accurately_build","dynamic_compile_static":false,"func_name":"mul","input_desc":[{"L1_addr_offset":0,"L1_fusion_type":-1,"L1_workspace_size":-1,"addr_type":0,"data_type":"float32","dtype":"float32","format":"ND","name":"x_0","ori_format":"NCHW","ori_shape":[3],"param_type":"required","range":[[3,3]],"shape":[3],"slice_offset":[],"split_index":0,"total_shape":[],"valid":true,"valid_shape":[]},{"L1_addr_offset":0,"L1_fusion_type":-1,"L1_workspace_size":-1,"addr_type":0,"data_type":"float32","dtype":"float32","format":"ND","name":"y_0","ori_format":"NCHW","ori_shape":[3],"param_type":"required","range":[[3,3]],"shape":[3],"slice_offset":[],"split_index":0,"total_shape":[],"valid":true,"valid_shape":[]}],"int64mode":false,"max_kernel_id":10,"miss_support_info":"","module_name":"impl.mul","name":"Default/Mul-op0","op_name":"mul_1256030608483155855_0","ori_name":["Default/Mul-op0"],"output_data_desc":[{"L1_addr_offset":0,"L1_fusion_type":-1,"L1_workspace_size":-1,"addr_type":0,"data_type":"float32","dtype":"float32","format":"ND","ori_format":"NCHW","ori_shape":[3],"param_type":"required","range":[[3,3]],"shape":[3],"slice_offset":[],"split_index":0,"total_shape":[],"valid":true,"valid_shape":[]}],"output_desc":[{"L1_addr_offset":0,"L1_fusion_type":-1,"L1_workspace_size":-1,"addr_type":0,"data_type":"float32","dtype":"float32","format":"ND","name":"output","ori_format":"NCHW","ori_shape":[3],"param_type":"required","range":[[3,3]],"shape":[3],"slice_offset":[],"split_index":0,"total_shape":[],"valid":true,"valid_shape":[]}],"pattern":"ElemWise","py_module_path":"/usr/local/Ascend/opp/op_impl/built-in/ai_core/tbe","type":"Mul","unknown_shape":false}],"scope_id":-1},"job_id":2,"job_type":"Compile","source_id":0}[ERROR] KERNEL(58768,ffffb6d865c0,python):2022-04-26-14:11:00.795.010 [mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_kernel_compile.cc:143] PrintInfo] Job id:2, name :mul_1256030608483155855_0, message:2022-04-26 14:11:00.793943: Error message:Traceback (most recent call last): File "/opt/miniconda3/envs/vgg_train4/lib/python3.9/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/tbe_job_manager.py", line 124, in job_handler res = func(job) File "/opt/miniconda3/envs/vgg_train4/lib/python3.9/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/tbe_job_manager.py", line 193, in compile_handler return self.single_op_build(job) File "/opt/miniconda3/envs/vgg_train4/lib/python3.9/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/tbe_job_manager.py", line 205, in single_op_build res = build_single_pre_op(job) File "/opt/miniconda3/envs/vgg_train4/lib/python3.9/site-packages/mindspore/_extends/parallel_compile/tbe_compiler/tbe_adapter.py", line 484, in build_single_pre_op dispatch_single_op_compile_task(job.source_id, job.id, l1_size, op_module_name, op_name, op_type, op_func_name, File "/opt/miniconda3/envs/vgg_train4/lib/python3.9/site-packages/te_fusion/compile_task_manager.py", line 414, in dispatch_single_op_compile_task inputs, outputs, attrs, options = op_argsTypeError: cannot unpack non-iterable bool object[ERROR] KERNEL(58768,ffffb6d865c0,python):2022-04-26-14:11:00.795.081 [mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_kernel_compile.cc:143] PrintInfo] Job id:2, name :mul_1256030608483155855_0, message:2022-04-26 14:11:00.794002: Exception message:cannot unpack non-iterable bool object[CRITICAL] KERNEL(58768,ffffb6d865c0,python):2022-04-26-14:11:00.795.129 [mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_kernel_compile.cc:288] PrintCompileResult] mul_1256030608483155855_0 Compile running failed.MindSpore running check failed.mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_kernel_compile.cc:288 PrintCompileResult] mul_1256030608483155855_0 Compile running failed.
-
【功能模块】cpu\gather_cpu_kernel.cc【操作步骤&问题现象】1、正常运行过程中,固定跑到第6个epoch 第 10 个 iteration 会报下列错误2、RuntimeError: mindspore\ccsrc\runtime\device\cpu\cpu_kernel_runtime.cc:473 Run] mindspore\ccsrc\backend\kernel_compiler\cpu\gather_cpu_kernel.cc:112 ParallelRun] SyncRun error!【截图信息】【日志信息】(可选,上传日志内容或者附件)训练代码 见附件, txt格式
-
JupyterLab里有很多kernel,这些kernel是什么样的概念呢? ![image.png](https://bbs-img.huaweicloud.com/data/forums/attachment/forum/20223/23/1648039083033355175.png) 那么接着往下看kernel的解释: Jupyter notebooks run in kernels in Jupyter notebook environments. The number of notebook Juypter kernels started in an environment depends on the environment type: CPU or GPU environments When you open a notebook in edit mode, exactly one interactive session connects to a Jupyter kernel for the notebook language and the environment runtime that you select. The runtime is started per user and not per notebook. This means that if you open a second notebook with the same environment definition, a second kernel is started in that runtime. Resources are shared. If you want to avoid sharing runtime resources, you must associate each notebook with its own environment definition. Important: Stopping a notebook kernel doesn't stop the environment runtime in which the kernel is started because other notebook kernels could still be active in that runtime. Only stop an environment runtime if you are sure that no kernels are active. If necessary, you can restart or reconnect to a kernel. When you restart a kernel, the kernel is stopped and then started in the same session, but all execution results are lost. When you reconnect to a kernel after losing a connection, the notebook is connected to the same kernel session, and all previous execution results which were saved are available. If you accidentally close your notebook browser window while the notebook is still running, or are logged out by the system if your job runs very long, the kernel will remain active. When you reopen the same notebook, the notebook is connected to the same kernel and all output cells are retained. The execution progress of a notebook can only be restored for notebooks that run in a local kernel. 粗暴的翻译一下就是这样: ------------ Jupyter笔记本电脑在Jupyter笔记本环境中的内核中运行 在环境中启动的笔记本Juypter内核数量取决于环境类型: **CPU或GPU环境** 当您在编辑模式下打开笔记本时,只有一个交互式会话连接到您选择的笔记本语言和环境运行时的Jupyter内核。运行时是每个用户而不是每个笔记本启动的。这意味着如果您打开具有相同环境定义的第二个笔记本,则在该运行时启动第二个内核。资源是共享的。如果要避免共享运行时资源,则必须将每个笔记本与其自己的环境定义相关联。 重要提示:停止笔记本内核不会停止内核启动的环境运行时,因为其他笔记本内核在该运行时仍可能处于活动状态。如果确定没有内核处于活动状态,则仅停止环境运行时。 如有必要,您可以重新启动或重新连接到内核。重新启动内核时,内核将停止,然后在同一会话中启动,但所有执行结果都会丢失。失去连接后重新连接内核时,笔记本将连接到同一内核会话,并且所有先前保存的执行结果均可用。 如果笔记本仍在运行时意外关闭笔记本浏览器窗口,或者如果作业运行时间很长,系统会注销,则内核将保持活动状态。重新打开相同的笔记本时,笔记本连接到相同的内核,并保留所有输出单元。笔记本的执行进度只能针对在本地内核中运行的笔记本恢复。
-
1. 开启pageowner功能1.1 下载对应版本内核源码uname –a 查看对应内核的version-release方式一:下载 https://gitee.com/openeuler/kernel源码并切换到对应tag方式二:下载对应版本号的src.rpm可 配置source repo:dnf download –source 下载对应src.rpm也可通过wget 对应repo源的软件包地址,openEuler-20.03-LTS-SP1的源码包路径如下:https://repo.openeuler.org/openEuler-20.03-LTS-SP1/source/Packages/https://mirrors.huaweicloud.com/openeuler/openEuler-20.03-LTS-SP1/source/Packages/ 1.2 开启page_owner选项构建软件包以下均以arm架构下下载src.rpm的形式为例安装src.rpmrpm –ivh kernel-xxxxx.src.rpmcd /root/rpmbuild/SPECSdnf builddep kernel.speccd /root/rpmbuild/SOURCEStar –zxvf kernel.tar.gz (如果没安装tar dnf install tar)cd kernelmake menuconfig键入/ 进入搜索界面搜索 page_owner 回车确认键入1 进入配置界面键入Y 选中page_owner选项然后退出grep –rn “PAGE_OWNER” .config 查看选项是否生效使用以下命令构建export INSTALL_MOD_STRIP=1make binrpm-pkg -j200 生成的rpm存放在/root/rpmbuild/RPMS/$basearch/下kernel-xxxx.rpmkernel-headers-xxxx.rpm该软件包的release为空和社区下载的名字明显不同,后面区分grub选项用1.3 获取page_owner数据安装kernel rpm包(只需要安装主包)rpm –ivh –force kernel-xxxxx.rpm 确认grub的菜单是否存在新内核在新的启动项后增加page_owner=on选项以上操作均可以在/boot目录下的grub.cfg或/etc/grub2.cfg中确认2. 获取page_owner数据2.1重启统计page_ownercd /root/rpmbuild/SOURCES/kernel/tools/vmmake –jcat /sys/kernel/debug/page_owner > page_owner_full.txtgrep –v ^PFN page_owner_full.txt > page_owner.txt./page_owner_sort page_owner.txt sorted_page_owner.txt2.2 计算page_owner数据总值最终查看sorted_page_owner.txt文件其中每块数据首行“ times:”前的数据为申请的次数:第二行order后的数据为申请的page数量,结果为2^n个page页大小根据 getconf PAGESIZE或getconf PAGE_SIZE查看最终启动后内存占用总量为 (∑(<times> * 2 ^ <order>)) * <page_size>2.3 简单的计算脚本如下:FILE_NAME = "sorted_page_owner.txt" # 根据获取的page_owenr结果文件位置修改 PAGE_SIZE = 4028 # 根据getconf PAGESIZE或getconf PAGE_SIZE的结果进行修改 def main(fn): result = 0 with open(fn, "r") as f: times = 0 order = 0 for line in f: if " times:" in line: times = int(line.split(" ")[0]) if "order" in line and times > 0: try: line = line.replace(",", "") order = int(line.split(" ")[4]) result += times * int(pow(2, order)) print("times: [{}], order: [{}], result: [{}]".format(times, order, result)) except Exception as e: print(e) finally: times = 0 order = 0 return result if __name__ == "__main__": result = main(FILE_NAME) print("sum: {}M".format((result * PAGE_SIZE / 1024 / 1024)))
-
【功能模块】环境:Linux + r1.5 + python3.7.5 + CPU参考 https://www.mindspore.cn/install 和https://gitee.com/mindspore/mindspore/blob/r1.5/mindspore/ccsrc/fl/server/kernel/round/start_fl_job_kernel.cc【操作步骤&问题现象】问题1:从gitee上下载的代码,分支是r1.5,用上述链接编译后生成的确是1.6版本的.whl文件(这里我拉取的分支只有master和r1.5,没有拉取过r1.6的代码呀,很奇怪)问题2:前几次编译还是r1.5的代码,能成功编译。这次编译修改了一部分代码(mindspore/schema/fl.fbs和mindspore/mindspore/mindspore/ccsrc/fl/server/kernel/round/get_model_kernel.cc和get_model_kernel.h后报错,发现代码是1.6版本的代码,这个编译是咋回事儿咧?从gitee上自动拉取了最新版本的代码吗?不是很能理解编译时start_job_kernel.cc报错(在此注明:没有手动改动过start_job_kernel.cc的代码)这个start_fl_job_req->sign_data()是1.6版本的,1.5版本没有这部分代码(在此注明:没有手动改动过start_job_kernel.cc的代码)r1.5版本代码里没有看到judgeFlJobCert()r1.6版本代码:请问是在哪个文件是在编译时有对应版本的设置吗?应该怎么解决这个问题呢?麻烦看到后能够回复一下下,谢谢各位老师
-
【操作步骤&问题现象】专家您好,我遇到了跟该版主类似的问题:动态图模式下,求grads报错未显式调用的算子Transpose_MindSpore_昇腾论坛_华为云论坛 (huaweicloud.com)我报错信息为:PrintUnsupportedTypeException] Select GPU kernel op[Mul] fail! Incompatible data type!而且调试中间还出现了warning: Fail to find bprop function for Multinomial. fn: None Fail to find bprop function for UniformReal. fn: None我使用专家给出的生成计算图的方式,观察到生成的计算图中的确出现了bool类型,但是在产生bool类型的算子上下文中都没有使用bool类型的算子,这个产生bool类型的算子就像是凭空出现的。希望您能不吝赐教,帮助我解决这个问题,谢谢您!【截图信息】【日志】
-
【功能模块】mindspore 1.5.0ascend 910【操作步骤&问题现象】在执行下面的代码时,loss=train_network(input_data, target)这句出错。报错为:错误和avgpool3d算子有关,可以确定的是:前向过程已经完成(在这个报错之前有个损失函数输入数据类型的错误,已经解决),avgpool3d可以正常接收输入数据,并输出正确的维度。错误应该发生在反向过程。错误的意思貌似是说kernel_size的维度不对,但kernel_size正常应该就是三维?输入的数据是5维的。另外,代码中avgpool3d算子定义如下:
-
【功能模块】mindspore 1.5.0gpu【操作步骤&问题现象】1、无论如何改变kernel_size参数的大小,输出维度都不会改变。如代码所示,当kernel_size为(7, 7, 7)时,正确的输出维度应为(2, 64, 32, 112, 112),但输出维度始终为(2, 64, 35, 115, 115)。另外在torch中测试,输出维度正常。【截图信息】mindspore代码:torch代码:【日志信息】(可选,上传日志内容或者附件)rose_is_blue 发表于2022-01-07 17:12:21 2022-01-07 17:12:21 最后回复 rose_is_blue 2022-01-08 09:59:282952 1
-
问题描述:在modelarts平台,使用单卡ascend910 cpu24核、kernel为mindspore的notebook运行程序,cpu使用率很低。以下是在上述环境与另一个modelarts平台 cpu2核、kernel为pytorch1.4的notebook运行相同程序时的性能对比:单卡ascend910 cpu24核、kernel为mindspore的notebook(kernel重启过后运行的):cpu利用率一直小于10%,基本出于%5的状态,程序运行时间为169.93scpu2核、kernel为pytorch1.4的notebook:(kernel重启过后运行的)cpu利用率始终大于50%,大多数时间处于95%以上,运行时间仅10.39s请问上述情况可能的原因是什么,modelarts平台,单卡ascend910 cpu24核、kernel为mindspore的notebook如何提高cpu利用率呢?
-
【操作步骤&问题现象】1、执行代码训练后,可以正常训练8个iteration,然后就报Launch kernel exception: Default/Dropout-op3281,为什么之前的迭代没问题?2、请问一下那些warning的信息要怎么解决,不然输出也太混乱了。【截图信息】【日志信息】(可选,上传日志内容或者附件)日志请查看附件。
-
漏洞名称 : Linux Kernel TIPC远程代码执行漏洞组件名称 : Linux Kernel 影响范围 : 5.10-rc1漏洞类型 : 远程代码执行利用条件 :1、用户认证:不需要用户认证2、触发方式:远程综合评价 : <综合评定利用难度>:未知。<综合评定威胁等级>:高危,能造成远程代码执行。漏洞分析:1、组件介绍Linux kernel是美国Linux基金会发布的开源操作系统Linux所使用的内核。Linux内核是一种开源的类Unix操作系统宏内核。整个Linux操作系统家族基于该内核部署在传统计算机平台(如个人计算机和服务器)。2、漏洞描述近日,安全团队监测到一则Linux Kernel TIPC远程代码执行漏洞的信息,漏洞编号:CVE-2021-43267,漏洞威胁等级:高危。TIPC是一种传输层协议,专为在动态集群环境中运行的节点而设计,以比其他协议(如TCP)更高效和容错的方式可靠地相互通信。远程攻击者可以通过TIPC功能以利用MSG_CRYPTO消息类型大小的验证不足来进行攻击。影响范围:Linux操作系统在全世界范围内应用十分广泛。在国内主要应用于浙江、江苏、广东、上海、北京等地区。目前受影响的Linux kernel版本:5.10-rc1解决方案:1、如何检测组件系统版本使用uname -r指令查看Linux内核版本。2、官方修复建议当前官方已发布最新版本,建议受影响的用户及时更新升级到最新版本。链接如下:https://www.kernel.org/
-
```cpp /** * Copyright 2020 Huawei Technologies Co., Ltd * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ #include "backend/kernel_compiler/host/host_kernel_mod.h" #include //引入智能指针,动态分配对象以及当对象不再需要时自动执行清理 #include //引入vector容器 #include #include //将自动包含通过 要帮助管理其键/值对键入的元素 #include "runtime/mem.h" #include "utils/ms_context.h" #include "runtime/device/kernel_runtime.h" #include "runtime/device/ascend/executor/host_dynamic_kernel.h" //使用命名空间mindspore namespace mindspore { namespace kernel {//在命名空间mindspore下建立命名空间kernel void HostKernelFactory::Registe(const std::string &name, HostKernelCreater &&fun) { hostKernelMap_.emplace(name, std::move(fun));//当容器中现有元素的键与这个元素的键不同时,构造这个元素 } std::shared_ptr HostKernelFactory::Get(const std::string &name) { const auto &map = Get().hostKernelMap_; auto it = map.find(name);//找到namw出现的位置赋给it if (it != map.end() && it->second) {//进行遍历查找 return (it->second)(); } return nullptr; } HostKernelFactory &HostKernelFactory::Get() { static HostKernelFactory instance; return instance; } const std::vector &HostKernelMod::GetInputSizeList() const { return input_size_list_; }//得到输入大小列表 const std::vector &HostKernelMod::GetOutputSizeList() const { return output_size_list_; }//得到输出大小列表 const std::vector &HostKernelMod::GetWorkspaceSizeList() const { return workspace_size_list_; }//运行空间大小表 bool HostKernelMod::Init(const AnfNodePtr &anf_node) {//初始化 MS_EXCEPTION_IF_NULL(anf_node);//节点信息是否为空 size_t input_num = AnfAlgo::GetInputTensorNum(anf_node);//获取输入张量 size_t output_num = AnfAlgo::GetOutputTensorNum(anf_node);//获取输出张量 //在输入值的范围内进行遍历 for (size_t i = 0; i input_num; i++) { std::vector shape_i = AnfAlgo::GetInputDeviceShape(anf_node, i);//获取输入设备形状 TypePtr type_ptr = TypeIdToType(AnfAlgo::GetInputDeviceDataType(anf_node, i));//获取输入设备数据类型 MS_EXCEPTION_IF_NULL(type_ptr); int64_t size_i = 1; for (size_t j = 0; j shape_i.size(); j++) { size_i = LongMulWithOverflowCheck(size_i, static_cast(shape_i[j]));//将size_i和shape_i[j]进行带溢出检查的相乘 } size_t type_byte = GetTypeByte(type_ptr);//得到类型字节 if (type_byte == 0) { return false; } size_i = LongMulWithOverflowCheck(size_i, SizeToInt(type_byte));//将乘法计算结果赋值给size_t input_size_list_.push_back(LongToSize(size_i));//结果存储到input_size_list_ } //在输出值的范围内进行遍历 for (size_t i = 0; i output_num; i++) { std::vector shape_i = AnfAlgo::GetOutputDeviceShape(anf_node, i);////获取输出设备形状 TypePtr type_ptr = TypeIdToType(AnfAlgo::GetOutputDeviceDataType(anf_node, i));//获取输出设备数据类型 MS_EXCEPTION_IF_NULL(type_ptr); int64_t size_i = 1; for (size_t j = 0; j shape_i.size(); j++) { size_i = LongMulWithOverflowCheck(size_i, static_cast(shape_i[j]));//将size_i和shape_i[j]进行带溢出检查的相乘 } size_t type_byte = GetTypeByte(type_ptr);//得到类型字节 if (type_byte == 0) { return false; } size_i = LongMulWithOverflowCheck(size_i, SizeToInt(type_byte));//将乘法计算结果赋值给size_t output_size_list_.push_back(LongToSize(size_i));//结果存储到output_size_list_ } return true; } bool HostKernelMod::Launch(const std::vector &inputs, const std::vector &workspace, const std::vector &outputs, void *stream_ptr) { return true; } std::vector HostKernelMod::GenTask(const std::vector &, const std::vector &, const std::vector &, uint32_t) { return {}; } } // namespace kernel } // namespace mindspore ```
-
** "mindspore\lite\src\lite_kernel_util.cc"注释2** ======================================= ```python //整合输出张量函数 std::vector LiteKernelUtil::SubgraphOutputTensors(const std::vector &kernels) { std::vector output_tensors;//输出张量 std::vector output_nodes = SubgraphOutputNodes(kernels);//图输出节点 for (const auto &output_kernel : output_nodes) {//遍历输出节点 auto &outer_out_kernels = output_kernel->out_kernels();//存储内核 auto &out_kernel_out_tensors = output_kernel->out_tensors(); for (auto out_kernel_out_tensor : out_kernel_out_tensors) {//遍历输出内核和张量 if (out_kernel_out_tensor->IsGraphOutput()) {//判断是否为GraphOutput if (!lite::IsContain(output_tensors, out_kernel_out_tensor)) { output_tensors.push_back(out_kernel_out_tensor);//添加相应的输出张量 } } } if (!outer_out_kernels.empty()) {//判断是否为空 for (auto outer_out_kernel : outer_out_kernels) {//遍历输出内核 auto iter = std::find(kernels.begin(), kernels.end(), outer_out_kernel); if (iter != kernels.end()) { continue; } auto &outer_out_kernel_in_tensors = outer_out_kernel->in_tensors();//输入张量 for (auto out_kernel_out_tensor : out_kernel_out_tensors) { auto outer_out_kernel_in_tensors_iter = std::find(outer_out_kernel_in_tensors.begin(), outer_out_kernel_in_tensors.end(), out_kernel_out_tensor); if (outer_out_kernel_in_tensors_iter != outer_out_kernel_in_tensors.end()) { if (!lite::IsContain(output_tensors, out_kernel_out_tensor)) { output_tensors.push_back(out_kernel_out_tensor);//添加相应Tensors } } } } } } return output_tensors;//返回输出 } //此函数用于对逻辑排序内核 int LiteKernelUtil::TopologicalSortKernels(std::vector *kernels) { auto old_kernels = *kernels; kernels->clear();//清空内核 std::queue kernel_queue;//创建一个内核queue for (auto kernel : old_kernels) {//遍历内核 if (kernel->in_kernels().empty()) { kernel_queue.push(kernel);//添加相应内核 kernels->emplace_back(kernel); } } while (!kernel_queue.empty()) {判断内核是否为空 auto cur_kernel = kernel_queue.front();//取首项 kernel_queue.pop(); MS_ASSERT(cur_kernel != nullptr); auto next_kernels = cur_kernel->out_kernels(); for (auto next_kernel : next_kernels) {//进行逻辑排序内核 auto in_kernels = next_kernel->in_kernels(); if (lite::IsContain(*kernels, const_cast(next_kernel))) { MS_LOG(ERROR) "TopologicalSortKernels failed, loop exist"; return RET_ERROR; } if (std::all_of(in_kernels.begin(), in_kernels.end(), [&](const kernel::LiteKernel *in_kernel) { return lite::IsContain(*kernels, const_cast(in_kernel)); })) { kernel_queue.push(next_kernel);//添加排序后的内核 } } } if (kernels->size() != old_kernels.size()) {//输出失败以及排序前后内核大小 MS_LOG(ERROR) "TopologicalSortKernels failed, kernels size before sort: " old_kernels.size() ", kernels size after sort: " kernels->size(); return RET_ERROR; } return RET_OK; } //初始化张量的初始refcount void LiteKernelUtil::InitTensorInitRefCount(const std::vector &kernels) { for (auto *kernel : kernels) { kernel->InitOutTensorInitRefCount(); } } int LiteKernelUtil::SetInput(const LiteKernel &kernelMod, const std::vector &inputs) { return -1; } //bool类型函数是否开关调用 bool LiteKernelUtil::IsSwitchCall(kernel::LiteKernel *kernel) { auto *subgraph_kernel = reinterpret_cast(kernel);//转换kernel类型 if (subgraph_kernel == nullptr) { return false; } for (auto &node : subgraph_kernel->nodes()) {//遍历进行判断是否需要开关调用 if (node->Type() == schema::PrimitiveType_Switch && InputsContainsSpecificNode(node, schema::PrimitiveType_PartialFusion) && node->out_kernels().size() == 1 && node->out_kernels().front()->Type() == schema::PrimitiveType_Call) { return true; } } return false; } //获取输入特定节点函数 kernel::LiteKernel *LiteKernelUtil::GetInputsSpecificNode(const kernel::LiteKernel *kernel, const schema::PrimitiveType &primitive_type) { for (auto input : kernel->in_kernels()) { if (input->Type() == primitive_type) { return input; } } return nullptr; } //输入包含特定节点函数 bool LiteKernelUtil::InputsContainsSpecificNode(const kernel::LiteKernel *kernel, const schema::PrimitiveType &primitive_type) { if (GetInputsSpecificNode(kernel, primitive_type)) { return true; } return false; } } // namespace mindspore::kernel ```
-
/** * Copyright 2020 Huawei Technologies Co., Ltd * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. *///导入自定义的文件和系统自带的文件#include "backend/kernel_compiler/cpu/mkldnn/pooling_avg_grad_cpu_kernel.h"#include <string>#include <utility>#include <algorithm>#include "utils/ms_utils.h"#include "backend/kernel_compiler/cpu/mkldnn/mkl_kernel_engine.h"#include "runtime/device/cpu/cpu_device_address.h"namespace mindspore {//声明一个空间namespace kernel {//空间进行嵌套void AvgPoolingGradCPUKernel::InitKernel(const CNodePtr &kernel_node) { MS_EXCEPTION_IF_NULL(kernel_node);//空指针异常 //函数调用 std::vector<size_t> src_shape = AnfAlgo::GetInputDeviceShape(kernel_node, 0); std::vector<size_t> dst_shape = AnfAlgo::GetInputDeviceShape(kernel_node, 1); dnnl::memory::desc src_desc = GetDefaultMemDesc(src_shape); dnnl::memory::desc dst_desc = GetDefaultMemDesc(dst_shape); //变量声明 std::vector<int> origin_kernel_sizes; std::vector<int> strides; std::vector<int64_t> kernel_sizes_me = AnfAlgo::GetNodeAttr<std::vector<int64_t>>(kernel_node, KERNEL_SIZE); std::vector<int64_t> strides_me = AnfAlgo::GetNodeAttr<std::vector<int64_t>>(kernel_node, STRIDES); (void)std::transform(kernel_sizes_me.begin(), kernel_sizes_me.end(), std::back_inserter(origin_kernel_sizes), [](const int64_t &value) { return static_cast<int>(value); }); (void)std::transform(strides_me.begin(), strides_me.end(), std::back_inserter(strides), [](const int64_t &value) { return static_cast<int>(value); }); //如果origin_kernel或者strides.size的大小,记录错误日志:无效大小 if (origin_kernel_sizes.size() != 4 || strides.size() != 4) { MS_LOG(EXCEPTION) << "Invalid kernel size " << origin_kernel_sizes.size() << " or stride size " << strides.size(); } std::vector<int> stride{strides[2], strides[3]};//创建集合 dnnl::memory::dims strides_dims{strides[2], strides[3]}; dnnl::memory::dims kernels_dims{origin_kernel_sizes[2], origin_kernel_sizes[3]}; const std::string pad_mode = AnfAlgo::GetNodeAttr<std::string>(kernel_node, PAD_MODE); //声明变量 std::vector<int> int_padding_l; std::vector<int> int_padding_r; //函数调用 std::vector<size_t> kernel_size({IntToSize(origin_kernel_sizes[2]), IntToSize(origin_kernel_sizes[3])}); std::vector<int> dummy_dilation{1, 1}; GetPadding(kernel_node, pad_mode, src_shape, kernel_size, stride, &int_padding_l, &int_padding_r, dummy_dilation); //如果origin_kernel或者strides.size的大小,记录错误日志 if (int_padding_l.size() != 2 || int_padding_r.size() != 2) { MS_LOG(EXCEPTION) << "Pooling avg get padding failed"; } dnnl::memory::dims padding_l{int_padding_l[0], int_padding_l[1]}; dnnl::memory::dims padding_r{int_padding_r[0], int_padding_r[1]}; // pooling_avg forward description dnnl::pooling_forward::desc desc = dnnl::pooling_forward::desc(dnnl::prop_kind::forward_training, dnnl::algorithm::pooling_avg, src_desc, dst_desc, strides_dims, kernels_dims, padding_l, padding_r); auto prim_desc = dnnl::pooling_forward::primitive_desc(desc, MKLKernelEngine::Get().engine()); // pooling_avg backward description dnnl::pooling_backward::desc backward_desc = dnnl::pooling_backward::desc( dnnl::algorithm::pooling_avg, src_desc, dst_desc, strides_dims, kernels_dims, padding_l, padding_r); auto backward_prim_desc = dnnl::pooling_backward::primitive_desc(backward_desc, MKLKernelEngine::Get().engine(), prim_desc); primitive_ = std::make_shared<dnnl::pooling_backward>(backward_prim_desc); AddArgument(DNNL_ARG_SRC, src_desc); AddArgument(DNNL_ARG_DST, dst_desc); AddArgument(DNNL_ARG_DIFF_SRC, src_desc); AddArgument(DNNL_ARG_DIFF_DST, dst_desc);}//判断输入输出大小是否符合要求,不符合则记录错误信息,符合则调用函数进行下一步操作bool AvgPoolingGradCPUKernel::Launch(const std::vector<kernel::AddressPtr> &inputs, const std::vector<kernel::AddressPtr> & /*workspace*/, const std::vector<kernel::AddressPtr> &outputs) { if (inputs.size() < 3 || outputs.empty()) { MS_LOG(EXCEPTION) << "Pooling avg grad error input output size!"; } //set handle类型函数 SetArgumentHandle(DNNL_ARG_SRC, inputs[0]->addr); SetArgumentHandle(DNNL_ARG_DST, inputs[1]->addr); SetArgumentHandle(DNNL_ARG_DIFF_DST, inputs[2]->addr); SetArgumentHandle(DNNL_ARG_DIFF_SRC, outputs[0]->addr); ExecutePrimitive(); return true;//返回值}} // namespace kernel} // namespace mindspore
上滑加载中
推荐直播
-
OpenHarmony应用开发之网络数据请求与数据解析
2025/01/16 周四 19:00-20:30
华为开发者布道师、南京师范大学泰州学院副教授,硕士研究生导师,开放原子教育银牌认证讲师
科技浪潮中,鸿蒙生态强势崛起,OpenHarmony开启智能终端无限可能。当下,其原生应用开发适配潜力巨大,终端设备已广泛融入生活各场景,从家居到办公、穿戴至车载。 现在,机会敲门!我们的直播聚焦OpenHarmony关键的网络数据请求与解析,抛开晦涩理论,用真实案例带你掌握数据访问接口,轻松应对复杂网络请求、精准解析Json与Xml数据。参与直播,为开发鸿蒙App夯实基础,抢占科技新高地,别错过!
回顾中 -
Ascend C高层API设计原理与实现系列
2025/01/17 周五 15:30-17:00
Ascend C 技术专家
以LayerNorm算子开发为例,讲解开箱即用的Ascend C高层API
回顾中
热门标签