-
RK3588 AI 应用开发 (FCN-语义分割)一、模型训练与转换FCN(全卷积网络,Fully Convolutional Networks)是用于语义分割任务的一种深度学习模型架构,引入了跳跃结构(Skip Architecture),通过融合浅层和深层的特征图,保留更多的细节信息,提升分割精度。此外,FCN还利用多尺度上下文聚合,捕捉不同层级的特征,增强了对不同大小目标的识别能力。FCN的成功推动了语义分割领域的发展,成为后续许多先进模型的基础。模型的训练与转换教程已经开放在AI Gallery中,其中包含训练数据、训练代码、模型转换脚本。在ModelArts的Notebook环境中训练后,再转换成对应平台的模型格式:onnx格式可以用在Windows设备上,RK系列设备上需要转换为rknn格式。二、应用开发1. 开发 Gradio 界面import cv2 import json import base64 import requests import numpy as np import gradio as gr def test_image(image_path): try: image_bgr = cv2.imread(image_path) image_string = cv2.imencode('.jpg', image_bgr)[1].tobytes() image_base64 = base64.b64encode(image_string).decode('utf-8') params = {"image_base64": image_base64} response = requests.post(f'http://{ip}:{port}{url}', data=json.dumps(params), headers={"Content-Type": "application/json"}) if response.status_code == 200: image_base64 = response.json().get("image_base64") image_binary = base64.b64decode(image_base64) image_array = np.frombuffer(image_binary, dtype=np.uint8) image_rgb = cv2.imdecode(image_array, cv2.IMREAD_COLOR) else: image_rgb = None except Exception as e: return None else: return image_rgb if __name__ == "__main__": port = 8000 ip = "127.0.0.1" url = "/v1/FCN" demo = gr.Interface(fn=test_image, inputs=gr.Image(type="filepath"), outputs=["image"], title="FCN 果蔬病虫害分割") demo.launch(share=False, server_port=3000) /home/orangepi/miniconda3/envs/python-3.10.10/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm * Running on local URL: http://127.0.0.1:3000 * To create a public link, set `share=True` in `launch()`. 2. 编写推理代码class FCN: def __init__(self, model_path): self.num_classes = 117 self.rknn_lite = RKNNLite() self.rknn_lite.load_rknn(model_path) self.rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0_1_2) self.color_list = np.random.randint(0, 255, size=(self.num_classes, 3), dtype=np.uint8).tolist() def preprocess(self, image): image = image[:, :, ::-1] image = cv2.resize(image, (224, 224)) return np.expand_dims(image, axis=0) def rknn_infer(self, data): outputs = self.rknn_lite.inference(inputs=[data]) return outputs[0] def post_process(self, pred): feat = pred.squeeze() return np.argmax(feat, axis=-1).astype(np.uint8) def predict(self, image): # 图像预处理 data = self.preprocess(image) # 模型推理 pred = self.rknn_infer(data) # 模型后处理 feat = self.post_process(pred) # 生成图像分割结果 canv = np.zeros_like(image) mask = cv2.resize(feat, image.shape[:2][::-1], interpolation=cv2.INTER_NEAREST) for i in range(1, self.num_classes): canv[mask==i] = self.color_list[i] return cv2.addWeighted(image[..., ::-1], 0.5, canv, 0.5, 0) def release(self): self.rknn_lite.release() 3. 图片批量预测import os import cv2 import numpy as np import matplotlib.pyplot as plt from rknnlite.api import RKNNLite model = FCN('model/FCN.rknn') for image in os.listdir("image"): image = cv2.imread(os.path.join("image", image)) image = model.predict(image) plt.imshow(image) plt.axis('off') plt.show() model.release() 4. 创建 Flask 服务import cv2 import base64 import numpy as np from rknnlite.api import RKNNLite from flask import Flask, request, jsonify from flask_cors import CORS app = Flask(__name__) CORS(app) @app.route('/v1/FCN', methods=['POST']) def inference(): data = request.get_json() image_base64 = data.get("image_base64") image_binary = base64.b64decode(image_base64) image_array = np.frombuffer(image_binary, dtype=np.uint8) image_bgr = cv2.imdecode(image_array, cv2.IMREAD_COLOR) image_rgb = model.predict(image_bgr) image_string = cv2.imencode('.jpg', image_rgb)[1].tobytes() image_base64 = base64.b64encode(image_string).decode('utf-8') return jsonify({ "image_base64": image_base64 }), 200 if __name__ == '__main__': model = FCN('model/FCN.rknn') app.run(host='0.0.0.0', port=8000) model.release() W rknn-toolkit-lite2 version: 2.3.2 I RKNN: [00:06:51.738] RKNN Runtime Information: librknnrt version: 1.4.0 (a10f100eb@2022-09-09T09:07:14) I RKNN: [00:06:51.738] RKNN Driver Information: version: 0.9.6 I RKNN: [00:06:51.739] RKNN Model Information: version: 1, toolkit version: 1.4.0-22dcfef4(compiler version: 1.4.0 (3b4520e4f@2022-09-05T12:50:09)), target: RKNPU v2, target platform: rk3588, framework name: TFLite, framework layout: NHWC * Serving Flask app '__main__' * Debug mode: off WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Running on all addresses (0.0.0.0) * Running on http://127.0.0.1:8000 * Running on http://192.168.3.50:8000 Press CTRL+C to quit 127.0.0.1 - - [02/May/2025 00:07:17] "POST /v1/FCN HTTP/1.1" 200 - 127.0.0.1 - - [02/May/2025 00:07:24] "POST /v1/FCN HTTP/1.1" 200 - 127.0.0.1 - - [02/May/2025 00:07:31] "POST /v1/FCN HTTP/1.1" 200 - 127.0.0.1 - - [02/May/2025 00:07:39] "POST /v1/FCN HTTP/1.1" 200 - 5. 上传图片预测三、小结本章介绍了基于RK3588平台使用FCN模型进行语义分割的AI应用开发全流程,包括模型训练与转换、Gradio界面开发、推理代码编写、批量预测实现及Flask服务部署。通过该流程,开发者可实现高效的图像分割任务,并在本地或云端进行预测和展示。----转自博客:https://bbs.huaweicloud.com/blogs/451998
-
RK3588 AI 应用开发 (YOLOX-目标检测)一、模型训练和转换YOLOX是YOLO系列的优化版本,引入了解耦头、数据增强、无锚点以及标签分类等目标检测领域的优秀进展,拥有较好的精度表现,同时对工程部署友好。模型的训练与转换教程已经开放在AI Gallery中,其中包含训练数据、训练代码、模型转换脚本。在ModelArts的Notebook环境中训练后,再转换成对应平台的模型格式:onnx格式可以用在Windows设备上,RK系列设备上需要转换为rknn格式。二、应用开发1. 开发 Gradio 界面import cv2 import json import base64 import requests import numpy as np import gradio as gr def test_image(image_path): try: image_bgr = cv2.imread(image_path) image_string = cv2.imencode('.jpg', image_bgr)[1].tobytes() image_base64 = base64.b64encode(image_string).decode('utf-8') params = {"image_base64": image_base64} response = requests.post(f'http://{ip}:{port}{url}', data=json.dumps(params), headers={"Content-Type": "application/json"}) if response.status_code == 200: image_base64 = response.json().get("image_base64") image_binary = base64.b64decode(image_base64) image_array = np.frombuffer(image_binary, dtype=np.uint8) image_rgb = cv2.imdecode(image_array, cv2.IMREAD_COLOR) else: image_rgb = None except Exception as e: return None else: return image_rgb if __name__ == "__main__": port = 8000 ip = "127.0.0.1" url = "/v1/fish_det" demo = gr.Interface(fn=test_image, inputs=gr.Image(type="filepath"), outputs=["image"], title="YOLOX 深海鱼类检测") demo.launch(share=False, server_port=3000) * Running on local URL: http://127.0.0.1:3000 * To create a public link, set `share=True` in `launch()`. 2. 编写推理代码%%writefile YOLOX/yolox/data/datasets/voc_classes.py #!/usr/bin/env python3 # -*- coding:utf-8 -*- # Copyright (c) Megvii, Inc. and its affiliates. # VOC_CLASSES = ( '__background__', # always index 0 VOC_CLASSES = ( "fish", ) Overwriting YOLOX/yolox/data/datasets/voc_classes.pyimport sys sys.path.append("YOLOX") from yolox.utils import demo_postprocess, multiclass_nms, vis from yolox.data.data_augment import preproc as preprocess from yolox.data.datasets.voc_classes import VOC_CLASSESimport cv2 import numpy as np import ipywidgets as widgets from rknnlite.api import RKNNLite from IPython.display import display class YOLOX: def __init__(self, model_path): self.ratio = None self.rknn_lite = RKNNLite() self.rknn_lite.load_rknn(model_path) self.rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0_1_2) def preprocess(self, image): start_img, self.ratio = preprocess(image, (320, 320), swap=(0, 1, 2)) return np.expand_dims(start_img, axis=0) def rknn_infer(self, data): outputs = self.rknn_lite.inference(inputs=[data]) return outputs[0] def post_process(self, pred): predictions = demo_postprocess(pred.squeeze(), (320, 320)) boxes = predictions[:, :4] scores = predictions[:, 4:5] * predictions[:, 5:] boxes_xyxy = np.ones_like(boxes) boxes_xyxy[:, 0] = boxes[:, 0] - boxes[:, 2] / 2. boxes_xyxy[:, 1] = boxes[:, 1] - boxes[:, 3] / 2. boxes_xyxy[:, 2] = boxes[:, 0] + boxes[:, 2] / 2. boxes_xyxy[:, 3] = boxes[:, 1] + boxes[:, 3] / 2. boxes_xyxy /= self.ratio dets = multiclass_nms(boxes_xyxy, scores, nms_thr=0.45, score_thr=0.25) return dets def predict(self, image): # 图像预处理 data = self.preprocess(image) # 模型推理 pred = self.rknn_infer(data) # 模型后处理 dets = self.post_process(pred) # 绘制目标检测结果 if dets is not None: final_boxes = dets[:, :4] final_scores, final_cls_inds = dets[:, 4], dets[:, 5] image = vis(image, final_boxes, final_scores, final_cls_inds, conf=0.25, class_names=VOC_CLASSES) return image[..., ::-1] def img2bytes(self, image): """将图片转换为字节码""" return bytes(cv2.imencode('.jpg', image)[1]) def infer_video(self, video_path): """视频推理""" image_widget = widgets.Image(format='jpeg', width=800, height=600) display(image_widget) cap = cv2.VideoCapture(video_path) while True: ret, img_frame = cap.read() if not ret: break image_pred = self.predict(img_frame) image_widget.value = self.img2bytes(image_pred) cap.release() def release(self): """释放资源""" self.rknn_lite.release() 3. 图像预测4. 视频推理5. 创建 Flask 服务import cv2 import base64 import numpy as np from rknnlite.api import RKNNLite from flask import Flask, request, jsonify from flask_cors import CORS app = Flask(__name__) CORS(app) @app.route('/v1/fish_det', methods=['POST']) def inference(): data = request.get_json() image_base64 = data.get("image_base64") image_binary = base64.b64decode(image_base64) image_array = np.frombuffer(image_binary, dtype=np.uint8) image_bgr = cv2.imdecode(image_array, cv2.IMREAD_COLOR) image_rgb = model.predict(image_bgr) image_string = cv2.imencode('.jpg', image_rgb)[1].tobytes() image_base64 = base64.b64encode(image_string).decode('utf-8') return jsonify({ "image_base64": image_base64 }), 200 if __name__ == '__main__': model = YOLOX('model/yolox_fish.rknn') app.run(host='0.0.0.0', port=8000) model.release() 6. 上传图片预测三、小结本章介绍了基于RK3588平台使用YOLOX进行目标检测的全流程,包括模型训练与转换、Gradio界面开发、推理代码编写、图像和视频预测实现,以及Flask服务部署。整体实现了高效的鱼类检测应用,适用于嵌入式设备部署与实际场景应用。----转自博客:https://bbs.huaweicloud.com/blogs/452001
-
RK3588 AI 应用开发 (InceptionV3-图像分类)一、模型训练与转换Inception V3,GoogLeNet的改进版本,采用InceptionModule和全局平均池化层,v3一个最重要的改进是分解(Factorization),将7x7分解成两个一维的卷积(1x7,7x1),3x3也是一样(1x3,3x1),这样的好处,既可以加速计算(多余的计算能力可以用来加深网络),又可以将1个conv拆成2个conv,使得网络深度进一步增加,增加了网络的非线性。模型的训练与转换教程已经开放在AI Gallery中,其中包含训练数据、训练代码、模型转换脚本。在ModelArts的Notebook环境中训练后,再转换成对应平台的模型格式:onnx格式可以用在Windows设备上,RK系列设备上需要转换为rknn格式。二、应用开发1. 开发 Gradio 界面import cv2 import json import base64 import requests import numpy as np import gradio as gr def test_image(image_path): try: image_bgr = cv2.imread(image_path) image_string = cv2.imencode('.jpg', image_bgr)[1].tobytes() image_base64 = base64.b64encode(image_string).decode('utf-8') params = {"image_base64": image_base64} response = requests.post(f'http://{ip}:{port}{url}', data=json.dumps(params), headers={"Content-Type": "application/json"}) if response.status_code == 200: image_base64 = response.json().get("image_base64") image_binary = base64.b64decode(image_base64) image_array = np.frombuffer(image_binary, dtype=np.uint8) image_rgb = cv2.imdecode(image_array, cv2.IMREAD_COLOR) else: image_rgb = None except Exception as e: return None else: return image_rgb if __name__ == "__main__": port = 8000 ip = "127.0.0.1" url = "/v1/InceptionV3" demo = gr.Interface(fn=test_image, inputs=gr.Image(type="filepath"), outputs=["image"], title="InceptionV3 动物分类") demo.launch(share=False, server_port=3000) /home/orangepi/miniconda3/envs/python-3.10.10/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm * Running on local URL: http://127.0.0.1:3000 * To create a public link, set `share=True` in `launch()`. 2. 编写推理代码class InceptionV3: def __init__(self, model_path): self.rknn_lite = RKNNLite() self.rknn_lite.load_rknn(model_path) self.rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0_1_2) self.label = ['antelope', 'badger', 'bat', 'bear', 'bee', 'beetle', 'bison', 'boar', 'butterfly', 'cat', 'caterpillar', 'chimpanzee', 'cockroach', 'cow', 'coyote', 'crab', 'crow', 'deer', 'dog', 'dolphin', 'donkey', 'dragonfly', 'duck', 'eagle', 'elephant', 'flamingo', 'fly', 'fox', 'goat', 'goldfish', 'goose', 'gorilla', 'grasshopper', 'hamster', 'hare', 'hedgehog', 'hippopotamus', 'hornbill', 'horse', 'hummingbird', 'hyena', 'jellyfish', 'kangaroo', 'koala', 'ladybugs', 'leopard', 'lion', 'lizard', 'lobster', 'mosquito', 'moth', 'mouse', 'octopus', 'okapi', 'orangutan', 'otter', 'owl', 'ox', 'oyster', 'panda', 'parrot', 'pelecaniformes', 'penguin', 'pig', 'pigeon', 'porcupine', 'possum', 'raccoon', 'rat', 'reindeer', 'rhinoceros', 'sandpiper', 'seahorse', 'seal', 'shark', 'sheep', 'snake', 'sparrow', 'squid', 'squirrel', 'starfish', 'swan', 'tiger', 'turkey', 'turtle', 'whale', 'wolf', 'wombat', 'woodpecker', 'zebra'] def preprocess(self, image): image = image[:, :, ::-1] image = cv2.resize(image, (224, 224)) return np.expand_dims(image, axis=0) def rknn_infer(self, data): outputs = self.rknn_lite.inference(inputs=[data]) return outputs[0] def post_process(self, pred): clsse = np.argmax(pred, axis=-1) score = pred[0][clsse[0]].item() return self.label[clsse[0]], round(score * 100, 2) def predict(self, image): # 图像预处理 data = self.preprocess(image) # 模型推理 pred = self.rknn_infer(data) # 模型后处理 label, score = self.post_process(pred) # 绘制识别结果 print(f'{label}:{score}%') image = cv2.putText(image, f'{label}:{score}%', (0, 100), cv2.FONT_HERSHEY_TRIPLEX, 4, (0, 255, 0), 8) return image[..., ::-1] def release(self): self.rknn_lite.release() 3. 图片批量预测import os import cv2 import numpy as np import matplotlib.pyplot as plt from rknnlite.api import RKNNLite model = InceptionV3('model/InceptionV3.rknn') for image in os.listdir("image"): image = cv2.imread(os.path.join("image", image)) image = model.predict(image) plt.imshow(image) plt.axis('off') plt.show() model.release() 4. 创建 Flask 服务import cv2 import base64 import numpy as np from rknnlite.api import RKNNLite from flask import Flask, request, jsonify from flask_cors import CORS app = Flask(__name__) CORS(app) @app.route('/v1/InceptionV3', methods=['POST']) def inference(): data = request.get_json() image_base64 = data.get("image_base64") image_binary = base64.b64decode(image_base64) image_array = np.frombuffer(image_binary, dtype=np.uint8) image_bgr = cv2.imdecode(image_array, cv2.IMREAD_COLOR) image_rgb = model.predict(image_bgr) image_string = cv2.imencode('.jpg', image_rgb)[1].tobytes() image_base64 = base64.b64encode(image_string).decode('utf-8') return jsonify({ "image_base64": image_base64 }), 200 if __name__ == '__main__': model = InceptionV3('model/InceptionV3.rknn') app.run(host='0.0.0.0', port=8000) model.release() W rknn-toolkit-lite2 version: 2.3.2 * Serving Flask app '__main__' * Debug mode: off WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Running on all addresses (0.0.0.0) * Running on http://127.0.0.1:8000 * Running on http://192.168.3.50:8000 Press CTRL+C to quit 127.0.0.1 - - [01/May/2025 20:37:00] "POST /v1/InceptionV3 HTTP/1.1" 200 - pig:99.95% 127.0.0.1 - - [01/May/2025 20:37:09] "POST /v1/InceptionV3 HTTP/1.1" 200 - swan:99.95% 127.0.0.1 - - [01/May/2025 20:37:20] "POST /v1/InceptionV3 HTTP/1.1" 200 - cat:97.02% 5. 上传图片预测三、小结本章介绍了基于RK3588平台的InceptionV3图像分类应用开发全流程,包括模型训练与格式转换、Gradio界面设计、推理代码实现、批量预测处理及Flask服务部署,实现了从本地到Web端的高效AI推理应用。----转自博客:https://bbs.huaweicloud.com/blogs/451978
-
一、Modelarts Lite Server 使用前必读1. Lite Server 使用流程总览ModelArts Lite Server提供多样化的xPU裸金属服务器,赋予用户以root账号自主安装 和部署AI框架、应用程序等第三方软件的能力,为用户打造专属的云上物理服务器环 境。用户只需轻松选择服务器的规格、镜像、网络配置及密钥等基本信息,即可迅速 创建弹性裸金属服务器,获取所需的云上物理资源,充分满足算法工程师在日常训练 和推理工作中的需求。流程图 2. Lite Server 高危操作一览表ModelArts Lite Server在日常操作与维护过程中涉及的高危操作,需要严格按照操作 指导进行,否则可能会影响业务的正常运行。 高危操作风险等级说明:高:对于可能直接导致业务失败、数据丢失、系统不能维护、系统资源耗尽的高 危操作。中:对于可能导致安全风险及可靠性降低的高危操作。低:高、中风险等级外的其他高危操作。操作对象操作名称风险描述风险等级应对措施操作系统升级/修改操 作系统内核 或者驱动。如果升级/修改操作 系统内核或者驱 动,很可能导致驱 动和内核版本不兼 容,从而导致OS无 法启动,或者基本 功能不可用。相关 高危命令如:ap高如果需要升级/ 修改,请联系 技术支持。操作系统切换或者重 置操作系统。服务器在进行过 “切换或者重置操 作系统”操作后, EVS系统盘ID发生变 化,和下单时订单 中的EVS ID已经不 一致, 因此EVS系 统盘将不支持扩 容,并显示信息: “当前订单已到 期,无法进行扩容 操作,请续订”。中切换或者重置 操作系统后, 建议通过挂载 数据盘EVS或 挂载SFS盘等 方式进行存储 扩容。 二、Modelarts Lite Server 资源开通图 2-1 Server 资源开通流程图 表 2-1 Server 资源开通流程阶段任务准备工作1、申请开通资源规格2、资源配合提升3、基础权限开通4、配置ModelArts委托授权购买Server5、在ModelArts控制台购买资源池步骤 1:申请开通资源规格 请联系客户经理/接口人确认Server资源方案、申请要开通资源的规格(如果无客户经理可提 交工单)。步骤 2:资源配额提升 由于Server所需资源可能会超出默认提供的资源(如ECS、EIP、SFS、内存大小、CPU 核数),因此需要提升资源配额。1 登录云管理控制台。2 在顶部导航栏单击“资源 > 我的配额”,进入服务配额页面。3 单击右上角“申请扩大配额”,填写申请材料后提交工单。 说明:配额需大于需要开通的资源,且在购买开通前完成提升,否则会导致资源开通失败。4 基础权限开通(若主账号/已有iam子账号可忽略)基础权限开通需要登录管理员账号,为子用户账号开通Server功能所需的基础权限 (ModelArts FullAccess/BMS FullAccess/ECS FullAccess/VPC FullAccess/VPC Administrator/VPCEndpoint Administrator),即允许子用户账号同时可以使用这些 云服务。步骤1 登录统一身份认证服务管理控制台。步骤2 单击目录左侧“用户组”,然后在页面右上角单击“创建用户组”。步骤3 填写“用户组名称”并单击“确定”。步骤4 在操作列单击“用户组管理”,将需要配置权限的用户加入用户组中。步骤5 单击用户组名称,进入用户组详情页。步骤6 在权限管理页签下,单击“授权”。图 2-2 “配置权限”步骤7 在搜索栏输入“ModelArts FullAccess”,并勾选“ModelArts FullAccess”。图 2-3 ModelArts FullAccess以相同的方式,依次添加:BMS FullAccess、ECS FullAccess、VPC FullAccess、VPC Administrator、VPCEndpoint Administrator。(Server Administrator、DNS Administrator为依赖策略,会自动被勾选)。步骤8 单击“下一步”,授权范围方案选择“所有资源”。步骤9 单击“确认”,完成基础权限开通。----结束5. 在 ModelArts 上创建委托授权ModelArts Lite Server在任务执行过程中需要访问用户的其他服务,典型的就是容器 使用过程中需要到SWR服务拉取镜像。在这个过程中,就出现了ModelArts“代表” 用户去访问其他云服务的情形。从安全角度出发,ModelArts代表用户访问任何云服务 之前,均需要先获得用户的授权,而这个动作就是一个“委托”的过程。用户授权 ModelArts代表自己访问特定的云服务,以完成其在ModelArts平台上执行的AI计算任 务。● 新建委托第一次使用ModelArts时需要创建委托授权,授权允许ModelArts代表用户去访问 其他云服务。进入到ModelArts控制台的“权限管理”页面,单击“添加授权”, 根据提示进行操作。 ● 更新委托如果之前给ModelArts创过委托授权,此处可以更新授权。a. 进入到ModelArts控制台的“资源管理>AI专属资源池>弹性节点 Server”页 面,查看是否存在授权缺失的提示。 图 2-4 弹性节点 Server 权限缺失提示 b. 如果有授权缺失,根据提示,单击“此处”更新委托。根据提示选择“追加 至已有授权”,单击“确定”,系统会提示权限更新成功。 图 2-5 追加授权 6. 购买弹性节点 Server 资源 购买弹性节点Server资源的过程即创建资源过程。登录ModelArts管理控制台。在左侧导航栏中,选择“资源管理 > AI专属资源池 > 弹性节点 Server”,进入“节 点”列表。图 2-6 购买弹性节点 Server 入口 单击节点列表页右上角的“购买AI专属节点”,进入“购买AI专属节点”页面,在该 页面填写相关参数信息。(根据申请的区域(贵阳 乌兰察布 华东)) 三、ModelArts Lite Server 软件环境NPU 服务器上配置 Lite Server 资源软件环境1.1 安装驱动(安装完毕后需重启服务器(可输入指令 reboot 重启服务器))wget "https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Ascend HDK/Ascend HDK 23.0.3/Ascend-hdk-910b-npu-driver_23.0.3_linux-aarch64.run" sudo sh Ascend-hdk-910b-npu-driver_23.0.3_linux-aarch64.run --full --install-for-all1.2 验证驱动(完整输出NPU各卡状态内容则为正常)npu-smi info1.3 安装固件wget "https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Ascend HDK/Ascend HDK 23.0.3/Ascend-hdk-910b-npu-firmware_7.1.0.5.220.run" sudo sh Ascend-hdk-910b-npu-firmware_7.1.0.5.220.run --full1.4 查看安装结果,如图3.2所示npu-smi info -t board -i 1 | egrep -i "software|firmware" 图 3-2 查看固件和驱动版本 1.5 安装CANN# install CANN Toolkit wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-toolkit_8.0.RC1.alpha001_linux-"$(uname -i)".run bash Ascend-cann-toolkit_8.0.RC1.alpha001_linux-"$(uname -i)".run –install # install CANN Kernels wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run bash Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run –install #注意安装顺序,先安装Toolkit 在安装Kernels #配置昇腾开发工具包运行所需的环境变量 source /usr/local/Ascend/ascend-toolkit/set_env.sh1.6 安装dockeryum install -y docker-engine.aarch64 docker-engine-selinux.noarch docker-runc.aarch64 #配置Ascend-docker-runtime。 #下载Ascend-docker-runtime wget https://mindx.obs.cn-south-1.myhuaweicloud.com/OpenSource/MindX/MindX%205.0.RC2/MindX%20DL%205.0.RC2/Ascend-docker-runtime_5.0.RC2_linux-aarch64.run chmod 700 *.run ./ Ascend-docker-runtime_5.0.RC2_linux-aarch64.run --install1.7 检查安装Ascend-docker-runtime、docker info |grep Runtime图 3-3 Ascend-docker-runtime 查询 1.8 配置docker #将新挂载的盘设置为docker容器使用路径 { "data-root": "/home/docker", "default-runtime": "ascend", "default-shm-size": "8G", "insecure-registries": [ "ascendhub.huawei.com" ], "registry-mirrors": [ "https://90c2db5e37a147c6afd970e45c68e365.mirror.swr.myhuaweicloud.com" ], "runtimes": { "ascend": { "path": "/usr/local/Ascend/Ascend-Docker-Runtime/ascend-docker-runtime", "runtimeArgs": [] } } } #保存后,执行如下命令重启docker使配置生效。 systemctl daemon-reload && systemctl restart docker 四、模型部署4.1下载模型镜像前往昇腾社区/开发资源(https://gitee.com/link?target=https%3A%2F%2Fwww.hiascend.com%2Fdeveloper%2Fascendhub%2Fdetail%2Faf85b724a7e5469ebd7ea13c3439d48f)下载适配,下载镜像前需要申请权限,耐心等待权限申请通过后,根据指南下载对应镜像文件。完成之后,请使用docker images命令确认查找具体镜像名称与标签。4.2 权重文件下载:Qwen2.5-72B-Instruct(下载地址:https://www.modelscope.cn/models/Qwen/Qwen2.5-72B-Instruct/files)4.3 启动模型docker run -itd --privileged --name=Qwen2.5_72b --net=host --shm-size=500g --device=/dev/davinci0 --device=/dev/davinci1 --device=/dev/davinci2 --device=/dev/davinci3 --device=/dev/davinci4 --device=/dev/davinci5 --device=/dev/davinci6 --device=/dev/davinci7 --device=/dev/davinci_manager --device=/dev/devmm_svm --device=/dev/hisi_hdc -v /usr/local/Ascend/driver:/usr/local/Ascend/driver -v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons/ -v /usr/local/sbin/:/usr/local/sbin/ -v /var/log/npu/slog/:/var/log/npu/slog -v /var/log/npu/profiling/:/var/log/npu/profiling -v /var/log/npu/dump/:/var/log/npu/dump -v /var/log/npu/:/usr/slog -v /etc/hccn.conf:/etc/hccn.conf -v /Qwen2.5/Qwen2.5-72B-Instruct:/Qwen2.5/Qwen2.5-72B-Instruct 2acbb0b1003d /bin/bash #这边下载的权重文件目录在/Qwen2.5/Qwen2.5-72B-Instruct 需要启动容器挂载上4.4 纯模型推理Docker容器跑起来后需要进入容器docker exec it 容器id /bin/bash对话测试#前置准备 # 通信优化 export HCCL_DETERMINISTIC=false # 关闭昇腾集群通信的确定性模式(提升性能,允许并行计算优化) export LCCL_DETERMINISTIC=0 # 关闭集合通信的严格同步(减少通信延迟,提升分布式效率) export HCCL_BUFFSIZE=120 # 设置通信缓冲区大小(默认32,增大可提升吞吐量,但占用更多内存) # 内存优化 export ATB_WORKSPACE_MEM_ALLOC_GLOBAL=1 # 全局预分配昇腾算子的内存空间(避免反复申请,加速计算) #测试开始 #进入modeltest路径 cd /usr/local/Ascend/atb-models #Step1.清理残余进程 pkill -9 -f 'mindie|python' #Step2.执行命令: bash examples/models/qwen/run_pa.sh -m ${weight_path} --trust_remote_code true ${weight_path}需要改成自己的模型权重文件路径 这里是Qwen2.5/Qwen2.5-72B-Instruct4.5 服务化推理前置准备【使用场景】对标真实客户上线场景,使用不同并发、不同发送频率、不同输入长度和输出长度分布,去测试服务化性能#配置服务化环境变量 export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True #修改服务化参数 cd /usr/local/Ascend/mindie/latest/mindie-service/ vim conf/config.json #修改以下参数: "httpsEnabled" : false, # 如果网络环境不安全,不开启HTTPS通信,即“httpsEnabled”=“false”时,会存在较高的网络安全风险 ... # 若不需要安全认证,则将以下两个参数设为false "interCommTLSEnabled" : false, "interNodeTLSEnabled" : false, ... "npudeviceIds" : [[0,1,2,3,4,5,6,7]], ... "modelName" : "qwen" # 不影响服务化拉起 "modelWeightPath" : "权重路径", "worldSize":8,參考修改{ "Version": "1.0.0", "LogConfig": { "logLevel": "Info", // 日志级别:信息 "logFileSize": 20, // 单个日志文件大小:20MB "logFileNum": 20, // 保留的日志文件数量:20个 "logPath": "logs/mindie-server.log" // 日志文件路径:logs/mindie-server.log }, "ServerConfig": { "ipAddress": "127.0.0.1", // IP地址:本地回环地址 "managementIpAddress": "127.0.0.2", // 管理IP地址:本地回环地址 "port": 1025, // 端口号:1025 "managementPort": 1026, // 管理端口:1026 "metricsPort": 1027, // 指标端口:1027 "allowAllZeroIpListening": false, // 是否允许监听所有零IP:否 "maxLinkNum": 1000, // 最大连接数:1000 "httpsEnabled": true, // HTTPS是否启用:是 "fullTextEnabled": false, // 全文搜索是否启用:否 "tlsCaPath": "/usr/local/Ascend/mindie/latest/mindie-service/security/ca/", // TLS CA证书路径 "tlsCaFile": ["ca.pem"], // TLS CA证书文件列表 "tlsCert": "/usr/local/Ascend/mindie/latest/mindie-service/security/certs/server.pem", // TLS证书文件路径 "tlsPk": "/usr/local/Ascend/mindie/latest/mindie-service/security/keys/server.key.pem", // TLS私钥文件路径 "tlsPkPwd": "/usr/local/Ascend/mindie/latest/mindie-service/security/pass/key_pwd.txt", // TLS私钥密码文件路径 "tlsCrlPath": "/usr/local/Ascend/mindie/latest/mindie-service/security/certs/", // TLS CRL路径 "tlsCrlFiles": ["server_crl.pem"], // TLS CRL文件列表 "managementTlsCaFile": ["management_ca.pem"], // 管理TLS CA证书文件列表 "managementTlsCert": "/security/certs/management/server.pem", // 管理TLS证书文件路径 "managementTlsPk": "/security/keys/management/server.key.pem", // 管理TLS私钥文件路径 "managementTlsPkPwd": "/security/pass/management/key_pwd.txt", // 管理TLS私钥密码文件路径 "managementTlsCrlPath": "/security/management/certs/", // 管理TLS CRL路径 "managementTlsCrlFiles": ["server_crl.pem"], // 管理TLS CRL文件列表 "kmcKsfMaster": "tools/pmt/master/ksfa", // KMC密钥主文件路径 "kmcKsfStandby": "tools/pmt/standby/ksfb", // KMC密钥备用文件路径 "inferMode": "standard", // 推理模式:标准 "interCommTLSEnabled": false, // 内部通信TLS是否启用:否 "interCommPort": 1121, // 内部通信端口:1121 "interCommTlsCaPath": "security/grpc/ca/", // 内部通信TLS CA证书路径 "interCommTlsCaFiles": ["ca.pem"], // 内部通信TLS CA证书文件列表 "interCommTlsCert": "security/grpc/certs/server.pem", // 内部通信TLS证书文件路径 "interCommPk": "security/grpc/keys/server.key.pem", // 内部通信TLS私钥文件路径 "interCommPkPwd": "security/grpc/pass/key_pwd.txt", // 内部通信TLS私钥密码文件路径 "interCommTlsCrlPath": "security/grpc/certs/", // 内部通信TLS CRL路径 "interCommTlsCrlFiles": ["server_crl.pem"], // 内部通信TLS CRL文件列表 "openAiSupport": "vllm" // OpenAI支持:vllm }, "BackendConfig": { "backendName": "mindieservice_llm_engine", // 后端名称:mindieservice_llm_engine "modelInstanceNumber": 1, // 模型实例数量:1 "npuDeviceIds": [[0, 1, 2, 3, 4, 5, 6, 7]], // NPU设备ID列表 "tokenizerProcessNumber": 8, // 分词器进程数量:8 "multiNodesInferEnabled": false, // 多节点推理是否启用:否 "multiNodesInferPort": 1120, // 多节点推理端口:1120 "interNodeTLSEnabled": false, // 节点间通信TLS是否启用:否 "interNodeTlsCaPath": "security/grpc/ca/", // 节点间通信TLS CA证书路径 "interNodeTlsCaFiles": ["ca.pem"], // 节点间通信TLS CA证书文件列表 "interNodeTlsCert": "security/grpc/certs/server.pem", // 节点间通信TLS证书文件路径 "interNodeTlsPk": "security/grpc/keys/server.key.pem", // 节点间通信TLS私钥文件路径 "interNodeTlsPkPwd": "security/grpc/pass/mindie_server_key_pwd.txt", // 节点间通信TLS私钥密码文件路径 "interNodeTlsCrlPath": "security/grpc/certs/", // 节点间通信TLS CRL路径 "interNodeTlsCrlFiles": ["server_crl.pem"], // 节点间通信TLS CRL文件列表 "interNodeKmcKsfMaster": "tools/pmt/master/ksfa", // 节点间通信KMC密钥主文件路径 "interNodeKmcKsfStandby": "tools/pmt/standby/ksfb", // 节点间通信KMC密钥备用文件路径 "ModelDeployConfig": { "maxSeqLen": 2560, // 最大序列长度:2560 "maxInputTokenLen": 2048, // 最大输入令牌长度:2048 "truncation": false, // 是否截断:否 "ModelConfig": [ { "modelInstanceType": "Standard", // 模型实例类型:标准 "modelName": "Qwen2.5-72b", // 模型名称:Qwen2.5-72b "modelWeightPath": "/Qwen2.5/Qwen2.5-72B-Instruct", // 模型权重文件路径 "worldSize": 8, // 世界大小(分布式训练):8 "cpuMemSize": 5, // CPU内存大小:5GB "npuMemSize": -1, // NPU内存大小:自动分配 "backendType": "atb", // 后端类型:atb "trustRemoteCode": false // 是否信任远程代码:否 } ] }, "ScheduleConfig": { "templateType": "Standard", // 模板类型:标准 "templateName": "Standard_LLM", // 模板名称:Standard_LLM "cacheBlockSize" : 128, // 缓存块大小:128 "maxPrefillBatchSize" : 50, // 最大预填充批处理大小:50 "maxPrefillTokens" : 8192, // 最大预填充令牌数:8192 "prefillTimeMsPerReq" : 150, // 每请求预填充时间:150毫秒 "prefillPolicyType" : 0, // 预填充策略类型:0 "decodeTimeMsPerReq" : 50, // 每请求解码时间:50毫秒 "decodePolicyType" : 0, // 解码策略类型:0 "maxBatchSize" : 200, // 最大批处理大小:200 "maxIterTimes" : 512, // 最大迭代次数:512 "maxPreemptCount" : 0, // 最大抢占计数:0 "supportSelectBatch" : false, // 是否支持选择批处理:否 "maxQueueDelayMicroseconds" : 5000 // 最大队列延迟:5000微秒 } } }拉起服务化# 以下命令需在所有机器上同时执行 # 解决权重加载过慢问题 export OMP_NUM_THREADS=1 #设置显存比 export NPU_MEMORY_FRACTION=0.95 # 拉起服务化 cd /usr/local/Ascend/mindie/latest/mindie-service/ ./bin/mindieservice_daemon执行命令后,首先会打印本次启动所用的所有参数,然后直到出现输出:Daemon start success! 则认为服务成功启动。
-
无人机巡检数据集:空中语义分割由计算机图形与视觉研究所(ICG开发的语义无人机数据集,旨在推动城市场景的语义理解研究,提升自主无人机飞行与着陆的安全性。以下从数量、类别、分布及分辨率四个核心维度展开说明:一、数据数量与划分该数据集包含600张高分辨率图像,按用途分为训练集和测试集两部分:训练集:400张图像,公开可获取,包含完整的标注信息,支持模型训练与算法验证。测试集:200张图像,为私有数据,主要用于评估模型的泛化能力,确保研究结果的客观性。此外,数据集还提供了丰富的辅助数据,包括1Hz采集的高分辨率图像序列、5Hz的鱼眼立体图像(同步配有IMU测量数据)、1Hz的热成像图像,以及3栋房屋的地面控制点和全站仪获取的3D真值数据,进一步扩展了其应用场景。二、语义类别与标注数据集针对语义分割任务定义了20个核心类别,覆盖城市场景中的典型元素,具体分类如下:自然元素:树(tree)、草(gras)、其他植被(other vegetation)、泥土(dirt)、 gravel、岩石(rocks)、水(water);人工构造:铺装区域(paved area)、泳池(pool)、屋顶(roof)、墙(wall)、栅栏(fence)、栅栏柱(fence-pole)、窗户(window)、门(door)、障碍物(obstacle);动态目标:人(person)、狗(dog)、汽车(car)、自行车(bicycle)。标注精度达到像素级,确保语义分割任务的准确性;同时,针对人物检测任务,提供了训练集和测试集的边界框(bounding box)标注,支持多任务研究。三、数据分布特点数据集的图像均通过无人机从天底视角(鸟瞰视角) 采集,覆盖超过20栋房屋的城市区域,拍摄高度在地面以上5至30米之间,确保场景的真实性与多样性。从分布来看:场景覆盖:包含居民区内的建筑、植被、道路、休闲区域(如泳池)等,兼顾自然与人工环境的混合场景;目标密度:图像中包含不同数量的动态目标(人、动物、车辆)和静态结构,适合测试算法在复杂目标交互场景下的表现;辅助数据分布:热成像、立体图像等辅助数据与主图像时空同步,可用于多模态融合研究,提升模型对环境的感知能力。四、分辨率与数据格式数据集的核心图像采用高分辨率相机采集,单张图像尺寸为6000×4000像素(2400万像素),确保细节信息的完整性,满足精细语义分割的需求。训练集提供多种格式的标注文件,包括:Python pickle格式的边界框数据;可选的XML格式边界框标注;可选的掩码图像(mask images),便于不同算法框架的直接使用。五、数据集下载地址# 数据集地址 https://developer.huaweicloud.com/develop/aigallery/dataset/detail?id=0efe9613-5248-4a43-925e-4d6d377b6996综上,该数据集凭借大尺寸、多类别、高精度的特点,为无人机视觉、语义分割、目标检测等领域的研究提供了高质量的基准数据,其丰富的辅助信息也为多模态感知与3D重建等任务奠定了基础。
-
我按照参考教程配置了如图所示的用户组权限,并且将新建的IAM子用户添加到了用户组,为什么子用户还是没有权限使用ModelArts呢?参考教程:https://support.huaweicloud.com/usermanual-maas-modelarts/maas-modelarts-0017.html
-
[问题求助] 为了确定环境的一致,在ModelArts上使用conda创建环境python3,10,在这个conda环境python3,10下安装不了CANN, Ascend-cann-toolkit_8.2.RC1安装Ascend-cann-toolkit出错,失败。/home/ma-user/var/log/ascend_seclog/ascend_toolkit_install.log文件的最后几行如下:
-
如果Modelarts下,下载文件 cann-toolkit工具包的网址如下: wget --header="Referer: https://www.hiascend.com/" https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.2.RC1/Ascend-cann-toolkit_8.2.RC1_linux-"$(uname -i)".run 因为默认的以.myhuaweicloud.com结尾的网址不走代理no_proxy=127.0.0.1,localhost,172.16.*,obs.cn-southwest-2.myhuaweicloud.com,iam.cn-southwest-2.huaweicloud.com,pip.modelarts.private.com,.myhuaweicloud.com(但是文件的网址是ascend-repo.obs.cn-east-2.myhuaweicloud.com,虽然是.myhuaweicloud.com,但不是obs.cn-southwest-2.myhuaweicloud.com区, 把最后的不走代理的 .myhuaweicloud.com 去掉,如下export no_proxy=127.0.0.1,localhost,172.16.*,obs.cn-southwest-2.myhuaweicloud.com,iam.cn-southwest-2.huaweicloud.com,pip.modelarts.private.com设置完成后,就可以正常下载了解决过程如下截图
-
本期直播聚焦X+AI驱动下的教育革新与产教融合实践,吸引超过7200人次的在线观看,累计社媒播放量突破6000+,本期企业开发者占比56%。
-
请问 同样命令在modelarts,下载不了Ascend-cann-toolkit,在其他主机上可以使用,不知道是什么原因?wget --header="Referer: https://www.hiascend.com/" https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.2.RC1/Ascend-cann-toolkit_8.2.RC1_linux-"$(uname -i)".run
-
开了一个 4 卡 910b1 的 Notebook,同样的代码,选用 0,1,2 卡都没问题,选到 3 卡就发生报错,大概是HCCL通信超时。即使重启 notebook 后,依然是同样情况主要的报错信息如下:```[rank0]:[W823 19:19:36.059936198 compiler_depend.ts:422] Warning: EI0006: [PID: 336613] 2025-08-23-19:19:35.912.992 Getting socket times out. Reason: Remote Rank did not send the data in time. Please check the reason for the rank being stuck Solution: 1. Check the rank service processes with other errors or no errors in the cluster.2. If this error is reported for all NPUs, check whether the time difference between the earliest and latest errors is greater than the connect timeout interval (120s by default). If so, adjust the timeout interval by using the HCCL_CONNECT_TIMEOUT environment variable.3. Check the connectivity of the communication link between nodes. (For details, see the TLS command and HCCN connectivity check examples.). For details:https://www.hiascend.com/document TraceBack (most recent call last): Transport init error. Reason: [Create][DestLink]Create Dest error! createLink para:rank[0]-localUserrank[0]-localIpAddr[172.16.0.151], dst_rank[1]-remoteUserrank[1]-remote_ip_addr[172.16.0.151] (function ExecFuncOpApi) [rank0]: RuntimeError: The Inner error is reported as above. The process exits for this inner error, and the current working operator name is HcclAllGather.[rank0]: Since the operator is called asynchronously, the stacktrace may be inaccurate. If you want to get the accurate stacktrace, pleace set the environment variable ASCEND_LAUNCH_BLOCKING=1.[rank0]: [ERROR] 2025-08-23-19:19:36 (PID:336613, Device:0, RankID:0) ERR00100 PTA call acl api failedW0823 19:19:43.775000 281473627685088 torch/distributed/elastic/multiprocessing/api.py:858] Sending process 336614 closing signal SIGTERM```
-
各位开发者们,本篇是为大家整理的实操教程集锦,包括官方输出、优秀开发者分享的实践案例。(持续更新)。请作为能量输入,开始构建专属的创意AI Agent吧~ <ModelArts Versatile-AI原生应用引擎 体验入口> 华为开发者空间 -- 开发平台 --Versatile Agent( 请在PC端打开 ) ——官方指导案例案例标题入口展示平台超详细攻略:教你3分钟在华为开发者空间构建专属AgentLINK博客-开发者空间快速使用华为开发者空间AI Agent打造你的私人营养师LINK论坛-开发者空间基于华为开发者空间开发平台 MCP资产快速构建AI Agent应用 LINK案例中心基于华为开发者空间开发平台构建We码会议助手LINK案例中心基于开发者空间开发平台工作流构建旅行行程规划应用LINK案例中心华为云ModelArts Versatile训练营基础实验手册——零基础秒变大师!快速开发帮你打造爆款AI Agent:热点新闻助手LINK论坛-开天aPaaS华为云ModelArts Versatile训练营基础实验手册——零基础秒变大师!快速开发帮你打造爆款AI Agent:出行规划助手LINK论坛-开天aPaaS ——精选用户共创案例案例标题入口展示平台案例贡献用户【案例共创】基于华为开发者空间构建实时股票分析助手LINK案例中心miyalian【案例共创】基于华为开发者空间-AI Agent开发平台构建旅游规划助手LINK案例中心yd_272483742【案例共创】基于华为开发者空间开发平台 MCP资产快速构建税务AI助手服务LINK案例中心小草飞上天【案例共创】基于华为云开发者空间-Versatile Agent开发平台构建昇腾C算子开发知识库LINK案例中心黄生【案例共创】基于华为云开发者空间的AI Agent【旅行灵感生成器】智能体LINK案例中心柠檬味拥抱 查看更多华为云社区-案例中心入口:cid:link_12
-
openai 接口测试,耗时 58 秒响应日志 启动脚本 显存占用情况
-
如题目前用了两张,还是报错[INFO|2025-08-19 18:40:33] llamafactory.api.chat:143 >> ==== request ==== { "model": "Qwen3-Coder-30B-A3B-Instruct", "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "hello" } ] } [WARNING|logging.py:328] 2025-08-19 18:40:34,022 >> `generation_config` default values have been modified to match model-specific defaults: {'top_k': 20, 'repetition_penalty': 1.05}. If this is not desired, please set these values explicitly. [E819 18:41:01.075574499 compiler_depend.ts:422] call aclnnIndexAdd failed, detail:EL0004: [PID: 39326] 2025-08-19-18:40:37.347.771 Failed to allocate memory. Possible Cause: Available memory is insufficient. Solution: Close applications not in use. TraceBack (most recent call last): Check param failed, mdl can not be null.[FUNC:LaunchKernelPrepare][FILE:context.cc][LINE:901] kernel launch prepare failed.[FUNC:LaunchKernelWithHandle][FILE:context.cc][LINE:1325] rtKernelLaunchWithHandleV2 execute failed, reason=[module new memory error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53] rtKernelLaunchWithHandleV2 failed: 207001 #### KernelLaunch failed: /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/built-in/op_impl/ai_core/tbe//kernel/ascend910b/slice/Slice_a32d70ed63082227463347a44bd5a08e_high_performance.o Kernel Run failed. opType: 43, Slice launch failed for Slice, errno:361001. Task execute failed, device_id=1, stream_id=4, task_id=0, flip_num=0, task_type=87.[FUNC:GetError][FILE:stream.cc][LINE:1119] Failed to synchronize stream, retCode=0x7150004.[FUNC:SyncGetDevMsg][FILE:api_impl.cc][LINE:5918] Sync get device msg failed, retCode=0x7150004.[FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:5937] rtGetDevMsg execute failed, reason=[tsfw param illegal][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53] [ERROR] 2025-08-19-18:41:01 (PID:39326, Device:1, RankID:-1) ERR01100 OPS call acl api failed Exception raised from operator() at build/CMakeFiles/torch_npu.dir/compiler_depend.ts:37 (most recent call first): frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0xb8 (0xffff9460f908 in /home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/lib/libc10.so) frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x6c (0xffff945be404 in /home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/lib/libc10.so) frame #2: <unknown function> + 0xe90bd8 (0xfffdec492bd8 in /home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch_npu/lib/libtorch_npu.so) frame #3: <unknown function> + 0x1644484 (0xfffdecc46484 in /home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch_npu/lib/libtorch_npu.so) frame #4: <unknown function> + 0x78d244 (0xfffdebd8f244 in /home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch_npu/lib/libtorch_npu.so) frame #5: <unknown function> + 0x78da58 (0xfffdebd8fa58 in /home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch_npu/lib/libtorch_npu.so) frame #6: <unknown function> + 0x78a1cc (0xfffdebd8c1cc in /home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch_npu/lib/libtorch_npu.so) frame #7: <unknown function> + 0x4c9e4c (0xffff9464ce4c in /home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/lib/libc10.so) frame #8: <unknown function> + 0x87c4 (0xffffabd3f7c4 in /usr/lib64/libpthread.so.0) frame #9: <unknown function> + 0xdbcec (0xffffabb79cec in /usr/lib64/libc.so.6) INFO: 127.0.0.1:41030 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/applications.py", line 113, in __call__ await self.middleware_stack(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__ raise exc File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__ await self.app(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 63, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/routing.py", line 716, in __call__ await self.middleware_stack(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/routing.py", line 736, in app await route.handle(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/routing.py", line 290, in handle await self.app(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/routing.py", line 78, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/starlette/routing.py", line 75, in app response = await f(request) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/fastapi/routing.py", line 302, in app raw_response = await run_endpoint_function( File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/fastapi/routing.py", line 213, in run_endpoint_function return await dependant.call(**values) File "/home/ma-user/work/LLaMA-Factory/src/llamafactory/api/app.py", line 110, in create_chat_completion return await create_chat_completion_response(request, chat_model) File "/home/ma-user/work/LLaMA-Factory/src/llamafactory/api/chat.py", line 189, in create_chat_completion_response responses = await chat_model.achat( File "/home/ma-user/work/LLaMA-Factory/src/llamafactory/chat/chat_model.py", line 92, in achat return await self.engine.chat(messages, system, tools, images, videos, audios, **input_kwargs) File "/home/ma-user/work/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 363, in chat return await asyncio.to_thread(self._chat, *input_args) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/concurrent/futures/thread.py", line 52, in run result = self.fn(*self.args, **self.kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/home/ma-user/work/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 240, in _chat generate_output = model.generate(**gen_kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/generation/utils.py", line 2460, in generate result = self._sample( File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/generation/utils.py", line 3426, in _sample outputs = self(**model_inputs, return_dict=True) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/accelerate/hooks.py", line 175, in new_forward output = module._old_forward(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/utils/generic.py", line 965, in wrapper output = func(self, *args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func return func(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/models/qwen3_moe/modeling_qwen3_moe.py", line 1043, in forward outputs: MoeModelOutputWithPast = self.model( File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/utils/generic.py", line 965, in wrapper output = func(self, *args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/models/qwen3_moe/modeling_qwen3_moe.py", line 673, in forward layer_outputs = decoder_layer( File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/accelerate/hooks.py", line 175, in new_forward output = module._old_forward(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/models/qwen3_moe/modeling_qwen3_moe.py", line 391, in forward hidden_states = self.mlp(hidden_states) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/accelerate/hooks.py", line 175, in new_forward output = module._old_forward(*args, **kwargs) File "/home/ma-user/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/models/qwen3_moe/modeling_qwen3_moe.py", line 277, in forward idx, top_x = torch.where(expert_mask[expert_idx]) RuntimeError: The Inner error is reported as above. The process exits for this inner error, and the current working operator name is aclnnIndexAdd. Since the operator is called asynchronously, the stacktrace may be inaccurate. If you want to get the accurate stacktrace, pleace set the environment variable ASCEND_LAUNCH_BLOCKING=1. Note: ASCEND_LAUNCH_BLOCKING=1 will force ops to run in synchronous mode, resulting in performance degradation. Please unset ASCEND_LAUNCH_BLOCKING in time after debugging. [ERROR] 2025-08-19-18:41:25 (PID:39326, Device:1, RankID:-1) ERR00100 PTA call acl api failed. EL0004: [PID: 39326] 2025-08-19-18:41:01.882.925 Failed to allocate memory. Possible Cause: Available memory is insufficient. Solution: Close applications not in use. TraceBack (most recent call last): Task execute failed, device_id=1, stream_id=5, task_id=0, flip_num=0, task_type=87.[FUNC:GetError][FILE:stream.cc][LINE:1119] Failed to synchronize stream, retCode=0x7150004.[FUNC:SyncGetDevMsg][FILE:api_impl.cc][LINE:5918] Sync get device msg failed, retCode=0x7150004.[FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:5937] rtGetDevMsg execute failed, reason=[tsfw param illegal][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53] EL0004: [PID: 39326] 2025-08-19-18:40:37.347.771 Failed to allocate memory. Possible Cause: Available memory is insufficient. Solution: Close applications not in use. TraceBack (most recent call last): Check param failed, mdl can not be null.[FUNC:LaunchKernelPrepare][FILE:context.cc][LINE:901] kernel launch prepare failed.[FUNC:LaunchKernelWithHandle][FILE:context.cc][LINE:1325] rtKernelLaunchWithHandleV2 execute failed, reason=[module new memory error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53] rtKernelLaunchWithHandleV2 failed: 207001 #### KernelLaunch failed: /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/built-in/op_impl/ai_core/tbe//kernel/ascend910b/slice/Slice_a32d70ed63082227463347a44bd5a08e_high_performance.o Kernel Run failed. opType: 43, Slice launch failed for Slice, errno:361001. Task execute failed, device_id=1, stream_id=4, task_id=0, flip_num=0, task_type=87.[FUNC:GetError][FILE:stream.cc][LINE:1119] Failed to synchronize stream, retCode=0x7150004.[FUNC:SyncGetDevMsg][FILE:api_impl.cc][LINE:5918] Sync get device msg failed, retCode=0x7150004.[FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:5937] rtGetDevMsg execute failed, reason=[tsfw param illegal][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
-
训练模型用的大规模数据集一般怎么存储呀?是直接用 云硬盘EVS,还是用 对象存储服务OBS?
上滑加载中
推荐直播
-
华为云码道Agent集成与鸿蒙实战2026/08/11 周二 19:00-21:00
王一男-华为云码道产品规划专家;李炎-华为云码道产品专家;彭江敏-华为云鸿蒙端云一体化开发专家
本次直播带你解读华为云码道7月份产品新特性、新功能。更有专家演示码道Agent Space × 钉钉机器集成实战,从0到1打通消息通道;码道鸿蒙端云一体化实战,快速搭建员工签到系统。
回顾中 -
华为云开发者AI素养直播课·第五期2026/09/04 周五 16:00-18:00
林华鼎-华为云AI开发者运营负责人;蒋春阳-华为云AI开发者案例开发专家
本期直播内容: AI工具体验营 · 第5-8课连讲。Agent-Team 多智能体协作完成毕业设计实践
回顾中 -
华为云开发者AI素养ClassRoom·第六期2026/09/08 周二 19:00-20:00
樊渊-2026华为软件挑战赛冠军
高手来了:看软挑高手解析二维排样问题—从工业难题到算法突破
回顾中
热门标签