IOEHM large model inference all-in-one machine (digital human model)

Product Introduction:

1. The computing power card is combined with the Feiteng D2000 main control chip to create an integrated and portable product for large model application scenarios. The AI ​​large language model can be deployed on the edge and end sides, and local or offline computing and storage capabilities can be provided. The large model all-in-one machine can deploy the Tongyi Qianwen Qwen-72B large model. The power consumption of the whole machine is 120W. It has the advantages of high performance and low power consumption in the deployment of AI large language models. The whole machine is compact and convenient and can be used out of the box.

Product Parameters

2. Mechanical dimensions:

The product dimensions are 300 (length) × 150 (width) × 200 (height) (unit: mm). The mechanical interface diagram is shown in the figure below.

3. External interface description

4. Product advantages:

◼Support SIMT mainstream parallel computing programming model

◼ Compiler supports CUDA C, Open CL source code

◼ Graph compilation optimization tool, efficient support for AI1.0/AI2.0 model efficient deployment

◼ Support PyTorch framework: reasoning & training

◼ Provide general deployment tools at language level, operator level, and model level

◼ Intelligent video analysis framework--- DeepWeave (deep stream/triton), plug-in development method, fast construction of algorithm pipeline

◼ Support TGI framework, unified algorithm reasoning service, convenient for user integration

◼ Support RAG+LLM reasoning solution, and provide reference design

5. Support various deep neural networks
6. 

6. List of adapted models

7. Application features of large models in the financial industry

Available functions:

• We have built a nationwide private deployment service for our customers
• Employees are significantly more efficient in obtaining information and performing routine tasks
• Significantly fewer compliance issues due to reduced human error
• The speed and quality of decision making are significantly improved with instant access to market data and analytical reports