XPU: GPU Virtualization

    XPU software is a system software that provides GPU virtualization capabilities for AI applications in cloud or data centers and CUDA applications. Through the efficient communication mechanism, AI applications and CUDA applications can run on any physical machine in the cloud or data center, Container or VM without the need to mount a physical GPU. At the same time provide these applications with hardware computing power in the GPU resource pool.

    The XPU has two components: the host-side component and the container-side component. The host-side component includes: XPU toolkit, XPU service &XPU (kernel) driver module. XPU toolkit and service take charge of virtual GPU management, and XPU driver module virtualizes one physical GPU to many vGPUs which can be assigned and used by containers. It is the host-side component that enforces fault isolation, compute isolation & memory isolation among containers. The container-side component includes: XPU container runtime. XPU container runtime helps the container recognize the virtualized GPUs just as physical GPUs, so the AI application inside the container can use the virtual GPUs even without knowing those are virtual GPUs.