NVIDIA's TensorRT AI acceleration is now available on GeForce RTX GPUs, offering 2x the performance uplift over DirectML.
NVIDIA Offers 2x AI Acceleration Boost Over DirectML With TensorRT, Available Across All RTX GPUs
Today, NVIDIA is announcing that it's bringing TensorRT to its RTX platform. With TensorRT, general consumers running an RTX GPU will be able to get faster performance through the optimized inference backend.
With TensorRT, users will be able to see uplifts of up to 2x in AI applications versus DirectML. TensorRT is also supported natively by Windows ML, and it should be noted that TensorRT-LLM is already available on Windows.
Today’s AI PC software stack requires developers to choose between frameworks that have broad hardware support but lower performance, or optimized paths that only cover certain hardware or model types and require the developer to maintain multiple paths. The new Windows ML inference framework was built to solve these challenges.
Windows ML is built on top of ONNX Runtime and seamlessly connects to an optimized AI execution layer provided and maintained by each hardware manufacturer. For GeForce RTX GPUs, Windows ML automatically uses TensorRT for RTX — an inference library optimized for high performance and rapid deployment. Compared to DirectML, TensorRT delivers over 50% faster performance for AI workloads on PCs.
Windows ML also delivers quality of life benefits for the developer. It can automatically select the right hardware to run each AI feature, and download the execution provider for that hardware, removing the need to package those files into their app. This allows NVIDIA to provide the latest TensorRT performance optimizations to users as soon as they are ready. And because it’s built on ONNX Runtime, Windows ML works with any ONNX model.
But it's not just performance that is the prime aspect of TensorRT for RTX. The new backend allows for 8x smaller library file sizes and also comes with Just-in-time optimizations per GPU. TensorRT for RTX will be available in June across all NVIDIA GeForce RTX GPUs at developer.nvidia.com.
In one of the performance slides, NVIDIA showcases AI performance uplifts with TensorRT versus DirectML. In ComfyUI, users can get a 2x performance bump, while DaVinci Resolve and Vegas Pro offer a 60% bump. This leads to faster AI runtimes and workflows, allowing RTX GPUs and RTX PCs to fully unleash their potential.
Software innovations by NVIDIA don't end there, as the company is powering over 150 AI SDKs with 5 brand-new ISV integrations coming this month. These include:
- LM Studio (+30% performance with latest CUDA)
- Topaz Video AI (GenAI Video accelerated CUDA)
- Bilibili (NVIDIA Broadcast Effects)
- AutoDesk VRED (DLSS 4)
- Chaos Enscape (DLSS 4)
NVIDIA is also introducing new NIMs and AI Blueprints, which even include new Plugins for Project G-Assist, such as Discord, Gemini, IFTTT, Twitch, Spotify, and SignalRGB integration. Of course, users also have the option to build their plugins for Project G-Assist by going to github.com/NVIDIA/G-Assist.
About the author: A Software Engineer by training and a PC enthusiast by passion, Hassan Mujtaba serves as Wccftech's Senior Editor for hardware section. With years of experience in the industry, he specializes in deep-dive technical analysis of next-generation CPU and GPU architectures, motherboards, and cooling solutions. His work involves not only breaking news on upcoming technologies but also extensive hands-on reviews and benchmarking.
Follow Wccftech on Google to get more of our news coverage in your feeds.
