Following NVIDIA's Chat with RTX launch, AMD is now offering users their very own localized & GPT-based LLM-powered AI chatbot which can run on Ryzen AI CPUs & Radeon 7000 GPUs.
NVIDIA Chat With RTX Has A Competitor As AMD Unveils Its Own Localized LLM-Powered Chatbot That Can Run on Ryzen AI CPUs & Radeon 7000 GPUs
Last month, NVIDIA launched its "Chat with RTX" AI Chatbot which is available across its RTX 40 & RTX 30 GPUs & is accelerated with the TensorRT-LLM feature set which offers faster GenAI results based on the data you make available to it from your PC or in other terms, a localized dataset. Now AMD is offering its own LLM-based GPT chatbot which can run on a diverse range of hardware such as the Ryzen AI PCs which include Ryzen 7000 & Ryzen 8000 APUs featuring the XDNA NPUs along with the latest Radeon 7000 GPUs which feature AI accelerator cores.
3. In the search tab copy and paste the following search term depending on what you want to run:
a. If you would like to run Mistral 7b, search for: “TheBloke/OpenHermes-2.5-Mistral-7B-GGUF” and select it from the results on the left. It will typically be the first result. We are going with Mistral in this example.
b. If you would like to run LLAMA v2 7b, search for: “TheBloke/Llama-2-7B-Chat-GGUF” and select it from the results on the left. It will typically be the first result.
c. You can also experiment with other models here.
4. On the right-hand panel, scroll down till you see the Q4 K M model file. Click download.
a. We recommend Q4 K M for most models on Ryzen AI. Wait for it to finish downloading.
5. Go to the chat tab. Select the model from the central, drop-down menu in the top center and wait for it to finish loading up.
6. If you have an AMDRyzen AI PC you can start chatting!
a. If you have an AMDRadeon graphics card, please:
i. Check “GPU Offload” on the right-hand side panel.
ii. Move the slider all the way to “Max”.
iii. Make sure AMD ROCm is being shown as the detected GPU type.
iv. Start chatting!
Having a localized chatbot powered by AI can make life and work relatively easier if set up properly. You can be efficient in doing your work and get proper results based on your queries and the data path the LLM is targeted at. NVIDIA and AMD are accelerating the pace of AI-powered features for consumer-tier hardware and this is just the start, expect more innovations down the road as the AI PC segment reaches new heights.
About the author: A Software Engineer by training and a PC enthusiast by passion, Hassan Mujtaba serves as Wccftech's Senior Editor for hardware section. With years of experience in the industry, he specializes in deep-dive technical analysis of next-generation CPU and GPU architectures, motherboards, and cooling solutions. His work involves not only breaking news on upcoming technologies but also extensive hands-on reviews and benchmarking.
Follow Wccftech on Google to get more of our news coverage in your feeds.
