At GTC 2022, the NVIDIA Research Team unveiled an impressive technology called Instant NeRF that could turn 2D photos into 3D scenes in mere seconds.
To do this, the researchers leveraged the power of AI, more specifically through the so-called neural radiance fields. NeRFs use neural networks to reconstruct the scene from the images by predicting the color of light radiating in any direction. NVIDIA claims Instant NeRF to be the fastest technology of its kind yet, allowing for a multiplicative speed-up of over 1,000x in some instances so that rendering at 1080p occurs in mere milliseconds.
As discussed by Principal Research Scientist Thomas Muller in a GDC 2022 talk titled Instant Neural Graphics Primitives, this compounded effect is the result of three main improvements: a task-specific GPU implementation of the rendering/training algorithm, which uses the GPU's fine-grain control flow capabilities to be much faster than dense tensors; a fully fused implementation of a small neural network, which is faster than general purpose matrix multiplication routines; lastly, NVIDIA developed a technique called multiresolution hash grid encoding which is task agnostic and has better speed/quality tradeoff than preexisting work.
Unsurprisingly, the model for Instant NeRF was developed with the CUDA Toolkit and the Tiny CUDA Neural Networks library. You can dive deeper into the research and even access its code on this page; according to NVIDIA, the neural network is lightweight enough to run easily on a single GPU, especially if it is equipped with Tensor Cores.
David Luebke, vice president for graphics research at NVIDIA, said in a statement:
If traditional 3D representations like polygonal meshes are akin to vector images, NeRFs are like bitmap images: they densely capture the way light radiates from an object or within a scene. In that sense, Instant NeRF could be as important to 3D as digital cameras and JPEG compression have been to 2D photography — vastly increasing the speed, ease and reach of 3D capture and sharing.
The applications for the Instant NeRF technology can be many, from quickly scanning real environments or persons so that game creators can then use the digital scans in their projects to training self-driving cars or robots to understand the shape and size of real objects.
About the author: With over two decades of experience in gaming journalism, Alessio Palumbo has led the gaming vertical at Wccftech since August 2015. He started working at a young age for Italian websites like Everyeye.it, Gamestar.it, Nextgame.it, and Multiplayer.it before kickstarting the indie English-language publication Worlds Factory as its founder and Editor in Chief. In the last decade, he has coordinated the overall output of Wccftech's gaming section, managed PR relations, assigned reviews, produced daily news coverage, edited gaming content as needed, and delivered game reviews. Arguably, his trademark content is the long series of exclusive developer interviews that have been cited by Wikipedia and by the biggest news media and gaming publications. His passion for technology also makes him knowledgeable when it comes to gaming hardware and tech. His favorite genres include RPGs, MMORPGs, and action/adventure games.
Follow Wccftech on Google to get more of our news coverage in your feeds.
