Steam Audio Introduces AMD TrueAudio Next Support
Valve announced earlier this week the release of Steam Audio 2.0 beta 13, which adds support for AMD’s TrueAudio Next sound technology. According to Valve, this will allow developers to enable increased acoustic complexity and a greater sense of presence in both VR and non-VR applications; below you can find their longer, technical explanation.
As a reminder, TrueAudio Next was introduced with the Radeon 400 series as a means to utilize the GPU in order to simulate audio physics; the SDK is open source and available through GPUOpen.
How is TrueAudio Next useful to Steam Audio?
Steam Audio uses physics-based simulation to model two ways in which sound travels from a source to a listener. Direct sound is sound that travels in a straight line from the source to the listener, possibly passing through solid objects. Indirect sound is sound that is emitted from the source, bounces off solid objects repeatedly, and then reaches the listener.
Steam Audio models indirect sound by calculating an IR using ray tracing. This IR is then used in a convolution reverb effect to add indirect sound effects to either individual sound sources (in the case of source-centric convolution reverb) or a submix of sound reaching the listener (in the case of listener-centric convolution reverb).
Simulating and rendering indirect sound is the most computationally intensive part of what Steam Audio does, and TrueAudio Next helps manage this workload.
Source-centric convolution reverb can be directional in nature: somebody else’s distant footsteps heard through an open doorway sound more directional than one’s own footsteps in a large room, for example. Steam Audio uses Ambisonics to represent the directional variation of indirect sound. The higher the Ambisonics order, the more spatial detail that can be rendered in the indirect sound. On the other hand, the higher the Ambisonics order, the more IRs that are needed in the convolution reverb effect. For example, 2nd order Ambisonics requires 9 convolutions per source. This too, increases the computational cost of indirect sound.
Note that Steam Audio does not use TrueAudio Next for applying HRTF-based 3D audio rendering. The computational cost of HRTF processing is significantly lower than that of convolution reverb: thousands of sources can be rendered with HRTF-based 3D audio using a single CPU core.
Why does Steam Audio support TrueAudio Next?
Steam Audio’s existing CPU-based convolution has sufficient performance for many applications, particularly those that rely heavily on listener-centric reverb. However, with TrueAudio Next, Steam Audio can offer more flexibility to developers.
There are two main reasons why we chose to support TrueAudio Next in Steam Audio:
- If we can provide developers with more flexibility in choosing how their audio processing workloads are balanced on a user’s PC, we want to do that. TrueAudio Next allows developers to choose how CPU and GPU resources work together to provide a compelling audio experience on the user’s PC.
- If we can allow developers to offer an optional higher level of audio detail with their existing content on PCs that are powerful enough, we want to do that. With Steam Audio, developers just specify higher settings to be used if a TAN-capable GPU is installed on the user’s PC. Developers do not have to re-author any content to work with TAN. Steam Audio can be used to seamlessly select either CPU- or GPU-based convolution, depending on the user’s hardware configuration.
Steam Audio’s support for TrueAudio Next does not in any way restrict Steam® Audio to specific hardware or platforms. We hope that our support for TrueAudio Next encourages hardware and platform vendors to provide more options for developers to balance their audio workloads against graphics, physics, and other workloads, which in turn will help them create better audio experiences for their users.
What are the benefits of TrueAudio Next?
Benefits of GPU convolution
TAN’s OpenCL convolution algorithms move convolution from the CPU to the GPU, and help achieve significant audio processing speedups. This has the following advantages:
- Steam Audio’s CPU-based convolution is highly efficient, and is sufficient for a wide variety of games and VR applications, especially if only using listener-centric reverb. However, TAN allows us to give developers the option of reaching an even higher level of auditory complexity: increasing the number of sources, or the Ambisonics order for indirect sound, or the IR length, etc.
- By moving convolution from the CPU to the GPU, the fraction of time spent by the CPU on audio processing can be kept within the CPU budget chosen by the developer. This also frees up more CPU resources for use by other tasks such as physics simulation or AI.
Figure: GPU convolution performance vs CPU convolution performance, for 1s IRs, 1st order Ambisonics, 1024-sample frames, and a 48 kHz sampling rate. The performance advantage of GPU convolution increases with number of sources. CPU convolution is usable up to a moderate number of sources; GPU convolution is necessary for large numbers of sources.
Benefits of resource reservation
Resource reservation causes TAN’s convolution algorithms to run in isolation from any rendering, physics, or other tasks that the GPU is being used for. This has two broad advantages:
- Increasing the number of reserved CUs predictably results in improved performance, even under the heaviest workloads.
- Because audio processing does not interfere with graphics, the audio processing does not result in sudden frame rate drops and stuttering.
- Because graphics does not interfere with audio processing, rendering a complex scene cannot cause audio processing to slow down, which in turn helps avoid unacceptable clicks, pops, or other audio glitches.
Figure: Performance improvements with increasing numbers of reserved CUs, for 1s IRs, 1st order Ambisonics, 1024-sample frames, and a 48 kHz sampling rate. The plot shows the maximum number of sources that can be processed within a 6ms budget. Increasing the number of reserved CUs allows more sources to be processed within the same amount of time.
Figure: Impact on GPU rendering performance when reserving 4 CUs for TrueAudio Next processing. The plots show the average FPS observed on various GPU benchmarks when run at the same time as a TAN benchmark using Steam® Audio. Measurements obtained using a Radeon RX 480 GPU with 4 reserved CUs, 256 total convolution channels and 1.3s IRs. All values are shown as a percentage of a baseline that is obtained when the TAN benchmark is not running. For example, reserving 10% of the available CUs reduces the average FPS in the Timespy benchmark by about 10%.
Figure: Impact on TAN processing time when heavy graphics workloads are running concurrently. CU reservation limits the impact of heavy graphics workloads on time-sensitive audio processing tasks on the GPU. Measurements obtained using a Radeon RX 480 GPU with 4 reserved CUs, 256 total convolution channels, 1.3s IRs, 1024-sample audio frames, and a sampling rate of 48 kHz.