NVIDIA: We Don’t Believe AotS Benchmark To Be A Good Indicator Of DX12 Performance
As you all know, NVIDIA released the 355.60 driver specifically for Ashes of the Singularity's Alpha, which is in itself a rare occurrence for a game still in development. Even so, we registered mixed results in our DX12 performance benchmarks with NVIDIA cards and clearly the company noticed all of this on its own, as they reached out to the press in order to give their side to the story.
We were able to get a detailed statement from NVIDIA's Brian Burke, Senior PR Manager. Here's what he had to say on the matter:
This title is in an early Alpha stage according to the creator. It’s hard to say what is going on with alpha software. It is still being finished and optimized. It still has bugs, such as the one that Oxide found where there is an issue on their side which negatively effects DX12 performance when MSAA is used. They are hoping to have a fix on their side shortly.
We think the game looks intriguing, but an alpha benchmark has limited usefulness. It will tell you how your system runs a series of preselected scenes from the alpha version of Ashes of Singularity. We do not believe it is a good indicator of overall DirectX 12 gaming performance.
We’ve worked closely with Microsoft for years on DirectX 12 and have powered every major DirectX 12 public demo they have shown. We have the upmost confidence in DX12, our DX12 drivers and our architecture’s ability to perform in DX12.
When accurate DX12 metrics arrive, the story will be the same as it was for DX11.
It should be noted that NVIDIA's mention of a MSAA performance bug while running on DX12 has been contested by developer Oxide Games, which published a blog post of its own talking about some "misinformation" being spread on the Ashes of the Singularity benchmark. They also dispute the fact that this test is not useful, of course:
It should not be considered that because the game is not yet publically out, it’s not a legitimate test. While there are still optimizations to be had, Ashes of the Singularity in its pre-beta stage is as – or more – optimized as most released games. What’s the point of optimizing code 6 months after a title is released, after all? Certainly, things will change a bit until release. But PC games with digital updates are always changing, we certainly won’t hold back from making big changes post launch if we feel it makes the game better!
There's also this cryptic but seemingly ominous tweet by Brad Wardell, CEO of Stardock, which is publishing Ashes of the Singularity.
— Brad Wardell (@draginol) August 16, 2015
NVIDIA and Oxide/Stardock are at odds right now, and it's easy to understand why. This is the first publicly available benchmark of DX12 performance and obviously NVIDIA would have liked to get different results than these ones; on the other hand, Oxide and Stardock aren't interested in taking the blame for all of this.
Obviously, it is impossible for us to say where the issue really lies, but there clearly is one right now. Of course, when it comes to consumers, there seems to be no cause for concern - whether there's something wrong on Ashes of the Singularity's Alpha or on NVIDIA's 355.60 driver, it will probably be fixed way before the game's release.
There's no reason to think that NVIDIA cards won't enjoy fairly similar DX12 performance boosts to what we have seen on AMD cards once the software is mature. As mentioned by Oxide in their blog post, DirectX 11 was quite terrible at first and that went on for a few years; DirectX 12, on the other hand, seems in much better shape already at least on the AMD side.
Let's give it a bit more time and it will likely shine on NVIDIA cards as well.