The Mark Cerny Q&A – We Ask About VRR Improvements, PSSR FG, Path Tracing, and the ML Project with AMD

Last month, Wccftech was invited to join a small and exclusive number of outlets for an exclusive presentation hosted by PS5 Pro Lead System Architect Mark Cerny. While the main topic was, of course, the new console, there was even an exciting announcement regarding a collaboration with AMD on a machine learning project called 'Codename Amethyst.'

Following the presentation, we had a one-on-one chat with the legendary programmer and game designer (in case you didn't know, he worked on the first Crash Bandicoot and Spyro installments, as well as the Jak & Daxter franchise and, more recently, directed Knack and its sequel). Since this event took place ahead of the launch of the PlayStation 5 Pro, some of the topics discussed may have been revealed and/or confirmed by the publication of this article.

However, we also asked him a lot of future-facing questions, such as if PSSR would get a frame generation component, whether variable refresh rate on the console would be updated to support Low Framerate Compensation, and we also inquired about the feasibility of path tracing on PS5 Pro, among other things. Keep scrolling for the full transcription of our interview with Sony Interactive Entertainment's Mark Cerny.

The main focus of your presentation was arguably PlayStation Spectral Super Resolution (PSSR), so let's kick off with that. You previously commented on when the PlayStation 5 Pro project started internally, but when did you begin working on PSSR?

Mark Cerny: We started on PlayStation Spectral Super Resolution at roughly the same time as we started on the hardware itself. Though, the neural network we are using today is something that we came up with, I believe, in late 2021.

How has the development of the PlayStation 5 Pro hardware evolved over the past few years in tandem with PSSR?

Mark Cerny: The hardware definitely came first, because you can't know that your neural network is implementable unless you understand a bit about the hardware that it's going to be running on top of. So, simply put, in 2020 we did early hardware work. In 2021, we did more detailed hardware work and fully kicked off the PSSR project.

How do you see PSSR evolving after the launch of the console?

Mark Cerny: We're definitely going to continue our work in this space. There are so many ways that we can take that, though, right? We could continue to improve PSSR or start working on some of the other targets that I mentioned, like frame generation, frame extrapolation, ray tracing, denoising, and the like. I think it'll be interesting to see over the next few months what the full reaction from the development community is.

Are there any plans to maybe add a frame generation component to PSSR?

Mark Cerny: Well, at the current moment, our focus is definitely on Super Resolution. I'm just saying that there are a number of other very attractive targets out there.

Speaking about Super Resolution, does it require per-game training as the first version of NVIDIA Deep Learning Super Sampling, or is it more of a general model like DLSS 2.0 and later?

Mark Cerny: It's generalized training.

Can you talk about roughly how long it takes to implement PSSR compared to NVIDIA DLSS, AMD FSR, or Intel XeSS?

Mark Cerny: It's almost a drop-in replacement for the other strategies. The difficulty is that game engines tend to evolve along with the Super Resolution or the upscaling strategy that they're using. So, if you're trying to add something in after the fact that the game was not developed with, then there's a bit of rejiggering of the game engine that may be needed. Much of the time I think the developers spent bringing it over was the logistics of where in their pipeline they're going to insert PSSR.

Moving on from PSSR to the PlayStation 5 Pro hardware, the new SSD's size has obviously been greatly increased from the 768 GB of the PlayStation 5. Did the SSD's speed also improve in the PS5 Pro?

Mark Cerny: No, the speed of the SSD on PS5 Pro is the same as the speed of the SSD on PS5.

OK. Did Sony ever consider expanding the RAM during the development of the PS5 Pro?

Mark Cerny: Well, definitely more RAM is needed to implement those new features because PSSR needs a bit of RAM, and the ray tracing implementation definitely needs RAM. The question was just how much RAM is needed. We worked out that probably a gigabyte or so would be enough to support the big three that I was talking about.

We have GDDR 6, which is very fast memory—16 gigabytes of it. Then we added slow DDR5 RAM and started moving over to that from the fast RAM. That frees up fast RAM for the games, and that's the one or so GB I was talking about.

I know that during Remote Play, the PlayStation 5 disables some of the video capture capabilities. Are those enabled on the PlayStation 5 Pro, or does it have the same limitations in this regard?

Mark Cerny: The same limitations. There are many features that the gamer can use that require memory, and not all of them can be used simultaneously simply because there isn't enough memory on the system side. We want to get the bulk of the memory to the games, and the limitation set is the same on PS5 Pro.

Will Sony add Low Framerate Compensation or LFC to improve the Variable Refresh Rate (VRR) experience on PS5 and PS5 Pro?

Mark Cerny: There are ongoing discussions about where we should be taking VRR, and that's definitely part of those discussions.

Another major advancement of the PlayStation 5 Pro is in the ray tracing capability. Will that enable the console to run full path tracing, like we see on PC with games like Cyberpunk 2077, Alan Wake 2, and Indiana Jones and the Great Circle?

Mark Cerny: One difficulty in answering your question is that there are so many strategies for doing ray tracing or path tracing. Just to give an example, the engine that drives Alan Wake 2 is very, very different from the engine that drives Cyberpunk 2077.

There were talks on both of these at the Game Developers Conference that I went to. I really hesitate to say that PlayStation 5 Pro can or can't do path tracing, but you need to have a highly optimized strategy for doing it if you're going to do it on PlayStation 5 Pro.

Some PS5 Pro titles have implemented AMD Fidelity FX Super Resolution Frame Generation. Have there been any complications with that running in tandem with PSSR?

Mark Cerny: No, PSSR is Super Resolution, so whatever strategy a game has for frame generation should be completely orthogonal.

Alright. This is something I was curious about. At the launch of PS5, there were some PlayStation 4 titles like Bloodborne and The Last Guardian that, throughout their life cycle, were patched to have a locked 30 frames per second mode. If you go back to the 1.0 version of the disk, they also feature an unlocked frame rate. Has Sony been in conversations to kind of revisit that to unlock those frame rates on the PS5 Pro?

Mark Cerny: I'm afraid I can't say anything about Bloodborne or that other title that you mentioned on PS5 Pro. Typically, the games on PS5 Pro are running in what we're calling Boost Mode. In this mode, we do give the game the entire GPU and faster memory to use, but whatever limitations it had are still there on the PS5 Pro version. So there's things that are definitely going to get better and things that will stay the same. I will say if your game has VRR, you should see substantially higher frame rates on PS5 Pro.

Will the PlayStation 5 Pro substantially boost PlayStation VR 2 games compared to the original console?

Mark Cerny: PS VR 2 titles are definitely being boosted on PS5 Pro. There's a couple of different rendering strategies that VR titles take, depending on the strategy. PSSR is either something that works out of the box or something that needs a little bit more care and handling to be functional, and I think right now, we're at the point where we're getting pretty good uptake across the portfolio on PS VR2 for PlayStation Spectral Super Resolution.

Forgive me if you have mentioned this before - is PSSR going to be a toggle in the system settings for players who may want to play the title as it was originally developed?

Mark Cerny: It's all whatever the developer would like to do. It's what they feel is best for their game. Some of the developers, like Naughty Dog, are offering the PS5 modes on PS5 Pro, so you can pick Performance Pro or you can pick the original Performance or Fidelity modes, but other developers decided that what is best is that you would have strictly Pro modes when playing their game on the PlayStation 5 Pro.

The big surprise of today's presentation was the Amethyst machine learning project with AMD. What are the first steps of that collaboration going to look like?

Mark Cerny: Well, we've already started talking. Pretty much when PlayStation 5 Pro was wrapping up, we began discussions and development of those various aspects of Amethyst. To get more specific, I think we were getting pretty good momentum by the end of 2023. But as you know, it takes a phenomenally long amount of time to go from discussions and concepting to actually having a piece of hardware out in the marketplace.

One thing you touched on during the presentation was the democratization and standard libraries between SIE and AMD, which would be developed in parallel. Can you talk about how that partnership is going to be built out?

Mark Cerny: Certainly. Well, for hardware first, really this is not about PlayStation proprietary hardware. We're looking to co-develop performant new AI architectures that can be used freely in any space that anybody chooses to use them, whether that is PC, console, or cloud.

For the software, if I look at it today, we have games that run on a TV and there's a certain set of developers that make them, and AMD for PC has games that run on PC monitors that have a somewhat overlapping set of developers that make games for them. There are different needs and different relationships there, so really what we're looking to do is to co-develop those network architectures and those training strategies that can then be freely used in both contexts.

Another keyword from the presentation was discovering the holy grail of graphics through fully fused neural networks. What does that look like in your mind?

Mark Cerny: Ultimately, what you would see is really high quality libraries based on CNN's (convolutional neural networks) for game graphics. To me, that's a bit of a prerequisite. You need to have hardware that is capable of doing that.

Game developers are using so many different engines to say the least, between Decima, Unreal, Frostbite, and many more. How does machine learning tie those together? Is there kind of a unified path for the architecture here, whether it's PSSR or the other features on the PS5 Pro, to kind of mold itself to the requirements of those different engines?

Mark Cerny: I think we look out there globally, NVIDIA DLSS and AMD FSR are doing a pretty good job supporting a broad variety of game engines. So, yes, I believe that with sufficient care taken in meeting the needs of the individual game developers, it's definitely possible to create these broadly usable libraries,

So much of the focus has been obviously on the graphics and the visual aspect of the new PS5 Pro console. Does it also feature any additional capabilities for the audio component or anything else in the hardware, though?

Mark Cerny: Largely, the key improvements on the PS5 Pro are those big three. There are a few smaller features. The audio clock is a bit higher in case anybody wanted to use that. They can do a few more convolution reverbs; that is in there. Probably more significantly, you can run the CPU at a 10% higher frequency if you want to do so. By and large, though, the focus was on the big three key improvements.

To wrap up our chat, is there one title that you feel is the standout centerpiece that shows players what the PS5 Pro can do at launch?

Mark Cerny: There have been a number of pretty amazing titles on PlayStation 5 Pro, so I would really hate to pick one. I did personally enjoy seeing the eleven games we had at our hands-on event. On a personal note, that was really fun because I got to, in advance of that event, play and experience pretty much every game that was in development on PS5 Pro.

Wonderful. As always, thank you so much for your time.

Follow Wccftech on Google to get more of our news coverage in your feeds.