Progress Report August 2022
Written by GoldenX86 on September 16 2022

Welcome back, yuz-ers! August was a month packed with progress. A plethora of graphical changes, kernel and file system improvements, and more work pumped into network emulation. Next slide, please!

Graphical changes

This month included a number of notable fixes for July’s blockbuster release, Xenoblade Chronicles 3. While Xenoblade Chronicles 3 did technically work on release day in yuzu, it took us some time to work out the stability issues.

The first of these, mentioned last month, was a fix for yuzu’s shader recompiler. yuzu does not use a GLSL compiler to generate SPIR-V, the Vulkan shader language. Instead, it generates SPIR-V directly, which is much faster, but has more potential for error. In this case, the shader recompiler failed to declare some variables in the SPIR-V shaders, and left references to them as null, causing the shaders to crash the driver when compiling. Maide and byte[] quickly found and fixed the problem, allowing the game to run in Vulkan on release day.

 Wotah! (Xenoblade Chronicles 3)

Wotah! (Xenoblade Chronicles 3)

The set of pipelines used by Xenoblade Chronicles 3 is huge, with many pipeline caches clocking over 15000 entries after extended gameplay. With all graphics backends, yuzu precompiles shaders before the game starts, and doesn’t allow starting it before compilation is finished. With OpenGL on GLSL, compilation takes over a full minute to complete with a pipeline cache as large as the ones generated by Xenoblade Chronicles 3. To make matters worse, you previously could not cancel out of the process while it was compiling. If you tried, yuzu would freeze up and continue to wait for the compilation to finish before allowing you to close the game. byte[] resolved this minor UX annoyance by having the shader compiler check for cancellation after each shader is compiled, allowing compilation to instantly be stopped at any time.

 Name's Kermit the Frog, pleased to meet you (Xenoblade Chronicles 3)

Name's Kermit the Frog, pleased to meet you (Xenoblade Chronicles 3)

Xenoblade Chronicles 3’s massive set of pipelines also causes problems for emulation. Because yuzu tries to give you as stutter-free an experience as it can, it tries to load all known pipelines into memory before you even start playing. This might be a reasonable proposition if a game had just a few thousand pipelines. However, keeping Xenoblade’s 15000+ distinct pipelines all in memory simultaneously causes the memory usage in the GPU driver to skyrocket. Cemu users of old may remember facing a similar issue when using NVIDIA GPUs.

To this end, byte[] looked for any issues in yuzu that might be causing the GPU driver to hold on to more memory than it needs to. byte[] noticed that OpenGL tracks individual objects for shader source code in addition to just for compiled shaders. Properly allowing these shader source code objects to be freed reclaims a substantial amount of wasted memory. Users can expect much lower VRAM and RAM usage now while running OpenGL and suffering emotional damage.

 That's a big snuffin' difference (Xenoblade Chronicles 3)

That's a big snuffin' difference (Xenoblade Chronicles 3)

One advantage yuzu’s OpenGL backend has had for Xenoblade Chronicles 3 for some time was stability. It didn’t really crash. However, Vulkan was another story. Even after the initial Vulkan shader fixes previously mentioned, the game would still randomly crash during cutscenes. byte[] was able to reliably reproduce the issue and found that yuzu was receiving some invalid references in the GPU processing. Whether or not this was something the game was legitimately doing, or just an emulation bug, he identified that yuzu could avoid the crash by passing VK_NULL_HANDLE for an image view, instead of trying to create an image view for a null image, which was guaranteed to crash.

 Vulkan no longer cooks itself (Xenoblade Chronicles 3)

Vulkan no longer cooks itself (Xenoblade Chronicles 3)

AMD users on Windows were left in the dark about Vulkan support for Xenoblade Chronicles 3 for a bit. One of our testers, your writer, obtained a log with Vulkan validation errors, and determined that the crash was due to trying to use some vertex formats that were not supported by the Windows driver. However, they were supported by RADV, the Linux Mesa driver. byte[] additionally identified a case of the opposite, where a format was supported by the Windows driver but not RADV, and implemented a generic workaround that would substitute a compatible format for both cases, fixing multiple crashes at once.

 Progressively getting fit (Xenoblade Chronicles 3)

Progressively getting fit (Xenoblade Chronicles 3)

A few GPU changes not related to Xenoblade also made it in this month.

After years, since the beginning of our Vulkan backend, yuzu finally added support for VSync in Vulkan by new contributor djrobx. As described by the pull request adding it, if the emulation and display are very close to precise sync, there can be fits of juddering when the newly generated frames don’t decisively fall as next to be displayed. This change allows yuzu to completely synchronize frames with the monitor refresh rate, preventing any juddering. Thank you very much!

 No more tearing, Vulkan edition

No more tearing, Vulkan edition

Merry, the author of dynarmic (and a real life White Mage), implemented an optimization for the process of converting between tiled and untiled images for the GPU. This doesn’t impact the performance of games very much, as they generally avoid converting between tiled and untiled images, but it significantly improves the performance of homebrew apps, which convert every frame to display to the console.

On the topic of unusual techniques, vonchenplus implemented support for a legacy OpenGL format called rectangle textures.

Legacy features seem to be popular with some game studios Legacy features seem to be popular with some game studios

Legacy features seem to be popular with some game studios

As a side effect of this change, a pipeline cache purge had to be issued by Morph.

You may be thinking, but aren’t all textures rectangles? That is true, but the thing making rectangle textures distinct is that they are indexed in shaders with coordinates from 0 to their width/height – texel coordinates, instead of 0 to 1 – normalized coordinates. This mode is not very useful, so it is a surprise to see it still used by games. This change fixed a lot of rendering issues in Fast RMX and The Touryst (two games using the same engine).

No more vacations in the Abyss (The Touryst) No more vacations in the Abyss (The Touryst)

No more vacations in the Abyss (The Touryst)

In the case of Fast RMX, disabling DYNAMIC CAMERA EXPOSURE in its in-game graphic settings fixes an overbright rendering bug. Testing shows that OpenGL seems to be more stable for this title.

F-Zero for Switch when, Nintendo? (FAST RMX)

Also focusing on homebrew, vonchenplus implemented support for a few things needed to get the RetroArch homebrew working inside yuzu. Yup, more “emulator in emulator” shenanigans.

The first step is the clkrst service, used to control clock speeds on the console, which was stubbed to allow RetroArch to boot. Emulation of clock speed is not very meaningful for yuzu, so it’s not a loss. The second is a specialty behaviour by switch-mesa, which uses a different framebuffer crop and appears halfway down the screen without proper emulation.

While preparing the Project London changes for release, Tobi found some issues with Rocket League that were preventing it from working properly. byte[] found that it could boot in single core, but the game had terrible rendering in Vulkan. To do this, byte[] added support for the R16G16B16X16 format used by the game, fixing the rendering issues. Still, more work is needed to make this game playable on yuzu.

The better League (Rocket League) The better League (Rocket League)

The better League (Rocket League)

Working on his Magnum Opus, byte[] also implemented a workaround for a previous performance fix to Super Mario Galaxy and Super Mario Sunshine. While the performance fix should have been regression-free, since it was a bug in the buffer cache, many games needed the less accurate behaviour from before the change. Mario + Rabbids Kingdom Battle, Splatoon 2, and Super Smash Bros. Ultimate all had some extra flickering that was introduced after this pull request, so byte[] made the feature, pessimistic flushing, optional, and disabled by default, since most games were unaffected.

 Feel free to test it, but expect a performance loss

Feel free to test it, but expect a performance loss

Finally on the list for GPU changes this month, german77, along with bylaws from the Skyline emulator, added emulation for the viewport swizzling extensions used by some games. These were supported by NVIDIA GPUs with an extension, but not AMD or Intel GPUs, and so a few games would appear to render upside down or flipped. Games like Street of Rage 4, Celeste, Axiom Verge, and many others are now playable on AMD and Intel!

A new meaning for inverse climbing (Celeste) A new meaning for inverse climbing (Celeste)

A new meaning for inverse climbing (Celeste)

There should be a difficulty mode called upside-down (Axiom Verge) There should be a difficulty mode called upside-down (Axiom Verge)

There should be a difficulty mode called upside-down (Axiom Verge)

CPU, kernel, and file system emulation

We have some good news for users still running out-of-support versions of Windows! Thanks to an investigation done by vladkosi, Merry was able to implement the required changes in fastmem to support Windows versions older than Windows 10 version 1803 again.

While this is great news for the Windows 7 gang, it does not change our position regarding operating system version support: only Linux, Windows 10 version 1803, and newer are officially supported. Time will tell what happens to compatibility with older Windows versions after future changes to Vulkan and memory management are implemented, but for now, they work for a while longer.

But the good news doesn’t end there. Here are some changes intended to help the modding community!

In the past, inline comments added to pchtxt mods weren’t allowed, as we parsed the text in a way that made them incompatible. Morph now delimits the hexadecimal value string, allowing modders to add comments. For example:

Before: here             to here
         v                 v
12345678 12345678 // comment
         ^      ^
After:  here  to here

Our current file system emulation can be considered spaghetti code due to old guesses made when less information was available, and patchwork done by developers that are no longer contributing to the project. There’s no way to sugar-coat it, this is the main reason Morph has been working so hard for over a year on Project Gaia, a full rewrite of the current virtual file system implementation.

Meanwhile, some further patchwork can be done to reduce the headaches developers and modders currently endure. For example, Build IDs are padded by a bunch of zeroes, and modders are more used to the short form without including the irrelevant zeroes at the beginning. To remedy this, byte[] parses the Build ID string using fmt to dynamically add the required zeroes when needed.

Continuing to focus on improving RetroArch’s experience in yuzu, vonchenplus discovered that recursively calculating the size of the files when parsing the directories has a negative effect on performance, in this case perceived as “loading times” when booting a game with the emulator (inside an emulator). Simply ignoring this information is a good way to improve the user experience. Note that this change doesn’t help in running native Switch games on yuzu, it only provides faster load times for RetroArch.

UI changes

People have been asking for stronger audio volume in yuzu for a long time, and Maide delivers. Now, if the user goes to Emulation > Configure… > Audio, they can set the volume up to 200%, really cranking it to 11!

 Don't wake up your neighbours

Don't wake up your neighbours

Something to keep in mind is that some games have their volume tuned up to the output they are using. A game may be louder in handheld over docked, for example, so don’t stress your poor ears.

Part of yuzu’s code is still based on Citra, even some sections that are no longer relevant, or have become outdated over the years. Cleaning up code is a boring but necessary part of any project, more so for open source ones, and yuzu is no exception. One feature that was very useful in the early days is loading ELF binaries. Instead of dealing with XCI and NSP dumps, back in the early days, devs dumped and extracted the whole file structure of the game, and loaded the ELF binary directly, in a similar way to how on Windows the user opens a .exe file to load a program.

Some years down the line and not only is this feature tarnished out of grace, but it also stopped working. Since it’s not a priority at the moment, and it can lead to user confusion, byte[] removed it from the File dropdown.

Asian Language Windows users noticed that the Size column in the game list and the Speed Percent setting displayed very weird characters.

Alien numbers

Of course this isn’t acceptable, so Docteh implemented the necessary changes to workaround this issue in Windows.

 Much better

Much better

Adding to the list of Asian-language-only UI bugs, if a user started yuzu, for example, in Chinese and then switched to English, the Filter bar at the bottom and the game list columns at the top would remain untranslated, refusing to switch. Forcing Qt to retranslate these objects solves the issue, making Docteh happy.

Docteh implemented a couple more UI fixes as well, cleaning up the About dialog and making the default Port number in the Direct Connect UI ignore translations.

yuzu About dialog changes yuzu About dialog changes

yuzu About dialog changes

Users complained that the Controller Profile drop down list’s character limit was too short, inhibiting their creativity. german77 stepped up and increased the character limit.

Input improvements

UI navigation with a controller is a relatively recent addition to yuzu, so there is room for improvement. For example, users reported that if the “Prompt for user on game boot” feature is enabled, controller inputs will be able to navigate the options and start the game, but it won’t be able to actually change the profile or cancel the boot process. Newcomer jonesyUK solves both issues with a bit of Qt magic. Thank you!

Some users prefer to depend on DS4Windows instead of yuzu’s native support for game controllers. While there is a good reason for that, it’s a simple way to have a single app handle XInput emulation, it can also lead to some interesting issues german77 discovered. For example, current versions of DS4Windows can’t handle two subsequent vibration commands, producing (depending on the person you ask) an uncomfortable, constant vibration. To solve this brrrrrracing issue, german77 added a small delay so the app doesn’t skip the stop signal.

On a more colourful topic, and with the intention of fixing a bug with Mario Party Superstars, german77 spent many hours cooking how to properly emulate controller colours, with a side dish of fixing battery level icons too!

 Art Attack!

Art Attack!

Navigate to Emulation > Configure… > Controls > Advanced and go to town!

And to close this section out, we have to own up to an oopsie on our side. With the initial implementation of IR camera support, the output was saved to storage. Every single video frame as an individual image file… This led to TONS of files being written to disk, making navigation with the file manager a pain, and eating a lot of free storage. Oops. german77 acted quickly and moved the output to a buffer.

If any user suffered from this and can’t remove the files, open the Command Prompt or Terminal, navigate to the Pictures folder, and run del IMG_00*.jpg to wipe them all out in a single blow.

Hardware section

Intel ARC joins the fun

We received information that early adopters of A380 Intel GPUs had issues running their games using Vulkan. Since this hardware is not globally available yet, it’s not easy for our developers and testers to get their hands on it, so please post issues in our GitHub including log files, or contact us on our forums or our Discord server. We’ll ask for more information, if needed, to find the reason for this. Hurray for the early adopter woes.

AMD, making use of OpenGL

Users with AMD GPUs reported that specific points in Xenoblade Chronicles 3 always resulted in vertex explosions. A workaround for this is to run the game with OpenGL and a 2X resolution multiplier, get past the area, save, and swap to Vulkan.

Why the oddly specific 2X resolution multiplier? For the same reason our performance chart last month had no results for Xenoblade Chronicles 2 on this new OpenGL driver, it leads to a driver related crash. 2X works around this crash and AMD has been properly alerted to this issue. No news from them so far, but since we got access into their Vanguard beta testing program, we’ll tell you as soon as the fix is out in a public driver. The game is not that much slower under OpenGL now, but shader stuttering will always be significantly better under Vulkan.

We recommend more emulator developers to apply to Vanguard, as the indirect approach of the built-in driver bug report tool is not ideal for the specific needs of console emulators.

NVIDIA making us go compute ourselves

We confirmed that at least a big part of the crashes affecting Maxwell and Pascal GPUs by the current 516.XX series of drivers is caused by some driver change in compute shaders, which resonates with our compute shader based ASTC texture decoder. As a solution, and to avoid this problem in the future as it isn’t the first time an NVIDIA driver update broke it, we’re considering switching to a multithreaded CPU decoder developed by Morph.

The cause for the shadow flickering on KOEI games is still under investigation.

Future projects

The delays in merging Project Y.F.C. don’t only affect Mainline users, they also impede progress in development. In another example of Hell freezing over (no one expected AMD fixing OpenGL), NVIDIA of all people open sourced some documentation (partially of course, this is NVIDIA we’re talking about, they’re so closed source they don’t even get fresh air). This includes proper names for Maxwell 3D registers, completely new registers, and even previously unknown values. Maide has been checking, replacing, and cleaning a lot of code thanks to this new official information, but the merge for this change will have to wait. No game fixes are expected out of this, but the clean-up and ease of understanding is very much worth it.

An interesting experiment that will benefit all users in a few bandwidth-starved games like The Legend of Zelda: Breath of the Wild, Xenoblade Chronicles 3, and SHIN MEGAMI TENSEI V, among others, is increasing the size of Vulkan’s stream buffer size. Testing shows a performance boost of 10-50% depending on the system, even without ReBAR support, which limits the buffer size to 256MB. Having access to all available VRAM opens the possibility for even higher gains!

Look at the performance difference! (Xenoblade Chronicles 3) Look at the performance difference! (Xenoblade Chronicles 3)

Look at the performance difference! (Xenoblade Chronicles 3)

After too much nagging from your writer, byte[] implemented a draft to easily test the feature with a slider, but the final goal is to manage it dynamically, so don’t expect to see this feature as a UI option in an official release.

Working on the finishing touches for Project London, Tobi has not been passive at all:

 The most fun ever doing internal testing

The most fun ever doing internal testing

Oh! How could we almost forget? Project London, LDN support, is finally out now! More information in the dedicated article and Multiplayer Guide. We’ll recap the release in the next Progress Report.

That’s all folks! Thank you so much for sticking with us, and I hope to see you all next time!

Special thanks to Mysterious Writer B for your help on this article!

 The suspense bites me

The suspense bites me

 

Please consider supporting us on Patreon!
If you would like to contribute to this project, check out our GitHub!


Advertisement

Advertisement