As with any game, audio such as music and sound effects play a large role in the general atmosphere the game’s world provides. This is no different in Shenmue, and in the opinion of many, Music in particular is a primary focal point of how Shenmue makes its game world feel vibrant and alive. From the smooth jazz of Heartbeats, to the lighthearted jingle emanating from the Tomato Store, the Shenmue games really do seem to go out of their way to place the soundtrack in the forefront of the atmosphere; and we intend to do no different here. A tall order, to be sure, but one I am certain we can fulfill, so let’s all take a few minutes, to discuss and understand exactly how we intend to handle this in Shenmue: Dragon & Phoenix Collection.
Music in Shenmue, for the most part, is encoded in a proprietary format, that makes the most of the AICA sound processor present in the Sega Dreamcast. It is a MIDI based format, making use of samples compressed to various degrees, along with other features provided by the AICA processor, such as panning and multichannel mixing, to present the almost CD-quality soundtrack present within the game. The D3t releases does in fact utilize the same format, however for reasons that are not fully understood, it appears they weren’t entirely certain how the format and/or AICA processor worked, and as a result, had to alter these files to facilitate playback, leading to inaccuracies. This is noticeable in some areas more than others; and while it is overall a rather small detail in what is otherwise a generally faithful port, it does tend to stick out like a sore thumb given the prominent nature of music in creating Shenmue’s unique atmosphere. As such, I’m happy to say that as a result of a collaborative effort between Lemonhaze and team friend Sappharad (https://github.com/Sappharad), we now fully understand the changes D3t made to these files, can reverse them to be 1:1 with the Dreamcast releases, and are now able to accurately play back these tracks through Unreal Engine 4. To illustrate this, let’s take a moment to listen to one of these tracks, and compare between the Dreamcast and PC releases, and then let’s take a listen to what it sounds like in Unreal Engine 4. Today, we’re going to listen to Queen’s St. from Shenmue II; I believe this to have been a particular sore spot for many players in the rereleases, so I feel this should be a good illustration as to the level of accuracy and authenticity we are striving for, particularly in regards to the Dragon Edition portion of this project.
Dreamcast
PC
Unreal Engine 4
Now, this is all well and good for Dragon Edition…. But what about Phoenix Edition? What about our audio upgrades? Well, my friends, I have good news for you. I’m happy to announce that we are working with Richard Cartlidge, who has been diligently working on recreating a large variety of songs from the Shenmue soundtrack. We’re very excited to have Mr. Cartlidge on the team, and have the utmost confidence that he can revitalize the soundtrack and make the soundtrack to Shenmue: Phoenix Edition every bit as entrancing as the original. Now, being that this partnership is only a few days old, we don’t have much to show to this end just yet. However, you can check out the work he has already done on Youtube here.
Of course, this now leaves the elephant in the room. Dialogue. Unfortunately, at this time, there is nothing to announce in the way of enhancement for dialogue audio for Phoenix Edition. As it stands, there is simply too much, and the quality of it is too varied, for any of us currently involved with Shenmue Dragon & Phoenix Collection to do much in the way of true enhancement. AI upscaling of audio is a very involved process, doubly so when speech is involved. With where we are now, it would require a separate, dedicated team to go through the dialogue, and process it to a point where it would be worth using, not to mention we don’t have anyone on hand with experience in AI audio upscaling. Even then, results are not guaranteed, and are very limited by current AI audio restoration technology. This regrettably means we have to relegate this to the “someday” pile. Other ideas have been suggested to us, such as using dialogue from Shenmue: The Movie (which is, notably, the highest quality source we have for some dialogue), but unfortunately, this also has its own drawbacks. In order to use this as a source for dialogue, the movie’s audio must be extracted, re-cut, re-timed, and then processed to remove background noise such as music and sound effects. This introduces its own artifacts, and it is simply a matter of personal taste as to which is more pleasant to listen to; the compression artifacts from various releases of the game, or the editing artifacts from covering up background noise in the movie’s extracted audio. For this project, I firmly believe that the former is the better option here, as not only do I find the compression artifacts less intrusive, Shenmue: The Movie only accounts for a small fraction of the overall dialogue present within Shenmue and Shenmue II. Introducing these other artifacts, as well as the generally enhanced quality of these lines of dialogue, will not only highlight both types of artifacts, but will equally make the disparity in quality stand out even further. For these reasons, the current plan is to leave the dialogue audio in place as is for the time being, in the interest of creating a more consistent experience. That being said, as we do intend for Shenmue: Dragon & Phoenix Collection to have mod support, there is absolutely no reason why a separate team could not create an enhanced dialogue package after release. Theoretically, one could even begin work on such a package now, as all the tools and documentation available (on the Shenmue side of things) are readily available, however it is just not feasible for us to undertake such an endeavor at this time.
This news, however, is not all bad. There is a silver lining that I hope we can all take solace in. This move to Unreal Engine 4 affords us a lot of flexibility in terms of visual fidelity. We can take these old models and textures, and slap a new coat of paint on them. Truly make the best out of these assets made before the turn of the century. This flexibility, however, also extends to audio. Modern engines have a host of audio features that allow us to truly make the best out of a bad situation here. Unreal Engine 4 natively supports spatial audio rendering, which in and of itself goes a long way towards covering up the hiss of audio compression, but atop that, we have a bevy of options towards the attenuation of these sounds, allowing us to really bring out everything we can from these ancient recordings.
Well, that’s about all I have for this entry. I’m sorry the news couldn’t be all good; but as always, we’re open to suggestion, so feel free to stop by our Discord and we’ll be happy to discuss whatever it is you have on your mind.
https://youtu.be/_VL7A8ymVVs
Until next time,