For this devlog I decided to take the opportunity to talk about the animations in Shenmue, and more specifically about the challenges met when we first took a look at the animations, and what we ultimately decided our plan of action should be. This specific topic is a bit long-winded, and may be a bit too technical for some people, but I will try to keep it as simple as possible.
The way that animations work in Shenmue is different from how most games deal with animation. Just like real life, characters in games are animated by manipulating virtual “bones” which are assigned sections of the 3D model itself. For example, the upper body is comprised of a spine, shoulders, biceps, forearms, hands, as well as the neck and head. These bones are commonly called a hierarchy as that’s how they’re created and ultimately operate in real life, in a parent and child relationship. This hierarchy affects how bones can move/rotate; for example, a hand bone which is a child of a forearm bone can only rotate itself, however rotating the forearm will also rotate the hand with it as it is a child of the forearm.
An example of this hierarchy can be seen here:
As for animation, there are generally two ways to approach this in games, Forward Kinematics (FK) and Inverse Kinematics (IK). To put it simply, Forward Kinematics refers to the animation working from beginning to end; this means that the animation data will rotate and move bones in order, such as from shoulder, to biceps, to forearms, and finally on to hands. An animator using FK will have to rotate and position each bone manually in order to achieve the pose they desire. Inverse Kinematics, on the other hand, is the complete opposite; goal positions are provided to the engine and it calculates the rotation and position of the bones.
Here are two examples of FK and IK:
FK IK
IK has the added benefit of feeling more natural for animators to create animations with and appear more fluid and natural compared to FK, as it allows easier posing through “control rigs”, which are custom made virtual objects which are used by the animators to manipulate the body, without having to manually rotate each bone in the chain. IK also has another use-case which is to plant feet on the ground when a character is on uneven terrain or to align an IK chain with another object, for example attaching a cup to a hand.
When we first started our research into the game files we already had an idea that the game used IK to some extent, because Ryo has foot planting when walking around in FREE and the Virtua Fighter series was credited for being the first game to use IK. Since there are a lot of similarities between VF and Shenmue, it was assumed there would be at least some IK at play. After finishing our research on models, we set out to decode the animation format and quickly found that it wasn’t a regular FK animation format, but a custom format which only describes IK animations. This brings about a few issues as IK has a number of implementations, as its main use case is in the robotics and manufacturing industries where they use robotic arms to aid in the manufacturing of various products, as well as its application in the games industry.
There are numerous types of IK solvers, each with their own requirements and performance profiles, so understanding how the Shenmue IK solver works is imperative to implementing the animations as accurately as possible. This is simply due to the nature of IK – instead of supplying rotations for each bone in the hierarchy (like FK), there are (for the most part) only values for the end positions of a chain with IK. This means that one goal position (for example; the end position for a hand) designed for use with one IK solver may create a different pose for the forearm/arm when used with a different IK solver.
Because both games feature hundreds of unique characters and animals etc, it could be noted that the choice to move to an IK animation format was a conscious one in the hopes of reducing disk space used by FK animations. This is because multiple rotations can be expressed from a single goal position of an end effector for an IK chain and thus there is less data to store to disc when compared to FK animations, which have to store a rotation for every frame for every bone in the hierarchy. Another reason why this is useful is for animation sharing.
Animation sharing allows a single animation to be used with more than one character and in the context of Shenmue, this also aids in the fighting system which is brought over from Virtua Fighter 3. In Shenmue, since there are a multitude of different types of humans, with different proportions, it is highly advantageous to be able to reuse a single generic animation like a walk animation on a number of different characters, while still giving an authentic and natural human-like motion based on their unique characteristics and proportions. This is also useful for the fighting system employed in Shenmue as different characters use different ‘fighting styles’ which play on the movesets among other variables to alter their in-game fighting behaviour.
The investigation began with the usual file format research which already had some attempts at it, but none successful enough to really use as a base. Starting with what we already knew, which was foot planting, I decided to use this as a jumping off point for research because there are generally 2 ways to deal with foot planting, pre and post animation. After only a short time researching how the foot planting works in Shenmue, I discovered that although the foot planting itself all happens post-animation (which is usually a sign that IK is not an integral part of the underlying animation system). I also found that most of the code was similar to what I saw in the main function that I already suspected may be the IK solver.
Here is a GIF showing the removal of the footplanting function at runtime:
With that over with, we proceeded to look further into exactly how the IK solver seems to be working. IK solvers usually have some sort of constraints, and so far we hadn’t found anything that resembled them, as we would expect those to be clearly defined within the model formats. After a lot of research we found that the rotation keyframes declared within the animations were actually describing the rotational limits of a bone, which is used by the IK solver when solving the position and orientation of bones within a chain.
This can be shown relatively simply by showing an animation with just IK positions and the same animation with IK positions and the “preferred rotation” keyframes.
IK positions only
IK positions + Preferred Rotations
Shenmue uses a relatively simplistic IK solver – there are 3 primary uses for it within both games.
Animation / Fighting
The first one is the most obvious, which is animation. The game relies on everything detailed above to drive all character animations; this includes non-humanoids like cats, dogs and other animals. This system is beneficial for a game like Shenmue as it provides the ability to play back the same animations on different characters (even with widly different hierarchies) in a way which looks realistic and more natural in general thanks to inverse kinematics.
Fighting also makes heavy use of the IK system, mainly for accurately positioning characters hands and feet during complex blended animations (for example going from a ‘damaged’ animation into a kick/punch). Throughout the game, Ryo has the ability to practice moves which directly impact his proficiency in performing those moves when engaged in fights. The way this is achieved is by storing various pieces of data such as animation length inside an additional chunk in the animation data. This data is read and used as limiters for the animation itself. At the beginning of the game you’ll start with no proficiency in most moves and over time, as you train throughout the game, your proficiency values are used to determine when that particular fight animation should end. A fully trained move should show more of the animation and a move which has never been trained for will show much less of its animation.
Foot planting
The other primary use is foot planting and this is only really used for Ryo. The reason for this is not exactly clear right now as my research focuses mainly on the d3t re-releases of the games. Foot planting uses a lot of the same principles as what is used for the main animation system, with the main difference being that the solver is only being asked to calculate the correct position for the feet in order to be perfectly aligned with the ground, to stop any clipping/collision issues. Interestingly, although IK is employed across all animations and thus all NPCs, only Ryo has footplanting enabled at all times and I’ve yet to see NPCs with footplanting enabled.
Another important aspect to foot planting which is worth noting is that it is disabled when fighting. This is likely because of hardware constraints and the performance profile of the IK solver they decided to use. Most fights take place on flat ground and there’s not a need for foot planting in Shenmue’s fights. However, there were early screenshots which show a fight taking place on uneven ground. This may show that there were some difficulties regarding bringing over Virtua Fighter’s entire IK solution as Virtua Fighter 3 implemented footplanting for this specific reason, and it would make sense they would want to try and bring this over.
The game uses a single 1-byte memory address to toggle footplanting on and off when necessary:
Object interaction
Other benefits include aligning objects to hands, or aligning an IK chain to another IK chain, which Shenmue does often, whether it’s attaching the Phoenix Mirror to the hands of Lan Di or it’s handcuffing Ren and Ryo together in Shenmue II. Inverse kinematics solves problems that these designs would bring, so it makes sense why it was chosen for both games. Object interaction in general is something which benefits greatly from IK as it makes dealing with the logic and look-and-feel of the in-game event/scenario much easier for developers and animators. This is also beneficial when turning on light switches and interacting with cupboards at runtime.
Implementing and designing a system which utilises the animation format used in Shenmue is limited by the data it stores. Since the animation data stores only IK animations, this means our system also has to carry just this data and reproduce the same animation. This comes with some challenges as there isn’t a ‘standard’ IK solver which is used across the board; and unlike FK, there are a number of different ways to implement it. Given that Shenmue uses a relatively simple IK solver, I’ve decided we should use a similar solver like CCD/regular IK-FK as opposed to its modern equivalents, this is to keep the original animations intact and to also keep things as simple as possible.
The way this is achieved is by isolating the IK solver and ensuring it can be freely moved around, this is extremely useful during R&D as it allows me to prototype all of the functionality we need in something like a plugin for a 3D app which allows us to use the solver we choose. Then once we were happy with the results we were producing, we made the move to start porting this functionality into UE4, or otherwise seeking equivalent methods which are native to UE4 already.
How animations were designed to run on the original Dreamcast is very impressive and the features they were able to implement are groundbreaking, even to this day.
โ
Further research/analysis of the animation format used in Shenmue can be found here, here and in the Shenmue Reverse Engineers discord server.