CryEngine 3: Dynamic Difficulty System
Development Overview:
Game: Crysis 2
Engine: CryEngine 3
Development Time: 6 Months
Genre: First-Person-Shooter
Project Goals:
Research the current and past uses of Dynamic Difficulty Adjustment (DDA) in games
Outline the positives and negatives of using this type of system in modern FPS games
Develop own DDA system that measures aspects of player style and appropriately adjusts difficulty
Document the development and iteration of the system outlining what changed and why
Project Overview:
As part of my Masters requirements for the Guildhall, I was tasked with researching a thesis topic of my choosing, developing some type of artifact, and writing a paper on my findings. While at first I looked at several different options, for example a hex-based tabletop wargame, I eventually decided to pursue a topic that I felt would help me grow as a level designer and scripter. After some discussion with professors and other advisors, I chose to delve into the topic of creating my own Dynamic Difficulty system or DDA. DDA is the idea of using a system that tracks a player's actions and process through a game or level. Depending on how the player performs, the system alters certain aspects of the level and experience to bring things more in line with the player's skill level. For example, if the system records that the player is taking a lot of damage during encounters, depending on the goal of the system, the system could adjust any number of factors to either make the game easier or harder. This could include, but not limited to, dropping extra health packs and ammunition, spawning more enemies or one type and less of another, or even moving cover and world objects around. Having never done something like this before, I was understandably concerned about a number of potential problems. However, I still decided to take on the task and really challenge myself.
Start at the Beginning:
As a starting idea, I decided to take a look at basing the DDA system off of time. The idea was that as a player proceeds through a first-person-shooter, and many action games for that fact, they encounter moments of intense action, followed by lulls, all of which play into the game's interest curve. The idea was that those players who are more skilled would take less time in each encounter than their less skilled counter parts. My system would start a timer when the player began one of these encounters and stop when the encounter was over, when all the enemies involved in that encounter were dead. I believed I could use this idea to try and make a simple system that would be later be expanded upon given both the game's and system's needs.
To test the base theory, I created a simple equation that would compare a player's time for an encounter to a collection of preset times and alter a player's encounter score. At the start of the level, the player received a number known as their Encounter Score. For the first equation, the encounter score was on a scale of one to ten and the player began the level with a score of five. The player would start the encounter's timer by crossing a triggering volume and stop it by killing all the enemies that belonged to that encounter. The first encounter of the base system had three preset times to compare against the player's time, less than fifteen seconds, less than twenty-five seconds, and greater than twenty-five seconds. Once compared, the system would add a value to the player's encounter score based on their time, +2 for less than fifteen seconds, 0 for less than twenty-five seconds, -2 for greater than twenty-five seconds. This left players with a potential score of 3, 5, or 7 for their encounter score. As the players moved into the the next area, they would cross another trigger volume. The DDA system looked at the player's encounter score , then spawned a different collection of enemies depending on that score, 3 receiving an easier challenge, 5 receiving a medium challenge, 7 receiving a difficult challenge. With all the enemies dead in the second encounter, the second encounter's timer stopped and the player's time for the second encounter was compared against that encounter's collection of preset times. So on the process went until the player completed the level, fighting enemies and encounters that would give them an appropriate challenge. The tests for this base system provided plenty of ideas for improvement and outlined some of the potential problems, but most importantly it proved that the engine and system could work together.
Development Summary:
With the concept proven and tests proceeding well, the project continued forward. As the project progressed, we encountered three key problems that forced major iteration and revision. The first big obstacle my adviser and I encounter was the idea that the base system, only served to assess the player at the singular moment when they triggered each encounter. This lead to potentially massive swings in difficulty, especially if a player got a few lucky shots. As such, the equation needed to change in order to assess the player over the entire span of the level. To address this, the system was changed to retain the player's previous encounter scores for each encounter. As the player proceeded through the level, the player's three most recent encounters would be averaged in and taken at a portion of their original value. The just completed encounter was kept unchanged, the previously completed encounter was taken at half value, and the encounter before that taken at a third of the value. This was all combined together then divided by 1.83 in order to keep the data within a 1-100 scale. Additionally, this iteration also saw the expansion of the preset times from three possibilities to five adding more variations beyond just the main easy, medium, and hard.
The preliminary tests for the second iteration showed promise, but my adviser and I soon encountered another problem. The original premise of the system was that skilled players take less time to complete an encounter than unskilled players. However, my adviser was an exceptionally skilled shooter, taking well-aimed shots only when the A.I. exposed themselves, sticking to cover, taking little if any damage, but taking several times longer than other skilled players who tested the system. As such, I knew I needed at least one other way to evaluate a player's skill so the system could accurately score them. At first I looked at a number of options such as seeing if the player was using the crysis suit's special abilities and powers. However this idea proved difficult not only due to Flowgraph limitations, but also unclear what exactly the data meant. For example, if a player deployed their armor ability, did that mean that they were using it defensive to stay alive, or offensively so they could charge into the enemy? And should either of these option positively or negatively affect a player's encounter score or was it better for a player to not use their powers?
Ultimately, I was able to find another variable of player performance to track, the player's total damage taken per-encounter. This system worked similarly to the timing system with the player's final total damage being compared to a series of preset health averages. When the player took damage, the amount of damage they took was compared against 10 gauges of total damage. Depending on how much damage the player took within a span of time, the total amount of damage was added together. This value was then subtracted from what was at the time called the health buffer value. As players took more damage, the resulting buffer value would grow smaller until it went negative, there by reducing a player's final encounter score. However, not only was this initial stab at a health tracking system overly complex, broken due to the system sometimes outright skipping some of the gauges, but also the health variable was tacked onto the equation, not evaluated on equal terms as the player's time. As such, we needed another, final revision.
After reworking the health system into a much simpler one that compared the player's health every few seconds then adding the difference, I reworked the whole equation into the 4th Iteration as shown on the right. By normalizing a player's time and their total health damage, I managed to get a simple to understand system that output a value between -2 and 2 with 0 being average. Negative numbers were good, positive numbers were bad, much like golf. The equation still took into account the player's previous scores throughout the level, leading to an experience that slowly adjusted to the player, giving them appropriate experiences based on their skill.
Final Version, What does it do?:
There were a few different ideas as to what the system could ultimately adjust in order to make an encounter more or less difficult. However, after reading Asher Einhorn's article on, "Reducing Difficulty Dynamically and Invisibly," I knew I wanted to make sure the system did so without being obvious. Knowing that my system was still built on time, it would be easy for a player to potentially break the system if they ever discovered a DDA system was in play. All they would need to do is kill all but one enemy then wait. As such, I decided to focus mainly on changing the types of enemies that spawned and altering their initial starting location as it would be unlikely that a gamer would pick up on the difference. As seen in the pictures to the left, this is the same encounter first with the easiest variation, then a medium variation, then the hardest variation. At first, the major difference between the three is the number of enemies. The easiest only has five enemies, the medium has six, and the hardest has seven. While this is true, another important thing to note is the position and type of enemy. As notated by the enemy symbols explained in the Encounter Key, the easiest version of the encounter has four machine gunners and one shotgun user. While that at first does not seem strange, it is important to note that the machine guns are not effective at long range. This means players who struggle to aim and successfully kill their opponent still feel like they have a decent challenge as dozens of bullets whiz by their head. Meanwhile players in higher difficulties not only get more shotgun users, high damage weapons that could spell the end for a less experienced player, more skilled players also face riflemen which are very effective at longer ranges. In the hardest version, players also end up facing a captain, an enemy with a higher than average life total who is also equipped with a rifle. Additionally, note the starting position of all the enemies in each version. In the easiest version, enemies are essentially spaced out into smaller groups, the two in the front and the other three towards the back. This allows less skilled players the opportunity to engage and kill one or two enemies before the rest of the squad can intercede. Meanwhile, the harder difficulties have more clumped units or units that are better positioned to make the best use of their weapons. Riflemen towards the back, shotgun users towards the front, it all works together to make each version of the encounter slightly more difficult than the previous version. The video below shows three separate runs at the above encounter, each one with a different intended level of skill.
Conclusion:
The purpose of this thesis was to push myself to design a Dynamic Difficulty system. To learn not only the purpose and potential value of such a system, but also understand what goes into building and implementing a complex system like this for a game. While I learned plenty about DDA's, how they work, and their role in modern games, I would say the most important lesson I learned from this whole process is: a game system only needs to be as complex as it absolutely needs to be. If a project requires a particular system, the simpler and easier it is to understand, the better it is for both the designer and the customer.
As for my DDA system, the system as a whole provides a good foundation on which to build other, future systems. With access to more parameters such as player accuracy, more data could be gathered from the player and the encounters adjusted to better fit them. Personally, I feel that the system as is could work on an encounter to encounter basis, reading a player's skill level and adjusting individual encounters. For example, a player enters an area and normally must face off against three snipers and a few riflemen across a small ravine. The foundation of the encounter focuses on long range combat while staying mobile so riflemen cannot pin the player down and snipers cannot zero in. If a player is particularly skilled, the DDA system could adjust the difficulty by spawning three snipers and four riflemen, giving the player a true challenge. However, for the less skilled players, the system could adjust the encounter to have only one or two of the snipers active at any one time, stretching out the encounter, but keeping the encounter’s foundation intact. I feel there is plenty yet to be explored with such a system and look forward to delving into those options some time in the future.