Behind the Veil #30: Change is Good for the WalletPosted 15 July 2005 by Rushster
Change is Good for the Wallet
So you’re a programmer. You’re working on some game for some anonymous company that treats you like a small number, when it suddenly occurs to you there’s a better way to do something. A slight formula change, for instance, or a sword
isn’t as long as it should be. You decide you like your idea so much that you want to put it in the game.
Hold on a second though; we discussed this last time. If you just go ahead and implement your change, you’ll end up costing the company millions of dollars, weeks of release time, and probably some chocolate fudge as well. So what’s the solution?
The answer to this dilemma is called the Change Control Protocol (CCP). In quick and dirty terms, this means before you actually change anything, you run it past the people who know more than you, exactly the way many politicians don’t.
A good CCP will not only look neat when acronymed, but it will save you all kinds of time and money. Next to the game itself, the CCP just might be the most important part of the project. Here’s how it works:
Some programmer notices a glitch that needs fixing, or something that could be done another way. Rather than fix it immediately, a memo is sent up to the department head, who passes it on to the head of Concept.
The head of Concept looks at the proposed change, and decides whether it’s worth considering for inclusion to the game. If it seems like a reasonable request, it gets brought up with the entire Concept group (or a select portion depending on what else is going on in the company) and the Lead Programmer (or a proxy) to find out ahead of time what other parts of the game it would impact, and how easy or difficult it would be to incorporate in to the game.
Concept spends probably a day or two, or anywhere up to a week or more, looking at the domino effect this proposed change would cause. It is their job to find every far-reaching influence of this idea, and figure out how to keep the game balanced. When this task is complete, they have either figured out how to include the change or feature, or they have decided it’s not worth the time to put it in and the idea gets shelved. If they feel it’s worth including in the game, a list of changes that must be made to accomodate it has been written, which we’ll call the CCP document.
When this decision comes back down to the coders, it’s up to them to either implement the required changes throughout the system, or swallow the decision and forget about how cool their idea was. If there’s a list of changes to be carried out, it will probably take a day or two to go through the code and update the required pieces.
Seems unnecessarily complex, right? Want me to prove to you that this will save you time and money? Let’s look at our example from last time, which used only dates.
Day n (1 for convenience): On schedule. Coder thinks of a neat idea, and passes it up the line according to the CCP. Coder then continues working on whatever it is he was doing.
Day n+1: The idea gets passed to Concept for CCP review. Code is still on schedule.
Day n+4: After three days of intensive discussion and coffee consumption, it’s decided that the change can be implemented, and a CCP document is produced.
Day n+5: The CCP document is passed down to the coders, who start fixing the items outlined in it. By this time, the coder with the original idea has figured out the details of the required code, so it should only take a day or two to write.
Day n+7: Everything is fixed, and coding proceeds as planned, only one day behind. It will probably be caught back up by the end of the development cycle, but for dramatic effect let’s say it ends up cutting in to testing time by one day.
So following the CCP saved you 30 days and a million dollars. It looks kind of trivial if it still takes up a day of coding time, but what if there are fifty such ideas to be looked at? If they all get included, the CCP seems to dictate that the game will launch fifty days late, which is a fairly significant amount. At least, it looks significant until you consider the game would likely launch more than a year late if no CCP was in place. Remember, it cost 30 days for one change in the last column. Fifty days is much better than a year, and don’t think for one second that this doesn’t happen in the real world.
Of course, that assumes every change is implemented. Let’s now draw on the example from last time, only assume it ends up being a catastrophic failure that must be removed post-launch. I can’t think of any instances where this has actually happened, but I’m sure someone somewhere has done it before. Picking up where we left off last time…
Release Day (RD for convenience) +30: The change turns out to be unrealistic for the game. It breaks too many things, causes discomfort to players, and there is more violence in schools. The order comes down from above to remove the feature.
RD +31: The coder responsible starts digging through the code to find all the places he had to change to make his idea work.
RD +45: Almost there…
RD +50: The code is finally removed, but now the game is broken because things didn’t get put back quite right. Concept and Coders meet for the first time in two years to figure out how things are supposed to be.
RD +60: The game is fixed, and a brand new 30mb patch is put up online. Because this column takes place in a perfect world, all 500,000 users download the patch, chewing up 15 Terrabytes of bandwidth and slowing the servers to a crawl
RD +70: All the kinks are finally worked out, and the game is running as it should. Millions of dollars have been lost to bad publicity, people leaving the game, and extra money paid to the coders who had to fix the game.
Now, consider the model where the CCP is followed:
Day n+4: After three days of intensive discussion and coffee consumption, it’s decided that the change would require far too much to be altered in order to be implemented, and the decision made is to not include it. Someone goes to the coder and slaps him in the back of the head for coming up with the idea, and no delays are incurred.
I don’t know about you, but I prefer the instance where there isn’t 70 extra days of development time. It becomes even more important when one feature has to be ripped out, but another feature that was coded is dependant on it; pulling out the bad feature endangers the good feature, like removing a cancerous tumor from your spine endangers your ability to walk. It’s better to just never get the tumor.
The CCP also has to be applied to Cowboy Programming, which is basically Cool Stuff Syndrome post-launch. A developer thinks of a way to fix a bug, or a cool feature to add, and simply adds it without documentation and without telling anyone. The CCP is especially important in these cases, because there won’t be an in-depth testing phase if it’s added to a game that’s already been launched. I considered doing a column on Cowboy Programming, but it would look very similar to the last one, so this paragraph will have to do.
Of course, there is one fairly significant problem with setting up a CCP: How do you enforce it? Anybody can say that all changes must follow the CCP, but it only takes one programmer to ignore the protocol and go out on his own in an attempt to better the code. The answer to this problem is deceptively simple: announce the punishment for breaking the CCP, and then follow through with it.
When the coding jobs are assigned, it’s known well in advance who will be responsible for what parts of the game. This means if there’s a new feature in the spellcasting system, everyone already knows who’s responsible. If the punishment is to lose two months pay, I assure you the odds of someone breaking the CCP are significantly lower than if the punishment was mere chastizing. Of course, you have to follow through with the punishment if someone breaks the protocol, so don’t announce a punishment you’re not willing to dish out.
And that’s the CCP in a nutshell. Over the last three columns I’ve been picking on the programmers, but the same problems occur in every other part of the game as well. If you don’t believe me, check out the bloopers on the Shrek DVD, where someone changed a 7 to an 8 and completely messed up the models. As funny as Donkey looks with one tooth sticking through the top of his head like a spike, it just isn’t right.
Disclaimer: Behind the Veil was written by Chris Marks and hosted by Diabloii.net. The opinions expressed in these columns are those of the author, and not necessarily those of Diii.net.