June 29, 2008

p

Five Kinds of Programmers

Filed under: Engineering — Joe @ 12:23 pm

I recently had a conversation with one of the long-time programmers on Pirates that got me thinking about how I think about programmers. Over the course of my career I’ve run into several archetypes of professional programmers. I thought it might be interesting to formalize my thinking on the subject, and this is the result.

The Researcher

These programmers are more scientist than engineer. If your organization has a research lab, it is probably stocked with Researchers. Since academia is just one giant lab, it is almost entire filled with Researchers.

The Researcher loves to find solutions to problems that are poorly understood. They are on the bleeding edge of their technological specialty. If there are no papers out there that explain how to do something they will write one.

One downside of the Researcher is that there are so many interesting problems out there that need solving that they have trouble actually finishing any solution before they move on to the next thing. When you can get these guys to check in some code it’s usually great, but it takes them far longer than it would take other kinds of programmers to actually implement anything. They are also the most likely archetype to suffer from Not Invented Here Syndrome.

The Explorer

Like the Researcher, the Explorer is unafraid of the poorly defined dark corners of technology. The key difference is that when the Explorer delves those depths it is to get things done, not for the joy of the exploration itself.

When you have a really thorny problem that you don’t know how to solve, this is the programmer you give it to. Explorers will dig into unfamiliar code-bases and problem domains with a shocking level of energy. These programmers are by far the quickest learners, and are a great resource for other programmers who are trying come behind them into new territory.

The downside of Explorers is that their single-minded practicality can make their code a little sloppy. These programmers dedicate a lot more time to putting their current task behind them than they do to writing code they would want to maintain years down the road. This doesn’t mean that the code won’t work, but that if an extra #include or circular dependency will save an hour the Explorer is always tempted to cut that corner.

The Craftsman

The highest quality code in your code-base was probably written by a Craftsman. Your QA department loves Craftsmen. They value the quality of their work above all else.

When a new system just has to work, you give it to a Craftsman. They will do a great job coding it, and then test it until it is perfect. Craftsmen are absolutely the best programmers when it comes to handling exceptional conditions and corner cases. In my experience Craftsmen also excel at writing maintainable code because they know that they’re going to have to come back to it someday.

Unfortunately all that quality comes at a price. The Craftsmen on your team are the slowest programmers you have. When they estimate tasks they generate the most accurate estimates, but also the biggest. (Partly because they always include the debugging time that everyone else hopes won’t be necessary.) Their emphasis on quality and reliability also means that Craftsmen are terrified of unfamiliar parts of the code-base or poorly defined problems.

The Activist

You know that guy on your team who is pushing Test-Driven Development, is constantly refactoring code, and actually uses the names of design patterns? That guy is your Activist. They are the driving force for architectural and process improvements on your team.

Activists want the code quality in your project to be as high as it can be. They give tough code reviews, and even tougher design reviews, but that’s a good thing. Every time someone on the team listens to the Activist, they are improving as a programmer.

On the other hand, their ceaseless pursuit of perfect code hurts the productivity of the Activist. Quick hacks are physically painful to them, even when that is exactly what the situation calls for. Paradoxically, they also often introduce bugs with their refactoring that never would have come up otherwise. (On the plus side, the refactoring makes fixing that bug far easier.)

The Workhorse

In their various ways, all of the programmers above are sacrificing some of their capacity to their particular quirks. Workhorse programmers don’t do that. They are in a single-minded pursuit of adding as much to the system as possible, and as a result end up owning vast chunks of the code-base.

If you were count lines of code per programmer, the Workhorses would come out ahead. (That’s assuming you don’t count generated code from the Activists.) Sheer output is the domain of these kind of programmers. If you have a few great Workhorses on your team you will be able to do things that other teams only dream of.

The dark flip side of what a great workhorse can accomplish is that a bad one will do absurd amounts of damage to your code-base. Workhorses don’t have any significant dedication to quality that allows them to avoid doing bad things. Sometimes make up for this by having enough time to build the system two or three times in the time that a Craftsman would build it once, but that’s always painful. A single bad Workhorse can do enough damage to negate the positive effect of one or two other programmers.

What kind of programmer are you?

You will notice that none of these archetypes are particularly bad or particularly good. There can be good or bad programmers of any archetype. All the teams I’ve ever been on have had a mix of archetypes. For that matter, very few programmers could be assigned to one archetype.

Personally, I think I’m mostly a Workhorse with a little bit of Activist and Explorer mixed in. I am put to shame by the ability of the some of the programmers around me to suss out how to do some radical new thing. I’m not hard-core enough about process or code quality to keep up with the Activists on the team. The one way I compete is on quantity, and most of that code is fortunately good enough to not doom any projects I’ve been on up to this point.

What about you? Where would you fit in this taxonomy? Do you recognize any programmers you know?

June 9, 2008

p

My console MMO post is up on Gamasutra

Filed under: Administrativia — Joe @ 9:17 am

You can find it here. Of course if you’re reading this you probably already read the original post. Mostly I’m putting it here so I can find the Gamasutra version if I ever need it. (And because I think it’s really cool that they reposted it.)

May 19, 2008

p

Pirates Post-partum at ION

Filed under: Engineering, Day Job, Game Design, History, Game Industry, Production — Joe @ 12:14 pm

At ION I gave a talk on our development process for Pirates. Darius Kazemi has posted a transcript of the talk. It’s also up at the Vault Network. I wonder how much buzz it’s going to get.

I’m giving the same talk at AGDC this year, so if you missed me at ION you can catch it there.

May 17, 2008

p

Why aren’t there more console MMOs?

Filed under: Game Design, Game Industry — Joe @ 10:19 pm

A few weeks ago Dan Rubenfield posted this (as part of a larger rant putting all MMO developers on notice):

If you continue to refuse to acknowledge consoles as the de-facto standard for AAA gaming, you will go out of business.

Quit making PC games. It’s a waste of time and money.

(NPD respectfully disagrees with the waste of money part.)

I for one would love to build a console MMO. It’s not that MMO developers don’t acknowledge consoles as dominant, it’s that there are many barriers to building a console MMO that don’t exist on the PC. I mentioned a couple of those in my comment to the post above, but wanted to expand on them here.

Barrier #1: Platform Holders Demand a Share

Assuming a moderate success MMOs are almost unique in their ability to give game developers a revenue stream. Most studios live from milestone payment to milestone payment and rarely see royalties off the game after it ships. If they’re smart they make a little extra on each milestone and can build a buffer to help them tough it out between projects, but often failing to sign with a publisher for the next project drives the developer out of business. With a few very successful exceptions, just about all studios live on this edge.

Ongoing revenues from subscriptions or micro-transactions change all of that. These revenue streams require constant updates to keep going. That means that the publisher needs the developer to stay in business so they can keep working on the game. Assuming modest success, it also means that eventually the developer is going to pay back their advance and start earning royalties. This seems to have worked out pretty well for Cryptic who are developing Champions Online without a publisher.

When you introduce a platform holder to the mix the economics change. Microsoft, Sony, or Nintendo is going to demand their cut of all ongoing revenue, and that cut is rumored to be between 25% and 35%. With one more player getting a piece the revenues shrink for both the publisher and the developer and it becomes harder to turn a profit from a “modest success”.

Barrier #2: Certification

Absolutely everything released on any console goes through an extensive testing phase called certification. This is a slow, expensive process that is imposed by the platform holder to keep a consistent level of quality and a consistent user experience for all titles on their platform. It works too, so certification isn’t likely to go anywhere any time soon.

How does certification interact with the need to put out patches on a regular basis that add new features to the game? It’s bound to slow things down (and make patches more expensive.)

Barrier #3: No Keyboard

Voice chat is great for small groups. It even works pretty well for short messages from one player to another. It really doesn’t work so well for chat groups of 100. All the current consoles can take some kind of keyboard, but requiring one is something your users are going to object to. The game console is in their living room, after all, and they are probably running out of room after the drum set and all those extra Rock Band guitars.

Even if you could guarantee the players have keyboards, text chat is still problematic. People sit pretty far back from their televisions, and even HD displays really aren’t very high-res compared to PC screens.

Barrier #4: Long Development Times

MMOs take four to five years to build. People keep trying to convince themselves that they can do it in three years, but they’re wrong. They are going to schedule everything for three years and then end up slipping by a year or two.

The Xbox launched in November of 2001. The Xbox 360 launched in November of 2005. Playstation 2 launched in November of 2000, and Playstation 3 launched in November of 2006. The last major generation change on the PC was Windows 95, and it’s had a pretty smooth ramp since then. It’s really hard to spend four to five years building one title when your platform is only going to be current for five to six years.

Barrier #5: Consoles Have a Smaller Installed Base

Yes, I said smaller. There are 189 million NVIDIA GPUs installed in PCs, a number which doesn’t count any of the ATI cards out there or any NVIDIA cards older than the 5 series. There are 120 million Playstation 2s, 25 million Xbox 360s, 25 million Wiis, and 20 million Playstation 3s. That’s a total of 190 million consoles. Whatever ATI brings in installed base pushes the PC way over the top.

This entirely discounts the fact that every single game console was purchased to play games and every PC was not. It also discounts all those GeForce 2s and 4s that a PC developer really should use as their min spec.

Barrier #6: Duo Play

Many, many people play MMOs (and other games for that matter) in pairs. I’ve played 6 different MMOs with my wife. Lots of people play with their spouses, siblings, or kids.

As long as you have an appropriate min spec your game is likely to run on the second-tier PCs in the house. But how many people have a second Xbox 360 in their house? Some do, to be sure, but that number is tiny compared to the number of two-computer households.

Console MMOs really need to support split-screen play on a single machine, which adds to the development complexity. On the other hand, split-screen duo play would be fantastic for people who live in the same house and is actually a feature that consoles can offer over PCs.

So We’re Doomed Then?

In the short run, yes. None of these are insurmountable obstacles, but they do make a console MMO more difficult than a PC MMO. There is enough money to be made in console games that future MMO releases there are inevitable. It’s just a question of when they arrive.

Several console MMOs have already launched. The most successful of these by far is Final Fantasy XI on the Playstation 2. Everquest Online Adventures and Phantasy Star Universe (and Phantasy Star Online before it) are two more examples. There are probably more that I’m not coming up with. All of these games have seen some modest success, but none of them are either major console hits or major MMO hits.

To add to those, some new console MMOs are in the works. SOE is working on three PC/Playstation 3 titles, with Free Realms being the first one to come out. PS3 is the loser so far this generation, though, so that may not make much difference to most console gamers. There is a rumor that Nintendo was working on an Animal Crossing MMO, but it’s just a rumor at this point. Microsoft obviously doesn’t have the institutional fortitude to build MMOs; they have canceled Marvel twice. NCsoft also announced a partnership with Sony to bring an NCsoft game out on the PS3, though they aren’t saying what game yet.

Eventually MMOs are going to come to consoles. It’s just going to take them a while to get there, and they will probably never emerge in the same numbers as they do on PCs. Buck up, Dan. We’ll get there some day.

p

Scaling on a Dime at ION

Filed under: Uncategorized — Joe @ 11:07 am

I spent the week at the ION game conference. The first of my two speaking parts was a panel on scaling your development effort.  Darius live-blogged the thing.

April 28, 2008

p

Getting Feedback

Filed under: Day Job, Production, Uncategorized — Joe @ 9:33 pm

Andy Brice recently posted on getting feedback from software customers. With Pirates, our options are similar but somewhat tweaked.

We host our own forums for our user community to hang out on. On most MMOs about 10% of the player base actually uses these, and they self-select into a very hard-core and usually unhappy group. We can use the forums to find out what they’re unhappy about, but they probably don’t represent the actual player base very well. Still, listening to this segment of our community is important.

Click-cancel surveys are another common option. When someone goes to your site to cancel their subscription you ask them why they’ve canceled. SOE isn’t currently set up to run these, so we don’t have that data available, but many games do this kind of survey. This information is useful for finding exit points for players so you can eliminate them.

Recently I’ve started doing something a little different. I show up in game with no warning whatsoever and announce that I’m running an impromptu devchat. I offer to teleport any players who want to attend to an out of the way spot and then spend an hour or so answering their questions. I’ve run four of these so far (with one of our designers helping out on all but one of them.)

The biggest difference between what I hear in these impromptu devchats and what I read on the forums is the tone.  The forums are all about this OMG important issue or that OMG important issue.  The devchats have all been players asking about various new stuff that we might add to the game. (The answer is almost always “That’s a great idea that we want to implement, but we don’t know when we’ll get to it.”)  I think to get more feedback from players I’ll need to actually ask them some questions.

Maybe I’ll have to try that in the next one…

April 27, 2008

p

Coding Vacations

Filed under: Engineering — Joe @ 4:09 pm

A couple of weeks ago I took the week’s vacation from my job as the Producer of Pirates of the Burning Sea to sit at home, in my basement, and write code 12 hours a day for 8 straight days. It was a fantastic experience and I would love to do it again. If you aren’t a programmer that probably sounds crazy to you.

There are two things that made my coding vacation the awesome, relaxing, productive, and fulfilling experience it was. The first is that there is very little drag on writing code on the first few thousand lines of a project. The second is that I haven’t had much of a chance to code at Flying Lab in the past year and a half. Well those and the fact that I genuinely enjoy programming.

When you are at the very beginning of a project you have little to no drag on your efforts. There isn’t a large body of code to keep up and running when you make a new change. Your compile and startup times are incredibly fast. When you have a bug, there are far fewer places it could be. When you’re used to writing in a million-line code base, this is liberating. It’s also very productive, which feels great.

As the Pirates project has gone on, I’ve gradually been moving further from the code.  Way back when it was just me writing all the code (or even just Heidi and me) I had tons of coding tasks, but about the time we added the fourth or fifth programmer the amount of time I could devote to coding during daylight hours dropped to almost nothing. Once we signed with SOE, I picked up all of the management duties for the technical side of that relationship, which made it even worse. I wrote a little code here and there, but it was always late in the evening or on the weekend around all my other duties.

If you’ve been reading my blog for long, it shouldn’t surprise you that I think coding is fun. Ever since we got the TI 99-4/a for Christmas 1983, programming has been a hobby of mine. When I was deciding what to study in college, I really couldn’t imagine a major that didn’t involve tons of programming. It’s not work, it’s entertainment.

I assume that every other creative person who truly loves what they do has a similar attitude. I know plenty of artists who draw, sculpt, or paint on the weekends. Many game designers design card or board games that they never expect anyone else to see just for the fun of it. The writers I know can’t seem to stop writing for local newspapers, online outlets, or former employers. There’s no reason to think programming would be any different.

And I’m not alone.  One of my co-workers is just finishing up a coding vacation of his own. He took a week off from programming video games to program a video game. Good for him, I say.  He’s going to return to work more refreshed and relaxed than if he’d run off to some tropical island and it won’t have cost him a dime.  (Ok, maybe not as relaxed, but close.)

How about you?  Ever take a vacation to do more of what you already do at the office?

p

Greg McAdoo at Startup School

Filed under: Off Topic — Joe @ 7:39 am

This has nothing whatsoever to do with games, but it’s a great talk on what kinds of startups turn into giant successes. Greg McAdoo is a partner at Sequoia Capital.

April 20, 2008

p

An Augmented Reality Primer

Filed under: Augmented Reality — Joe @ 3:33 pm

Lately I’ve developed an intense fascination with Augmented Reality (AR). This doesn’t bode well for my Facebook game, but learning about the current state of AR is taking up quite a lot of my free time. This post is my attempt to distill what I’ve learned so far.

What is Augmented Reality?

Augmented Reality is the combination of artificial and natural sensory input in in such a way that the artificial input “augments” the natural input. That’s the all-encompassing definition anyway; when you hear someone talking about AR they are probably talking about adding computer-generated images to a video feed and showing the result to the user. AR is technically a subset of Mixed Reality owing to its position on the virtuality continuum (image from Wikipedia):

Augmented Reality has been a subject of countless academic research projects since the mid-1990s, but it hasn’t really broken through into very many commercial applications. There has been talk of applications in wiring jets at Boeing, enhancing exhibits in museums, and personal navigation, but it’s not clear that any of those are really being used. In fact, AR seems to be one of those research projects that people work on while they’re in grad school and then abandon for more practical pursuits after graduation. The number of people that have been researching AR consistently for the past ten years is very small.

Augmented Reality has obvious crossovers with Virtual Reality, specifically in display technologies. The most exciting AR applications use similar head-mounted displays to those used in Virtual Reality, and gain tremendous benefit from head tracking. Augmented Reality also uses rendered 3D images, but those are hardly the exclusive domain of VR. In terms of rendering both AR and VR are taking their cues from the game industry at this point.

Computer Vision is also mentioned frequently in AR circles. The biggest problem that AR needs to solve to work really well is to accurately determine the position of the viewer. Many AR researchers use Computer Vision techniques to recognize objects in the scene and work backwards from there to determine the position and orientation of the camera. Much of this research uses markers in the scene itself, including both ARTag and the various flavors of AR Toolkit. The markers make object recognition faster and far more reliable. Here is an example of such markers in action (from the ARTag website):

 

ARTag is GPL’d, so in theory you can download it and start building your own marker-based AR applications. Unfortunately Mark Fiala, the guy in the photo above, is no longer at his university and there seems to be some sort of custody battle going on over who can distribute ARTag. So you can buy his book to learn all about how to use the SDK, but you can’t actually download the SDK. (Incidently, I can’t really recommend that book to an experienced game programmer. It was about 75% Game Programming 101 and had precious few details on how ARTag actually works behind the scenes.)

Approaches to AR in the Current Research

There are three major approaches to AR in the research I’ve been able to find. The first is Magic Lens, where a handheld device of some sort shows the view from its camera with graphics overlayed. The second is Magic Mirror, where a camera is used to show the user an augmented version of himself. I wasn’t able to find a fancy name for the third approach, so I’ll call it First Person. In First Person the user’s view is overlaid with the augmented bits directly. All three of these approaches have appeared in many papers and research projects over the past ten years.

Magic Lens is the approach that seems to be getting the most traction. Today’s PDAs are powerful enough to run marker-detection algorithms and rendering engines in real time, and they all have cameras. By far the most interesting of these projects is Enkin, which is a prototype of a device that promises to offer Magic Lens navigation on Google’s Android. The paper on Enkin’s site outlines many of the very same problems I’m considering in my own nascent AR research. Magic Lens neatly side-steps the biggest problem with First-Person AR by using existing PDA screens instead of expensive and inadequate head mounted displays.

Magic Mirror is an approach that doesn’t seem to be useful for much more than making YouTube videos. The user’s own image is going to dominate the screen almost by definition. It could see some use in trying on hairstyles or clothing without actually trying them on, but I don’t think we’ll see widespread use of Magic Mirror as an AR display metaphor.

The third major AR approach is First Person. The user wears some sort of head-mounted display that tracks the motion of their head through inertial systems, computer vision, or (more likely) a combination of both. From what I’ve been able to find, most AR research using this approach completely obscures the user’s vision and they see only what’s coming through the video camera mounted to their head. Because everything the user sees is on the display, imperfections in head tracking or poor frame rate can cause serious motion sickness. There are a few head mounted displays that are coming on the market soon that allow graphics to painted in the user’s field of view without obscuring their natural vision, so those will help.

Of all these approaches, I think First Person is going to win out in the end. Magic Lens is powerful, but suffers from a need to pay close attention to a little two inch screen on your PDA. That isn’t a big problem in most of the demos, but it precludes using it while driving and eliminates many gaming applications. If you try to move much while your focus is on your PDA you are eventually going to get yourself run over by a bus. We will probably see a rise in Magic Lens AR over the next five years or so and then see a rather sudden shift to First Person once the display technology catches up.

How Far Out is Augmented Reality?

Given sufficient infrastructure to improve location detection in mobile devices, we could have mass-market AR now. Setting up transmitters to precisely determine location and orientation works fine in a laboratory environment. However, it’s unlikely that such infrastructure investment would happen without a strong push from the public, and that’s not going to happen for such an unproven technology. GPS gives AR devices only a very rough idea of where it is, down to 2m in the best cases. For many applications the accuracy needs to be 1cm or less to avoid horrible jittering. The hot new GPS technology coming out in 2013 will still only be accurate down to one meter.

AR markers are used as a solution for this problem, but they aren’t practical in the real world either. We aren’t going to carpet the world with AR markers just so the first few dozen geeks with AR goggles can find their way around without looking at a map. The fact that they can actually show results from their research when they use markers seems to have distracted many AR researches from the need to develop marker-less solutions.

Computer Vision offers an alternative to both markers and fine-grained GPS, and great advances have been made in determining camera position and scene geometry from an arbitrary set of 2D images. These algorithms seem to do a good job of figuring out what they’re looking at, but it’s not clear if they’re running in real time yet. Unlike AR researchers, Computer Vision researchers don’t seem to be into whiz-bang demos. If the problem with the current algorithms really is speed, simply waiting for computing power to catch up will likely resolve this issue.

My entirely unscientific gut feel is that we will see early Magic Lens AR applications hit the market in about two years. Early First Person applications will probably appear around the same time, but won’t be compelling enough to gain widespread use. About 5 years out I expect to see First Person applications of AR begin to take off thanks to increased mobile computing power and improvements in head-mounted displays. I believe mainstream adoption of AR is about ten years away.

On the other hand, I could be totally wrong and Augmented Reality will turn out to be “just around the corner” forever like Artificial Intelligence and Virtual Reality.

April 1, 2008

p

I need a pith helmet

Filed under: Uncategorized — Joe @ 9:48 am

This is what I looked like yesterday:

FLS had a Mustache Month contest in March.   I won the Teddy Roosevelt Memorial Prize.

I’m back to normal today. My wife is relieved.  :)

Next Page »