GDC Talk: Designing for Mobile VR in Dead Secret


It’s been a few months since the 2015 Game Developers Conference was held in San Francisco, but we’ve been so busy with Dead Secret that we barely noticed.  I gave a talk about the game, and how we changed it dramatically to meet the requirements of VR, which the kind folks who run the conference have posted for free.  It’s only 25 minutes, but if you’re short on time then UploadVR has a quick summary.


Posted in dead secret, game design, virtual reality | Comments Off on GDC Talk: Designing for Mobile VR in Dead Secret

Dead Secret at GDC


Hey! We’re going to the Game Developers Conference in March and we’ll be talking about Dead Secret.  The topic is designing for mobile VR, and the work we went through to convert Dead Secret from a tablet game to virtual reality experience.  Here’s the link:

And here’s a sneak preview:

Dead Secret GDC Preview


See you there!

Posted in dead secret, game engineering, game industry, virtual reality | Comments Off on Dead Secret at GDC

Custom Occlusion Culling in Unity


Here at the Robot Invader compound we are hard at work on our new game, a VR murder mystery title called Dead Secret.  There’s a very early trailer to see over at

Dead Secret is designed for VR devices, particularly mobile VR devices like the Gear VR.  But developing for VR on mobile hardware can be a performance challenge.  All the tricks in my last post apply, but the threshold for error is much lower.  Not only must you render the frame twice (once for each eye), but any dip below 60 fps can be felt by the player (and it doesn’t feel good).  Maintaining a solid frame rate is an absolute must for mobile VR.


For Dead Secret, one of the major time costs is draw calls.  The game takes place in the rural home of a recently-deceased recluse, and the map is a tight organization of rooms.  If we were to simply place the camera in a room and render normally, the number of objects that would fall within the frustum would be massive.  Though most would be invisible (z-tested away behind walls and doors), these objects would still account for a huge number of extraneous (and quite expensive) draw calls.  In fact, even though we have not finished populating all of the rooms with items, furniture, and puzzles, a normal render of the house with just culling requires about 1400 draw calls per frame (well, actually, that’s per eye, so more like 2800 per frame).

The thing is, you can only ever see a tiny fraction of those objects at once.  When you are in a room and the doors are closed, you can only see the contents of that room, which usually accounts for about 60 draw calls.  What we need is a way to turn everything you can’t see off, and leave the things around you that you might see turned on.  That is, we want to cull away all of the occluded objects before they are submitted to render.  This is often called occlusion culling.

There are many approaches to solving this problem, but most of them fall within the definition of a Potential Visibility Set system.  A PVS system is a system that knows what you can probably see from any given point in the game, a system that knows the “potentially visible” set of meshes for every possible camera position.  With a PVS system, we should know the exact set of geometry that you might see, and thus must be considered for render, at any given time.  Everything else can just be turned off.

visibility short-2

A rudimentary form of PVS is a Portal System, where you define areas that are connected by passages (“portals”).  When the camera is in one area, you can assume that only that area and the immediately connected areas are potentially visible.  Portals can further be opened and closed, giving you more information about which meshes in your game world are possible to see from your current vantage point.

More complex PVS systems typically cut the world up into segments or regions and then compute the visible set of geometry from each region.  As the camera passes from region to region, some meshes are activated while others are turned off.  As long as you know where your camera is going to be, you can compute a (sometimes very large) data structure defining the potentially visible set of geometry from any point in that space.

The good news is, Unity comes with a pretty high-end PVS system built right in.  It’s based on a third-party tool called Umbra, which by all accounts is a state-of-the-art PVS system (actually, it’s a collection of PVS systems for different use cases).  If you need occlusion culling in your game, this is where you should start.

The bad news is, the interface that Unity exposes to the Umbra tool is fairly cryptic and the results are difficult to control.  It works really well for the simple scenes referenced by the documentation, but it’s pretty hard to customize specifically for the use-case needed by your game.  At least, that’s been my experience.

Dead Secret has a very simple visibility problem to solve.  The house is divided into rooms with doors that close, so at a high level we can just consider it a portal system.  In fact, if all we needed was portals there are some pretty solid-looking tools available on the Asset Store.  Within each room, however, we know exactly where the camera can be, and we’d like to do proper occlusion culling from each vantage point to maximize our draw call savings.  If we’re going to go from 1400 draw calls a frame down to 50 or 60, we’re going to have to only draw the things that you can actually see.

My first attempt at a visibility system for Dead Secret was just a component with a list of meshes.  I hand-authored the list for every room and used an algorithm with simple rules:

  1. When standing in a room, enable only the mesh objects in that room’s visibility set.
  2. When you move to a new room, disable the old room’s visibility set and enable the new room’s visibility set.
  3. While in transit from one room to another, enable both the visibility set of the old room and the new room.

This works fine, and immediately dropped my draw call count by 98%.  But it’s also exceptionally limited: there’s no occlusion culling from different vantage points within the rooms themselves, and the lists have to be manually maintained.  It’s basically just a rather limited portal system.

As we started to add more objects to our rooms this system quickly became untenable.  The second pass, then, was to compute the list of visible geometry automatically from several vantage points within each room, and apply the same algorithm not just between rooms, but between vantage points within rooms as well.  Just as I was thinking about this Matt Rix posted code to access an internal editor-only ray-mesh intersection test function (why isn’t this public API!?), and I jumped on it.  By casting rays out in a sphere from each vantage point, I figured I could probably collect a pretty reasonable set of visible geometry.

Shoot a bunch of rays, find a bunch of mesh, what could go wrong?

Shoot a bunch of rays, find a bunch of mesh, what could go wrong?

Turns out that while this method works, it has some problems.  First, as you might have predicted, it misses small, thin objects that are somewhat far from the camera point.  Even with 26,000 rays (five degree increments, plus a little bit of error to offset between sphere scan lines), the rays diverge enough at their extent that small objects can easily be missed. In addition, this method takes a long time to run through the combinatorial explosion of vantage points and mesh objects–about seven hours in our case.  It could surely be optimized, but what’s the point if it doesn’t work very well?

For my third attempt, I decided to try a method a co-worker of mine came up with ages ago.  Way back in 2006 Alan Kimball, who I worked with at Vicarious Visions, presented a visibility algorithm at GDC based on rendering a scene by coloring each mesh a unique color.  If I remember correctly, Alan’s goal was to implement a pixel-perfect mouse picking algorithm.  He rendered the scene out to a texture using a special shader that colored each mesh a unique solid color, then just sampled the color under the mouse pointer to determine which mesh had been clicked on.  Pretty slick, and quite similar to my current problem.

To turn this approach into a visibility system I implemented a simple panoramic renderer.  To render a panorama, I just instantiate a bunch of cameras, rotate them to form a circle, and adjust their viewport rectangles to form a series of slices.  Then I render all that into a texture.  For the purposes of a visibility system it doesn’t actually matter if the panorama looks good or not, but actually they look pretty nice.

The second bit is to change all of the materials on all of the mesh to something that can render a solid color, and then assign colors to each based on some unique value.  The only trickiness here is that the color value must be unique per mesh, and I ended up setting a shader keyword on every material in the game, which meant that I couldn’t really leverage Unity’s replacement shader system.  This also means that and I must manually clean the materials up when I’m done, and be careful to assign each back to sharedMaterial so that I don’t break dynamic batching.  Unity assumes I don’t know what I am doing and throws a load of warnings about leaking materials (which, of course, there are none).  But it works!

I would actually play a game that looked like this.

I would actually play a game that looked like this.

Once the colorized panorama is rendered to a texture (carefully created with antialiasing and all other blending turned off), it’s a simple matter to walk the pixels and look each new color up in a table of colors-to-mesh.  The system is so precise that it will catch mesh peaking through polygon cracks, so I ended up adding a small pixel threshold (say, ten pixels of the same color) before a mesh can be considered visible.

The output of this function is a highly accurate list of visible geometry that I can plug into the mesh list algorithm described above.  In addition, it runs about 60x faster than the ray cast method (yep, seven minutes instead of seven hours for a complete world compute) before any optimizations.

What I’ve ended up with is an exceptionally simple (at runtime), exceptionally accurate visibility system.  Its main weakness is that it only computes from specific vantage points, but the design of Dead Secret makes that a non-issue.  It doesn’t handle transparent surfaces well (it sees them as opaque occluders), but that’s not an issue for me either.

The result is that Dead Secret is running at a solid 60 fps on the Gear VR hardware.  We have enough headroom to experiment with expensive shaders that we should probably avoid, like mirrors (the better to lurk behind you, my dear).  This performance profile gives us space to stock the house with details, clues, a dead body or two, and maybe even a psycho killer.  Ah, but, I mustn’t spoil it for you.  I’ve already said too much.  Just, uh, keep your eyes peeled for Dead Secret in 2015.


Posted in game engineering, unity | Comments Off on Custom Occlusion Culling in Unity

Performance Optimization for Mobile Devices


This week at the Robot Invader compound we’ve been putting the finishing touches on our new Nanobots game, Dungeon Slots.  This game started out as another week-long experiment and has stretched into a month-long development cycle because we like the concept so much.  The game itself is finished, and we spent this week working on polish and performance.


Some engineers treat performance optimization as something of a black art.  Folks are especially cautious on Android, where there are a wide variety of devices and the performance characteristics of a given device are not always obvious.  We’ve found, however, that despite large differences in the philosophical design of various mobile GPUs, there are a few simple rules we can follow that keep us running well on pretty much everything.  Here’s the checklist we follow when designing our scenes for performance:

  1. Fill is your enemy.  Every time you write to a pixel you incur a cost.  Filling the screen of pixels, even just a solid color, is an expensive operation on just about every mobile chipset available.  Even as mobile GPUs get better at this, screen resolutions seem to increase at exactly the same rate.  Our #1 source of performance slowdown is pixel overdraw–writing to too many pixels more than one time in the frame.
  2. Draw calls are expensive.  Every time you tell OpenGL ES to draw a buffer of verts, that call itself has a cost.  Actually, on most devices I think it is the state switch involved in selecting the verts that you wish to draw that incurs the real cost; if you were to draw the same buffer multiple times, the first draw call would be more expensive than the subsequent calls.  But generally speaking, we try to keep the number of draw calls as low as possible.  In Wind-up Knight 2 we have about 100 – 120 per frame.  Dungeon Slots is less than 40 per frame.
  3. Lights are expensive. Depending on how you’ve implemented your lighting, realtime lights can destroy your performance on a mobile device.  Lights often require multiple sets of geometry to be submitted to the GPU, or multiple passes over the pixels being lit, or more high-precision registers in a shader than your GPU has available.  The actual costs come down to the individual implementation, but there are a number of ways lights can eat into your perf.
  4. Watch out for vertex creep. Many mobile devices are actually pretty good at handling scenes with lots of verts.  But most GPUs fall down really hard after you pass a certain threshold of geometry per frame.  In order to run on lower-end hardware, we target 30k triangles per frame as a soft upper limit.  This might be a little conservative, but remember that some types of lights can increase your triangle count!

There are a few other rules of thumb, but they are less important: the rules above cover 95% of cases of poor performance.  And of those, I’d say that fill-related slowdown accounts for the vast majority of cases.


This scene is about 100 draw calls.

Our strategy for dealing with these problems also boil down to a few rules:

  1. Macrotexture everything. Macrotexturing is the process of using the smallest number of textures possible in the scene.  The levels in Wind-up Knight 2 all fit into 4 1024×1024 textures.  This is fast for a number of reasons, but one of the main benefits is that it allows us to batch all of the visible geometry using the same texture up into a single VBO and send it to the GPU all at once.  Unity does a good job of this automatically with its dynamic batching option.  Macrotexturing is hard, and it requires an artist with a lot of foresight, serious modeling skills, and a willingness to rework things to accommodate changes in the textures.  But it’s absolutely worth it.
  2. Batch everything. In addition to dynamic batching based on material, we also try to combine meshes that we know won’t move.  Unity calls this static batching, and it’s great for level geometry or other mesh elements that never move.  Rather than making our scene static in the editor, we usually mark all objects that can be made static with a particular layer, then use Unity’s StaticBatchingUtility to combine static meshes at level load time.  This increases load time a bit but dramatically reduces the size of our game binary.
  3. Control draw order.  On a PC, you probably draw your scene from back to front, starting with a skybox and ending with the closest bits to the camera, followed by a pass for transparent objects or other items needing blending.   On mobile, however, this incurs an unacceptable amount of overdraw.  So we try to draw as much as possible front-to-back, with the skybox and other large objects that can potentially touch a large number of pixels on the screen drawn as the last step before transparent objects.  Rejecting a pixel with a depth test is much faster than filling that pixel unnecessarily several times, so front-to-back for opaque geometry is a big win.
  4. Watch out for transparency.  Transparency is, by definition, the process of filling a pixel more than one time.  Therefore, on mobile, it’s very expensive to have large objects that covers part of the screen in semi-transparent pixels.  Even worse is layers of transparency.  You can get away with small small regions of the screen, but once a transparent object starts to touch a lot of pixels, the frame time cost will be high.  We try to organize our transparent objects such that there is minimal overlap and that they take up as few pixels on the screen as possible.
  5. Design to scale.  It’s hard to find a perfect balance between “looks good” and “runs fast” on mobile, mostly because there’s such a wide spectrum of power out there.  A modern device like the Nexus 5 or iPhone 5 can push scenes that are orders of magnitude more complex than their predecessors from three or four years ago.  Therefore, we design our scene such that we can tone down the graphics quality in exchange for performance on lower-end displays.  We drop the highest texture mip on displays smaller than iPhone 4 resolution.  We down-res the size of the final render target by 15% or 25% on very slow devices. We dynamically detect framerate changes and switch between pixel lights and spherical harmonics on the fly.  These are easy to do if you are thinking about them early.
Dungeon Slots

Dungeon Slots!

With those rules of thumb in mind, here’s how we optimized Dungeon Slots this week.

At the beginning of the week, Dungeon Slots ran great on a Nexus 5 and absolutely terribly on a 2012 Nexus 7.  Now, the Nexus 7 is a few years old, but it’s still quite a bit more powerful than what we’d generally consider to be our minimum spec.  The game was running at less than 15 fps on that device, and we needed to find out why.

The first thing I did was connect the Unity profiler to the device and look at the logs.  The profiler is a bit flakey, especially in situations where the CPU is hosed, but we could see that some of our GUI code (managed by the NGUI framework) was spiking every frame.  I looked at the scene we were rendering and noticed that it had been constructed out of a bunch of tiny sprites.  NGUI does a good job of maintaining a single texture atlas for those sprites, and it collects them all into a single draw call every frame.  But it also has to regenerate the verts for that draw call if anything in the scene (well, in NGUI terms, within the parent panel) changes.  This game has a number of rotating slot-machine-like slots, both for the slot machine itself and for various numerical displays, and those were implemented with a bunch of sprites that were clipped into a small window.  The main source of overhead, according to the profiler, was just updating the positions and clipping rectangles for all of those sprites every frame.  The clip regions are pretty expensive, too.

We replaced the numerical displays with a system based on a scrolling texture, which increased our draw call count slightly but dramatically reduced the number of sprites that NGUI needed to manage.  We also reorganized our NGUI panels such that bits of the scene that are static were separated from the bits that were animated to avoid unnecessary vertex buffer recreation.  This change caused NGUI to drop a number of large notches in the profiler, and while it’s still a little more expensive than it should be, it’s no longer the focus of our attention.

Even with that change, however, the game was still running very slowly on the Nexus 7.  The next step was to enable Unity’s internal profiler log and take a look at the output.  That output looks something like this:

cpu-player> min: 102.8 max: 132.7 avg: 117.5
cpu-ogles-drv> min: 0.9 max: 3.0 avg: 1.5
cpu-present> min: 0.0 max: 1.0 avg: 0.1
frametime> min: 103.8 max: 135.0 avg: 119.1
draw-call #> min: 44 max: 44 avg: 44 | batched: 2
tris #> min: 82126 max: 122130 avg: 122126 | batched: 64
verts #> min: 83997 max: 124005 avg: 123998 | batched: 50
player-detail> physx: 1.1 animation: 0.8 culling 0.0 skinning: 0.0
               batching: 0.1 render: 8.1 
               fixed-update-count: 5 .. 7
mono-scripts> update: 3.1 fixedUpdate: 0.0 coroutines: 0.0 
mono-memory> used heap: 1900544 allocated heap: 2052096 
             max number of collections: 0 
             collection total duration: 0.0

What this told us was that the CPU was still hosed, but not by mono scripts.  The incredibly high cpu-player time indicated that a lot of work was going on before the GPU even got any verts to draw.  The OMGWTFBBQ moment came when we noticed that the vertex and triangle count were averaging in the ~100k tris / frame, way over our target of 30k.

Switching back to the Unity editor, the Stats overlay window told the same story: our simple scene was pushing way more polygons than we expected. After some investigation we realized that while the meshes in the scene itself were right on target in terms of complexity, we’d started using a standard diffuse shader on them in order to achieve certain lighting effects.  Unity’s Mobile Diffuse shader only supports one directional light, but the stock Diffuse shader supports any number of pixel lights.  What was happening here is that our geometry was being submitted many times over, once for each light source that touched it, which caused our triangle count to skyrocket and our CPU to collapse. I modified the setup to use only the faster Mobile Diffuse shader.  This fixed our crazy triangle load but removed the neat lighting effects in the process.

Still, it was probably worth it: the game had gone from about 10 fps when we started to about 22 fps via these changes.  That’s a savings of about 35 ms per frame, which is pretty significant.  Still, 22 fps remains way too slow.

To delve deeper into where our frame time was going, I decided to bust out the big guns.  NVIDIA produces a neat performance tool called PerfHUD ES, which allows you to connect to an Android-based dev kit and get detailed profiling information about the scene you are rendering.  I have an ancient Tegra 2 dev kit that I got from NVIDIA years ago, and it’s fantastic for this kind of performance testing precisely because it’s pretty slow by modern standards.  Getting it to work requires a little dance of shell scripts, adb port forwarding, and prayers to various moon gods.  The process has been much improved by NVIDIA in more recent kits, but I like the old one because its performance characteristics are so easy to understand.

Dungeon Slots particles

A shot from NVIDIA’s PerfHUD ES showing that our transparent particle fog touches a lot more pixels than we intended.

The best thing about PerfHUD is that it can show you a step-by-step rendering of how your scene is put together by the hardware, draw call by draw call.  This tool, combined with timing information about each draw call, is usually more than enough to identify performance culprits.  When I ran Dungeon Slots through PerfHUD’s frame analyzer, I learned two important things:

  1. Though the UI completely covers the bottom half of the screen, we were rending the 3D world underneath it.  That’s overdrawing 50% of the pixels on the screen!
  2. A transparent particle effect we place on the ground in front of the camera was actually much larger than we anticipated, and most of it was hidden behind the UI.  More overdraw!

Once identified, these are easy problems to solve.  The first step was to just reduce the size of the 3D camera’s viewport to cover only half the screen.  This way the bottom half has no overdraw from frame to frame.  That also cut the size of the particle effect in half.  Even so, subsequent profiling showed that the particle effect was still touching too many pixels to be performant on an older device with a big screen like the Tegra 2 dev kit.  It needed to be turned off entirely. With these changes, the game now runs at 60 fps on the Nexus 7, and at a very respectable 30 fps on the ancient dev kit.  We lost a few graphical effects in the process (some animated lighting and a particle effect), but overall the game still looks good, and now will run well on devices we consider to be our minimum spec.

Dungeon Slots 2

Still, it’s tough to play the game without those extra effects now that we’ve gotten used to them being there.  A player who’s never seen them before won’t miss them, but there’s no denying that the game looks more dynamic, more interesting, and more polished with all the extras turned on.  And after all, the Nexus 5 ran Dungeon Slots at full speed even before we started with all of this optimization.  It sucks that folks with high-end devices get a degraded experience simply because there are also lots of low-end devices out there.

But maybe they don’t have to after all.  The last change I made this week was to add code that samples the framerate as the player plays their first round of monster-slashing slot madness.  If the device is performing well, over 50 fps, I go ahead and turn the particle effects back on and change the shaders back to Diffuse for full lighting.  In my tests, this produces a good middle ground: the game runs fast for everybody, and high-end devices get the extra graphical whiz-bang polish features as well.

We’ve still got a little bit of work to do before Dungeon Slots is ready to go, but you should be able to play it soon.

Posted in Android, game engineering, mobile games | 3 Comments

Android TV and the Video Game Middle Ground


We shipped Wind-up Knight 2 for Android TV last week.  If you were at Google IO, maybe you saw our logo flash up there for a moment.  If you actually have an Android TV, Wind-up Knight 2 is one of a small set of games that is already setup for it.  Playing with a controller is sublime, and it’s absolutely beautiful at 1080p on a giant screen.

Wind-up Knight 2 is pretty flippin' sweet on a TV.

Wind-up Knight 2 is pretty flippin’ sweet on a TV.

Two weeks ago we also shipped for another TV-based device, Amazon’s FireTV.  Both of these devices required almost no effort on our part; if your game has controller support built in, shipping an Android game on a TV is trivial.

Of course, not every game has built-in support for controllers.  Adding controller support to user interface screens can be particularly challenging for games designed around touches and swipes.  But in this day and age, I think that every mobile developer should be thinking about how to integrate controllers into their games.  Our approach is to do almost all of our development on the Nvidia SHIELD.  It’s a fantastic device for developing controller-based games because it’s rugged, highly standards-compliant, and very fast.  Once it works on the SHIELD, it’ll work everywhere else.

A lot of developers are quick to write these devices off.  They find the idea that cheap set-top boxes based on mobile chipsets might “kill consoles” incredible.  But I don’t think we should cast Android TV and similar devices as simple competitors to dedicated gaming devices.  As I told Gamasutra last year, I think commoditization of game-playing hardware is an eventuality that cannot be avoided, and that’s going to present some tough challenges for traditional consoles.  My interest has less to do with who wins and who loses and more to do with expanding the range of markets available to game developers.  The more choice developers have about how much money to spend, what kind of game to create, which sort of user to target, how much to charge, and which distribution channel to use, the healthier the industry will be.  And a healthy game industry makes more interesting games, takes more creative risks, and reaches more people.

Nobody bought an iPhone to play video games at first; it stealthily worked its way into pockets around the world before blossoming into a huge market for games.  I think Android TV, and similar devices, can do the same for games running on TVs and played with controllers.  In doing so, there is an opportunity to create a whole new channel for games that cannot exist elsewhere; a middle ground between consoles and mobile devices.

As I get older, I’m finding it harder to play games.  I have seven different consoles connected to my TV at the moment, and yet I haven’t felt the need to go out and buy a PS4 or Xbox One.  Not because those devices are poor, just because I appear to have aged out of the target demographic.  The games available on those devices all look rad, and there are a couple that have really caught my eye (Access Games’ D4 and Frictional’s SOMA are on the top of my list), but the catalog doesn’t interest me enough to actually go buy a new $399.99 device.  At the same time, it’s hard for me to find games on mobile platforms too.  This might be a discoverability issue, or it might be that the games I like are not well suited to free-to-play or touch controls.  Almost 100% of the games I play these days are on mobile devices, but there are very few that really hook me.  I’d buy a Steambox but they aren’t really ready for sale yet, and aren’t likely to be cheap.

The promise of devices like Android TV is not to replace consoles, nor to reach the insane scale of mobile games.  Rather, they present an opportunity to become a channel for games that can’t be played with a touchscreen but aren’t big, expensive console games either.  Experimental games, retro games, narrative-heavy games.  Games that can be built and shipped cheaply thanks to standardized chipsets and digital distribution infrastructure, but can still be run on a big screen and played with a controller.  Games like Neverending Nightmares, which is shipping for OUYA first and feels perfectly at home on that device.

I don’t know if devices like Android TV will actually blossom into an alternative channel for games, but the promise is there.   And that’s more than enough reason to support them with Wind-up Knight 2 and future titles.

Posted in Android, controllers, game industry, mobile games, wind-up knight | 4 Comments

Nanobots: Making Games in a Week


When Casey and I founded Robot Invader back in 2011, one of our goals was to build a team that could produce high quality games in a short amount of time with just a few people.  We produced Wind-up Knight with just three people (and an intern!) in about five months, which was a pretty good start.  Our second game, Rise of the Blobs, took us longer, but we didn’t just build a rocking puzzle game: we built a bunch of core tech and tools, not to mention our own network backend that now powers all of our games.  Wind-up Knight 2 took four of us six months to ship, and it’s significantly more complicated than its predecessor.  Of all our accomplishments thus far, I am most proud of our repeated ability to build high-end 3D games with a very small group on a very strict timetable.


Brimstone Deep. You’ll have a chance to burn in this flame soon.

Over the course of the last three years we’ve experimented with different ways of increasing our bang-for-the-FTE.  After finishing Rise, we split our team of four into two teams of two and spent three months building prototypes.  One of these microteams produced the game that eventually became Wind-up Knight 2.  The other, a first-person murder mystery game that we’ve teased a few times.  Both made it to a playable prototype phase, and after looking at them we decided to build Wind-up Knight 2 first.  Considering the short development period and the small team size, we got quite a lot done.

This month, as we’ve waited for various moons to come into alignment so that we can release the new Brimstone Deep world for Wind-up Knight 2, we’ve been experimenting with an even stricter goal: build a game to completion in one week.  It can be a simple game, but it has to be fun, and it has to be ready to ship in one week.  The actual mechanics of shipping a game (collecting screen shots, writing marketing copy, testing, integrating analytics APIs and the like) might take a few days more.  That’s ok.  But the game itself must be completely finished in the time it takes to kill somebody with a Japanese curse.


We came a long way in three months, but there’s lots of work left on this one.

What’s the point of making a game in a week?  Like many time-limited game jams, the main idea is to hone our ability to assess and control scope, and to experiment with ideas.  Putting development on such a short timeline forces us to cut any game idea down its bare bones.  We don’t have time for complex features, there’s no schedule for difficult engineering.  The core game idea has to be fun immediately.

We’re calling these one-week games Nanobots.  The first game in the Nanobots series is called Tapthello!, and we shipped it last week for Android (the iOS version is waiting for Apple’s review).  The second Nanobots game, also developed in one week, is called Cybergon, and we’ll release it in a few days it’s available now!  We’re now in our third week of development, working on our third game.  We don’t even know what it’s called yet but it’ll be out soon!

Cybergon1 Cybergon2

So far, the experience has been hectic and fun.  For every problem we have to ask ourselves, “how do we solve this immediately?”  If there’s no immediate solution, we might just have to kill the idea and move to something else.  It’s stressful, but also a bit liberating, to be able to make decisions so quickly.  And with each title, we seem to be increasing the scope ever so slightly as our confidence grows.

We’ll have more info on these new bite-sized games as we release them.  And stay tuned for the new Brimstone Deep update to Wind-up Knight 2, which is coming very very soon!



Posted in Uncategorized | Comments Off on Nanobots: Making Games in a Week

Learning about Free-to-Play from Devil May Cry


The first 20 minutes of Devil May Cry are really clever.  What happens is this: you breeze through several rooms, chopping evil marionettes left and right and flicking that silver hair around like you own the world.  After the second or third room the game offers you the option to switch to “Easy Automatic Mode,” in which the game becomes easier.  It’s pretty easy at that point, so you probably ignore the offer.  But a few minutes later Devil May Cry throws you at the first boss (a giant lava spider) and you die.  I mean, you die.  You get your butt handed to you on a platter.  The giant lava spider is way, way tougher than anything you’ve seen in the game so far.  It’s not even clear how to damage the thing at first.  You get beat down, hard.  You try again, but it’s futile. The lava spider is too difficult.  At this point you have four options:

It's hard to find a better arrogantly androgynous fashionista super hero.

It’s hard to find a better arrogantly androgynous fashionista super hero.

  1. Quit playing,
  2. Turn on Easy Automatic Mode and try again,
  3. Step up and become much more skilled to beat the boss, or
  4. Figure out that you can go back and keep killing marionettes to grind for items which you can then use in a shop to power yourself up to make the boss more reasonable.

This is a genius little bit of manipulation because what the game is really doing is forcing you to identify, early on, the kind of experience you want.  If you quit, well, you probably do not have the patience required to enjoy the rest of this game.  If you turn on Easy Automatic Mode, you’ve identified yourself as somebody who wants to play but isn’t interested in a pride challenge.  If you skill up, you’re a hardcore mofo and the game should throw everything it’s got at you.  The last option, the one that I think most players choose, is to go back and grind.

But grinding serves a purpose.  It forces you to learn about second- and third-order game systems, like controlling your combos to increase the number of style points you get.  It is here that you unlock the real meat of the game design: you’re not just supposed to button mash through dudes, you’re supposed to control your combos to destroy the opposition as stylishly as possible.  Doing so increases your style score, which increases the payout of each felled enemy, which lets you upgrade to become even more badass.  This is a much more interesting (and difficult) challenge, and adds a lot of depth to the moment-to-moment gameplay.  Devil May Cry uses its first boss as a forcing function to figure out what kind of player you are, and it compels you to unwittingly select a play style that you prefer.

Why does it do this?  Because the rest of the game is even harder than that lava spider boss, and becoming skilled enough to make progress requires that you be really invested in the core game mechanic.  Button mashing isn’t enough to sustain most players’ interest through the incredibly demanding difficulty curve that Devil May Cry employs, but by tricking you into learning about the extra depth in the combo system, the game is actually ramping you up for a more interesting experience.

Easy Automatic Mode pop-up in Devil May Cry 3.

Easy Automatic Mode pop-up in Devil May Cry 3.

It’s really quite manipulative if you think about it.  The system is designed to carry the maximum number of people through the level content, and hopefully make them feel good about it.  It’s a carefully constructed trick, aimed right at the psychology of the player who feels that he has something to prove.  Going a step further, we might even see the offer to switch to Easy Automatic Mode as a challenge: “hey, if you can’t take the heat, there’s always training-wheels mode over here.”  Players who aren’t putting their pride on the line will see Easy Automatic Mode as a welcome way to reduce frustration.  Players who want bragging rights for their skills see it as a personal insult.  The game uses this form of manipulation to ensure that both types of players are put on a path to having a good time.

This is the type of design that a good free-to-play game should pursue.  One of the many complaints about free-to-play games is that the addition of purchases leads to manipulative game design.  But manipulative design is not limited to attempting to extract money from wallets.  As with Devil May Cry, it can be used to improve the game experience, and widen access to players with different play preferences.  If you want to get technical about it, contemporary free-to-play experts would probably call Devil May Cry’s lava spider a retention system.  The lingo might make you feel yucky, but the concept is sound: it’s a way to get more people to give the game a second or third chance, to identify what kind of game they want to play, and prepare them for the dramatically deeper content ahead.

DMCs core loop supports different play styles, but it identifies which you prefer early.

DMCs core loop supports different play styles, but it identifies which you prefer early.

There’s no question that the addition of real-money purchases changes the emotional tone of the game for some players.  There’s no question that many of the mechanisms used by free games to make money could be considered manipulative.  Certainly there are games that are designed without respect for their audience, but what of games that manipulate out of respect?  But what if the purpose of that manipulation was to improve the fun factor?  What if the purpose was to force the player to choose between a skill-based challenge or content tourism?  What if we could use the emotional impact of a real-money purchase to manipulate some players into enjoying the game more?  Traditional games do this all the time; should free-to-play games be any different?

We think that in-app purchases can be put to better use than just fishing for a small fraction of the audience that likes to spend big.  We think they can be used as a way to increase the quality of the game, both for those who purchase things and for those who do not.  In the next post I’ll write about how we’re trying to do that with Wind-up Knight 2.

Posted in game design, mobile games, wind-up knight | Comments Off on Learning about Free-to-Play from Devil May Cry

Playing Wind-up Knight 2 with a Guitar (and why it matters)


Yesterday we posted this silly video about playing Wind-up Knight 2 with a variety of funny controllers.

Nothing in this video is faked; we used a converter box (like this one) to convert our various console controllers to USB, then a OTG USB-to-Micro USB adapter (like this one) to connect the box to an Android tablet (in this case, a Sony Xperia Z, though this should work with any modern Android device).  Most standard controllers will just work, but this collection of weirdo devices requires a little bit of button mapping (usually just two keys), which we have a simple setup screen for.

We had fun making this video, but it also forced us to ensure that our controller support was complete.  If you can play Wind-up Knight 2 with a Guitar Hero controller, I’m pretty confident that you’ll be able to play with any standard mobile controller that might be released in the future.  And because we had this infrastructure in place for Android devices, adding support for iOS7 controllers took us about five minutes.

Controller Madness

Controller support is important to us because there are entire genres of game that are difficult to translate from fingers-on-a-button to fingers-on-glass.  We’ve worked hard to make Wind-up Knight have tight, reliable touch screen controls, and there are plenty of games on mobile devices today that could never be made with a traditional controller.  But there’s also 30-odd years of game design behind sticks, d-pads, and buttons, not to mention a generation or two of people who know how to play games this way.  Our interest isn’t in competing with console games–we love our PlayStations and Xboxes.  Rather, we want to leverage the extreme ease of development and distribution that mobile platforms enjoy to make crazy controller-based games.

We fondly remember the short-lived Dreamcast golden era, when all kinds of wacky, innovative titles were readily available, when the cost of development was still low enough that experimentation was abundant.  These days, if you want to play a jaw-droppingly beautiful FPS, you should probably do it on a console.  If you want to kill five minutes while you wait in line for coffee, your phone’s got you covered.  But what if you want to play something like Power Stone 2, or Jet Grind Radio, or Cannon Spike?  Some games from that era have actually been released for mobile platforms and while they look great, they are neigh-unplayable.

I believe that controller-based games distributed as apps to devices based on mobile hardware could give us access to that sort of golden era again.  I don’t know if controller-based mobile games will gain enough traction to become a big deal, but I sure hope that they do.  Whether it’s controller peripherals for phones and tablets, TV-based devices with mobile hardware like the OUYA, or dedicated portable gaming devices like the SHIELD, we want to be prepared to deliver the best possible play experience.  And after playing Wind-up Knight 2 with my old Dreamcast controller, I am happy to report that it feels pretty rad.

Posted in Android, controllers, game design, game engineering, iOS, mobile games, wind-up knight | Comments Off on Playing Wind-up Knight 2 with a Guitar (and why it matters)

Checkpoints in Unity


One of the new features in Wind-up Knight 2 is checkpoints.  They work the way you  expect: if you die after passing a checkpoint you will restart at the checkpoint location with the game world restored to its previous state.  This sort of system is easy to implement if you plan for it from the beginning of development, but it can be tricky to retrofit to an existing game engine because entities tend to change their state subtly as the game is played.  What animation frame was the character on when the checkpoint was hit?  What was his velocity?  His collision contacts?  If you didn’t plan on saving this sort of state when you wrote your entity behaviors, adding it in later could represent a pretty major code change.


We didn’t have checkpoints in the original Wind-up Knight, and when we decided to add them for the sequel I realized we were looking at a potentially massive change to the codebase.  Our requirements for checkpoints were as follows:

  • Flexible.  Need to be able to serialize all the entities in levels we already built.
  • Instant.  No hitching when saving the checkpoint, and restarting after a death should be instantaneous.
  • Repeatable.  We only store one checkpoint at a time, but a level can have any number of checkpoints in them.
  • Efficient.  We can’t use too much runtime memory.

At first, my main concern was Flexibility.  We have a lot of entities that change their state in a lot of different ways.  Coins that are collected.  Enemies that animate, attack, and turn around.  Rocks that fall only once.  Gates that open and close.  I wanted to make a system that would deal with all of those things without having to make code modifications to each.

My first attempt was to use C# reflection.  I figured that if I walked the object tree and recorded all of the public fields and properties, I could restore those values on checkpoint reset and be 90% done.  Turns out that this isn’t so hard to do in C#, and I was able to knock out a prototype in a couple of hours.  But immediately there were problems: the number of fields to serialize was so large that there was a visible framerate hitch, and properties on many Unity objects have side-effects when set (e.g. rigidbody asserts if you try to write to velocity while isKinematic is true), making deserialization difficult.  I also noticed that most of the data getting serialized was not really information needed for a checkpoint restore.  Most fields never change.

This lead me to try the opposite extreme.  I would serialize only specific objects, and only a few parameters of those objects, and anything that wasn’t sufficiently covered would require custom code.  For this second attempt I decided only to record the following information:

  • Object transform
  • Whether or not the object was destroyed since the last checkpoint

That’s it.  Any other state would have to be dealt with on a per-script basis.  I figured that this method would get me about half way there and then I’d spend a lot of time modifying entity code to deal with state storage and loads.  Sort of a lame solution, I thought, but one that probably would meet all of the requirements above.

I implemented this in three parts:

  1. CheckpointRegistry, a singleton that maintains a record of all objects that are tracked for checkpointing.
  2. CheckpointAware, a script that marks an object (and all of its children) for tracking, and
  3. CheckpointBehavior, a child of MonoBehaviour that adds two virtual methods for dealing with checkpoint saves and loads.

CheckpointRegistry contains a HashSet of objects that have been marked for tracking.  When a new object is registered, all of its children are automatically registered as well, and top-level objects are called when save or restore events occur.  The Registry also provides methods for destroying objects; CheckpointRegistry.Destroy() makes an object inactive and adds it to another set, to be reactivated on the next checkpoint restore or actually deleted on the next checkpoint save.

The real work occurs in CheckpointAware.  This script, which is dropped on any object that should save its state when a checkpoint is hit, records the transforms of itself and all of its children.  CheckpointAware adds its object to the CheckpointRegistry when it is allocated, and waits to be told to save or restore its state.  When a call from the Registry comes in, CheckpointAware reads or writes all of the transforms in its subgraph and calls the appropriate method on any CheckpointBehaviors therein.

Bask in the warmth of the checkpoint's light.

Bask in the warmth of the checkpoint’s light.

To sum up the algorithm: CheckpointAware stores position, scale, and rotation of subsets of the hierarchy.  CheckpointRegistry stores references to all of the objects in all of those subsets, and manages object destruction from within those subsets.  A checkpoint is saved by calling CheckpointRegistry.SetCheckpoint(), which calls a similar method on each CheckpointAware instance.  CheckpointAware records the transform of itself and its children to a simple struct and calls CheckpointBehavior.SaveState() on any children with CheckpointBehavior-derived components.  Restoring the checkpoint is the same, but reversed: CheckpointAware walks its list of stored transforms and writes the cached values back into them, and then calls CheckpointBehavior.RestoreState() on any children that need it.  Whew.

In any given Wind-up Knight level there are thousands of objects that need to be serialized for checkpoints. Fortunately this method is very fast; there’s no visible hitch on saving or restoring the state, even on fairly low-end devices.  It’s also pretty efficient, as only the minimum amount of data required to restore a checkpoint is saved.  So there’s two of our four requirements right off the bat.  It’s also easy to manage multiple checkpoints with this system, so we can mark off Repeatable as well.

Which leaves us with one last requirement: Flexibility.  The weakness of this approach is that it’s only storing positional and lifetime information; nothing about entity state is recorded by default.  I thought it was only going to get me half way.  But once I had it up and running, I found that it is almost a complete solution.  For Wind-up Knight, storing position and lifetime alone proved to be about 90% of the final solution.

I did end up having to go through a bit of entity code to save and restore internal variables.  But even that turned out to be simple; most objects just need to record an int or a bool, and can cache them right in the script instance.  I spent a day converting existing scripts to CheckpointBehaviors and adding SaveState() and RestoreState() implementations, usually only three or four lines of code each, and then I was done.

Retrofitting a mass of existing code to perform new tricks isn’t fun (even when it’s simple and easy), but the payoff was worth it.  Now dying in Wind-up Knight 2 (which happens a lot) hardly slows you down; you’re sent back a bit and get to try again without missing a beat.  It’s a huge improvement in the general flow of the game compared to its predecessor (which required a full scene reload on every death–ugh).  The architecture of this checkpoint system occupies a nice middle ground between general purpose serialization and script-specific logic, and I’m glad (and a little surprised) that the design worked out so well.

Oh, and if you’re one of the hardcore group of folks that wants to play the game without checkpoints, never fear: you can turn them off.


Posted in game engineering, wind-up knight | 3 Comments

Leveling Up the Wind-up Knight Design Process


When we built the original Wind-up Knight, Robot Invader was only three people: Casey, our intrepid designer / animator, Mike, our one-man art machine, and me, the code guy.  We hired contractors (the mighty Jordan Fehr for sound design and intrepid Josh Whelchel for music) and an intern (Patrick Wagner, now at Arachnid Games), and went from founding the company in April of 2011 to shipping in October of the same year.  We managed this by leveraging Unity (though none of us had any prior experience with it) and by staging our releases (the iOS version came out slightly after the Android version).  Subtracting about a month of time spent buying computers and getting office space and generally spinning up the company, Wind-up Knight was developed from scratch in about five months.

Of those five months, two were spent prototyping.  We’d build out a game idea in a day or two, play with it, make improvements, and then move on to something else.  Almost nothing from this stage survived in the final game; if an idea didn’t work, or just wasn’t fun enough, we chucked it aside and tried something else.  Wind-up Knight went through a number of crazy iterations before it settled on the platformer that we eventually shipped.

WUK1 Alpha Shot

A very early shot of the original Wind-up Knight.

Two months of prototyping and three months of production work to make a platformer with over 50 levels in it, each with a completely unique look and feel, was a lot of work.  We do our level building in “blue-box rooms,” levels that have blue collision regions but no real art.  The levels are designed, tested, and polished in this state, then “skinned” with art as the very last step.  The skinning process is a bit like building unique structures with odd-shaped legos: we snap together bits of geometry provided by Mike to give every level a unique look while sharing lots of texture and mesh between levels.  But the hardest part is the actual blue-box phase, the creation of the core level geometry.

Casey, who designed every level in Wind-up Knight, spent pretty much all of August of that year in the office, working his fingers to the bone.  At the end of it he had produced 52 awesome levels.  He had also developed a level design workflow that included specific metrics for jump distances, spike placement locations, and wall slide heights.  Casey came out of that month of hell with a head full of rules about how to make a fun Wind-up Knight level.

Fast-foward to 2013 and Robot Invader is a little different.  We’re four people now (the fourth is Jonny, our seasoned design director), and have shipped a couple of games.  This year we split our team of four into teams and developed two games in parallel, one of which turned out to be (after a number of experiments and prototypes) Wind-up Knight 2.

WUK2 Blue Box

A shot of an unused Wind-up Knight 2 level at the “blue box” prototype stage of development.

WUK2’s development has been very different than its predecessor.  Rulebook in hand and a complete game at their disposal, Jonny and Casey spent several months developing levels for a new Wind-up Knight game.  The prototyping phase was much more focused, as the core mechanics were already established and implemented as production-quality code.  Furthermore, these levels were designed without the crazy time pressure of the first game; Casey and Jonny have been able to experiment with lots of ideas and then keep only the very best.  They produced almost 100 levels, then cherry picked the most fun of the lot.  When Mike and I joined their team to work on WUK2 full time, the game was already playable from beginning to end.

In total, Wind-up Knight 2 will probably take slightly longer than its predecessor for us to complete.  But the manpower is being spent in very different ways; this time around there’s a lot more space for pushing the content envelope and polishing the design.  We’re still working hard to get everything finished by the time we ship, but the core game has been rock solid for months.  It’s a design process that feels very different, probably one that only works when building upon an established game.  It feels good.



Posted in game design, prototyping, wind-up knight | 3 Comments