Update: Download a slice of Republique from the Asset Store!
A subset of the full Republique game is now available for free from the Asset Store. More than just a demo, this “mini” version of Republique will show you up close how Camouflaj used Unity 5 to remaster Republique. Read this blog post to learn more.
Remastering Republique:The Journey to Unity 5
Seattle-based studio Camouflaj wanted to remaster their mobile stealth action hit République for the PC. The game was initially made with Unity 4 for iOS and Android, and Camouflaj decided to use Unity 5 to take République to the next level for the new PC version on Steam.
They came to us with an idea we loved: Camouflaj would get early access to Unity 5 alpha and beta builds, and then document and share their journey of remastering République from Unity 4 to Unity 5 to deliver a whole new experience to gamers.
République enters the Next Gen
Why we are using Unity 5 to take République to PC.
Podcast: Episode #1Download (113MB)
Dev diary #1
République enters the Next Gen
Working in Unity has been a big part of our story since we began developing République back in 2011. Since then, Unity’s features have evolved, the Asset Store has swelled, and we really feel like we’ve been been riding the wave of the technology’s success.
When Unity 5 was announced we saw our chance to make good on our two-year old promise to make a PC and Mac version of Republique. In addition to spending months completely reworking the game’s controls and UI, we knew we’d benefit from an increased wow factor on this new platform. From our dumpy office in downtown Bellevue (surrounded by industry titans like Bungie and Valve), we’ve put our heart and soul into this ambitious and at times, difficult, project.
Unity has done a good job of keeping developers in the loop about new Unity 5 features and improvements, but we imagine many of you still have a lot of questions about the actual nuts and bolts of a true migration to Unity 5. We’re going to give you all a front row seat into how we remastered République in Unity 5, but bear in mind: our approach isn’t necessarily the only approach, nor the best one.
Think of this as part documentary, part walkthrough. Wherever possible, we’ll point you to resources, documentation, and general info to help you get going in Unity 5. We’ll also chronicle our journey like no one else but Camouflaj can, sharing our personal experiences and the methods to our madness in the most open and honest fashion possible.
République Migrates From Unity 4
How we moved our project from Unity 4 to Unity 5 in the midst of a chaotic, 20-person project, starting with our initial investigation.
Podcast: Episode #2Download (101MB)
Dev diary #2
République Migrates From Unity 4 to Unity 5
Our first big step in getting started with Unity 5 was migrating our project over from Unity 4. For our 20-person team, this was a huge endeavour that required a lot of planning. Your project may not be as big as ours, but hopefully you can learn from our experience.
We started by creating a new copy of our project in a separate Perforce branch. This branch served as an intermediary version of the project – the main project was broken for a few days, while we worked to fix compiler errors caused by plugins or our in-house code that didn’t work in Unity 5 (more on this later).
Creating this separate branch allowed our artists and designers to continue working in the Unity 4 version of the project without getting blocked, while we got the Unity 5 version in working condition. In addition, we made a stripped-down version of the project that we moved into Unity 5 a few weeks before our team migration. This was used by the artists to develop and test our own pipeline for Unity 5’s physically based shading. Stripping it down meant tearing all our 3rd party plugins out -- anything we were concerned could impede progress. Once we had the Unity 5 branch ready and problem-free, we moved the rest of the team over to it and archived the old branch.
We recommend branching your project if you’re using source control (actually, we highly recommend using source control, period!). Either way, make sure you create a backup of your project before going to upgrade it to Unity 5.
Our UI designer and source control guru, Charlie Helman, was responsible for our team-wide migration, and he was gracious enough to lend us notes and screenshots on the process.
Here’s our shiny new branch, created from our existing Unity 4 project.
République’s Unity project is enormous compared to most other mobile games. Weighing in at 45.7GB, the project can take 24 hours to import on some machines. Utilizing Cache Server brings this time down to about 30 minutes.
Before opening our project, we want to make sure Cache Server is set up in the Unity 5 preferences. Now is also a good time to make sure we set up our other preferences for things like external code and image editors (Visual Studio or Photoshop, for example). To do this, launch Unity 5 and create a new empty project.
Once the new project is open, we head to the Preferences (On Mac: Unity > Preferences; On Windows: Edit > Preferences)
On the Cache Server tab of the preferences, I made sure “Use Cache Server” was checked, and set the IP to our cache server’s address (which is simply “server”). After verifying it worked via “Check Connection”, I closed down the project.
Now we’re ready to open the real thing up. After a few minutes, we see the import progress bar. Periodically, we see this window.
The Unity 5 API features a good collection of changes, one of which being all “shortcut” references (like .rigidbody or .transform) that were previously doing a
.GetComponent<T>()call under the hood now need to explicitly call GetComponent. So,
myCharacter.rigidbodynow needs to be
myCharacter.GetComponent<Rigidbody>(). Unity’s ripped out all the shortcut references, and they’ve made an automated process to help do this for you.
When we select “I Made a Backup. Go Ahead!” Unity runs that process.
Unity 5’s AutoUpdater is pretty awesome - ideally, it will be able to parse through your entire codebase and make the proper updates.
Speaking of which, let’s examine some of the script changes in Unity 5, and how Camouflaj made these work for us.
Script Updates in Unity 5
In Unity 5, we saw several name and API changes to scripts. Generally the obsolete attribute was added to the old names, so the compiler would give advice about what to use instead. In some cases, however, it is not merely a name change: you might need to consider between multiple alternatives. For example, the AnimatorStateInfo nameHash was replaced with both shortHash and fullPathHash. The short hash is the name of the state without the layer or state machines prepended, which is more useful in most of our cases. If you need to differentiate states that have identical names, but are in different state machines or layers, then you’ll want to use fullPathHash.
If you want to maintain backwards compatibility with previous versions of Unity in your script, you can use the Unity version macros. We preferred the approach that treats unknown versions of Unity as 5 or later. That way, it’s less likely that we’ll have to revisit them in another upgrade.
Here’s an example for allowing code specific to different Unity versions to coexist.
#if (UNITY_3_0 || UNITY_3_1 || UNITY_3_2 || UNITY_3_3 || UNITY_3_4 || UNITY_3_5 || UNITY_3_6 || UNITY_3_7 || UNITY_3_8 || UNITY_3_9) // Code specific to Unity 3 versions. #elif (UNITY_4_0 || UNITY_4_1 || UNITY_4_2 || UNITY_4_3 || UNITY_4_4 || UNITY_4_5 || UNITY_4_6 || UNITY_4_7 || UNITY_4_8 || UNITY_4_9) // Code specific to Unity 4 versions. #else // Code specific to other versions of Unity. // Here will we shall assume this is Unity 5 and later, and in doing so choose not to support Unity 2 or earlier. // There may be a few defines for Unity 2 if you wish to support those versions. #endif
The shader compiler has been switched from Cg 2.2 to HLSL in Unity 5, and you’ll discover that it has some stricter requirements. With HLSL, output variables that aren’t completely initialized will cause a compilation error. There were many times where we hadn’t initialized the alpha in a color, or the w in a position. To address this, we initialized these values to zero, but as an alternative you can use the macro UNITY_INITIALIZE_OUTPUT. In some cases, we also found that we had completely unused data being passed from the vert to the frag which we removed from our v2f structs.
Manual Edits made beyond Script Auto Updater
One of the things we had to manually repair in several instances were places in code from where we we were trying to retrieve components by name and not by type, which Unity no longer allows. During the Script Automatic Updater process, Unity converts these older instances of component retrieval by type name from:
UnityEngineInternal.APIUpdaterRuntimeServices.GetComponent(go, "Assets/PathToMyMonoType/Scripts/MyMonoType.cs (207,8)", "MyMonoType")
You’ll probably see a few of these conversions littered through the console as errors, because these updates technically don’t compile.
The main offenders of this type of component retrieval were in third-party plug-ins, since we try our best to be as type-safe as possible within Republique’s codebase. However, in some of our older AI code, we had set up our GOAP system in such a way that Actions and Goals could be specified in a dynamic list format, which referenced MonoBehaviors by name. This allowed us the flexibility of a dynamic runtime configuration, but it wasn’t going to work with this new component retrieval restriction in Unity 5. In this particular case, we didn’t need to have the underlying Action and Goal types derive from MonoBehavior so converting them to just standard classes solved our problems.
For other cases where we needed to manually fix this kind of type retrieval, we were able to work around the issue using a new scheme for getting types by name like so:
This final bit is a sneak peek into our work on physically based shading, which we’ll cover in our next blog.
In order to prepare Republique’s existing materials for physically based shading, we had to rethink how we authored the materials’ corresponding textures. In the past, most of our materials only had a few textures associated with them. With Physically Based Shading, our materials must now crunch on a larger number of textures. In order to quickly process the existing materials, our programmers created a material conversion tool. The tool has sped up our conversion process big time: it archives the legacy textures associated with the source material, moves and renames the existing source material (using Unity’s AssetDatabase methods to retain existing links to prefabs, models, and the link). It then links the new physically based texture maps, sets a number of required texture import settings, and generates the final packed map and asset that represents the combined texture maps.
This allowed our art team to convert hundreds of materials per day, which greatly improved their workflow and reduced the chance for errors during the conversion process. If you’re moving from Unity 4 to Unity 5 and have a lot of materials to convert, creating a tool like this is highly recommended – we’ve found it’s sped up the rate of texture conversion for our artists by 30% or more. That might not sound like a lot, but if you’re in a fast-paced production setting like us, every little bit helps.
Note: most of this work was performed using Unity 5 various beta versions. Experience with final version might vary.
République in Physically Based Shading
This is the really exciting stuff. We go over a little bit of Physically Based Shading for the uninitiated, and explain how we put this to work in our game.
Podcast: Episode #3Download (99MB)
Dev diary #3
République in Physically Based Shading
When we were doing our initial investigations into moving Republique to Unity 5, most of our artists knew very little – if anything – about physically based shading, or PBS. The good news is that within a day or so, everyone was up to speed, converting our Unity 4 textures to take advantage of Unity 5’s physically based system.
So, what is Physically Based Shading? Well, there’s a ton of great breakdowns on the web to get you up to speed:
Definitely check these out before moving forward. We’re assuming you’ve investigated whether or not to make PBS a part of your Unity project (for what it’s worth, this was the number one reason we decided to make the jump mid-development). Just in case, though, let’s go over some of the big differences between the texture creation pipeline you’re used to, and the PBS pipeline new to Unity 5.
The biggest change is the number of maps that you need to create for a PBS asset. With the old method you might create an diffuse map, specular map and normal map.
After checking out the links above, hopefully you’re feeling pretty comfortable with the two major PBS pipelines: specular (or “spec-gloss”) and metalness (or “metal-rough”). Unity 5’s standard shader utilizes the former. To work in Specular, your textures will now need different maps. If you are missing any of these maps your material will not look correct, as they are all very interconnected.
This PBS stuff might sound like a lot more work up front. However, in the long run we’ve found it’s saving us time. We author the textures once, and assuming we’re doing things right, we should never have to touch them again. Plus, just look at the difference in fidelity between our Unity 4 textures, and the Unity 5 stuff:
Converting textures from Unity 4 to Unity 5
Republique started as a mobile title. Back in the days when we were authoring our textures specifically for mobile, we used a legacy shader system containing a diffuse map with embedded lighting information. We did this to fake what more expensive shaders were capable of doing because of the limitations of mobile (particularly older devices like the iPad 2 and iPhone 4). However, moving to PBS meant readjusting this approach. With the increased memory overhead on desktop, we could finally take advantage of all those maps!
Our artists each took slightly different approaches to texture conversion. Here’s one of our artists, Jeremy Romanowski, discussing his method:
Since about half of our materials for the mobile version of Republique were lacking normal map information, it was important to have a means to extract, or at least mimic, that data.
Quixel’s nDo plugin is the solution to that problem. I was able to very quickly visualize and generate normal map information for materials like wood trim and metallic engraving, as well as tiling textures like hardwood flooring, concrete walls, and ceramic tiles.
From that normal data I am able to extract an ambient occlusion map. Sometimes I used nDo to generate it directly from the normals; sometimes I used Substance Designer’s normal to height to ambient occlusion function. I found nDo was more useful for extracting AO data from micro and high frequency details while Substance Designer was better for large forms. Using a combination of both results and blending the two maps in Substance Designer was common practice.
The next map I tackle is the metalness map. This is an overwhelmingly black and white texture map. Since there are virtually no examples of materials in reality which have both dielectric and non-dielectric properties, it is important to stay away from everything in-between 0 and 255. If you’re attempting to mimic the real world, stylized or not, the only time a metal object behaves as a dielectric (non-conductor, wood, rubber, plastic, etc.) is when a thin film of dirt or dust is built upon the surface. Knowing this information, I use Photoshop to mask out areas of the material which should behave as metal (255,255,255) and those which should behave as everything else (0,0,0). The metalness map is one of the fastest to author, as well as one of the most important.
Once the base maps are generated (Albedo, Ambient Occlusion, Normal, Metalness) I’m finally ready to take advantage of the awesome power of the Roughness map. Most artists familiar with metal/rough physically based shading will agree the roughness map is king. With a single Contrast/Luminosity function you can watch a material transform from a 40-grit sandpaper to the smoothest material known to man - something akin to a sphere of mercury suspended in zero gravity completely undisturbed by the forces of nature. Yes, that smooth.
Using an edge detection function coupled with various node-based filters and effects (scratches, dust, abrasion, etc.) is one of the most powerful and satisfying ways to see the true power of physically based shading.
As a whole, I found Substance Designer was the perfect medium between control and automation. It also feels great to hook up various nodes and swap the order of operations very quickly to get an instantaneous preview of the result in an HDR lit environment. You can also import your own HDR environments if you want to get an even more accurate representation of how it will appear in your Unity scene.
Once I have achieved the desired look, I export the bitmap outputs as a .tiff at the highest possible native resolution and check them into our Unity branch. Of course, like most artists I will go back to my material authoring program for final adjustments. However, I will say I’ve been very pleased with the visual consistency from one physically based renderer (Substance Designer/Quixel dDo) to another (Unity 5).
Updating our directory structure
For Camouflaj, updating our directory structure from Unity 4 to Unity 5 was a teamwide endeavour. We needed a way to manage all of our physically based shaders, and the maps were so integral to how shaders worked; this meant restructuring our pipeline. We knew we had to be able to troubleshoot every material, and track what materials were converted. Updating over 1200 materials – each containing 5 unique textures each – is a big task. That’s why we created a material directory that had all of our converted physically based shaders, and their corresponding textures, in the same location, with individual folders defining each material. In the end it looked something like this:
This was incredibly useful. It made it clear that materials in Alloy’s directory were processed, and anything outside of the directory was not. We also added a legacy directory, containing all the textures that had been assigned to these updated materials.
We’ve converted a ton of textures over to Unity 5 now, and it’s really transformed the game. Here’s some more before and after shots:
While we’ve got the game looking great in editor, we want our users to be able to see and experience these massive changes as they were intended to be seen. Getting the most out of Unity’s lighting system was a crucial step in the process.
République Lighting & More
We explain how we made use of Reflection Probes, Global Illumination, cookies and other good stuff, plus we cover some physics and animation refinements.
Podcast: Episode #4Download (176MB)
Dev diary #4
As mentioned in our previous entry on physically based shading, all of the lighting information in République’s textures on mobile were placed inside our diffuse maps. In our transition to Unity 5, we made several sweeping changes to our lighting pipeline. Let’s talk about the first one: moving from forward to deferred rendering.
Forward vs. Deferred Rendering
If, say, you’re a mobile developer who’s never used deferred rendering in a project before, here’s a great explanation of the differences:
Forward rendering is the better option for mobile, and in a lot of cases you don’t need dynamic lights even on desktop, because it handles alpha sorting better and supports fog. Ultimately, we switched to deferred rendering because we wanted dynamic lights in our desktop version. Technically, we’re using both forward and deferred rendering in our game -- forward in OMNI View, (our hacker/detective mode) and deferred for the rest of the game.
So once we made the switch to deferred, we had some performances costs to keep in mind as we were reauthoring every light in the game. That’s where our use of light cookies came in.
Light cookies aren’t new to Unity 5, but they’re something we found useful. They’re basically masks you apply to lights that allow you to avoid paying the cost of shadows, which is the most expensive aspect of a real-time light. We made use of spot light and point light cookies in République, but directional light cookies are a great option for outdoor scenes.
The easiest way to set up a light cookie is to piggy backing off Unity’s reflection probe tools.Here’s how to replicate what we did:
Because we want a non-destructive pipeline, we first create a new scene and place a ground and wall plane in that scene. Drag in the object that needs a light cookie -- in this example we are going to create a cookie for a standing lamp with a lamp shade. We do this because we want to verify that the light’s placement fits our needs, and the only way to verify that is how it reacts to its surroundings.
Next we need to generate a reflection probe. The easiest way to do this is selecting your light and creating a reflection probe that is then parented to it. (If you don’t know much about reflection probes in Unity 5, they’re explained in greater detail below.) If you have a transform of 0,0,0 it will line right up to the position of the light.
Next, we need to get our geo ready for baking out our cookie. The best way to do this is applying black materials to the geo and setting your background color to white. By doing this, you are effectively determining where the light can shine, because of the way you have colored your scene. Here are the settings we used:
After hitting “bake reflection probes”, we now have a cube map that does a reasonable job of establishing our light map cookie. As you can see in the image below, our light is no longer set to shadow casting. Instead, the cookie was placed into the light. It looks a bit crude at this point, but that is easily fixed once we import our image into an editing program like Adobe Photoshop.
After some quick editing we take our image from this...
...to this. We’ve simply blurred out the major areas to soften up the shadows.
Once the reflection map is saved and updated, you have your finished cookie! It looks much better then the dynamic light by itself, because the cookie allows for subtle gradients.
In summary, light cookies are great if you have static objects in your scene and don’t want to pay the cost of real-time shadows. Of course, for characters and other moving objects, we made use of dynamic lights, with shadow casting wherever possible.
Making the Most out of Enlighten
Enlighten is the biggest change to Unity 5’s lighting system. It supports light maps just like Beast did in Unity 4, but now you can use Real Time Global Illumination (or GI for short). Getting this into République was a big win. Not only does it give our scenes subtle gradations in their ambient lighting, but it will help us when we take everything we’ve done in Unity 5 on desktop and get it back onto our mobile version.
Enlighten’s Real Time GI allows the user to precompute lighting data and simulate what has until now only been possible with dynamic lights. This data is generated by objects that have emissive values applied to their materials: flickering lights, candles, and so on. Once you setup your scene and bake Real Time GI, you will then be able to change the emission values of your lights, brightening and darkening the scene - no dynamic lights needed!
Be warned: you will want to set the Real Time GI settings for all the objects in your scene before you bake. If you don't, you will have dramatically increased bake times. The best way to do this is going to your lighting tab and filtering by mesh renders. Then, grab all your geo, set the Auto UV Max Distance to a value of 2, and the UV Max Angle to 120. Finally, set the advanced parameters to Default-LowResolution. We have found these are good generic values and speed up render times dramatically.
Emission values for materials are set to 0, resulting in a dark environment...
Here’s the same environment, lit only with Shader Emission values and the indirect lighting located in lighting tab. A great example of what you can do even without dynamic lights.
And here we have the lighting tab settings for this scene:
Reflection probes are another feature new to Unity 5. On mobile, Camouflaj created an in-house tool that used parallax corrected cube maps to fake reflectivity in our game. Like many games, République is filled with metals and other surfaces that would have reflective properties in the real world, so the physically based shaders we’re using go hand in hand with Unity 5’s reflection probes.
So, what are these things? It’s best to think of a reflection probe as a screenshot of a room or scene in your game, projected onto a cube and then forced onto a sphere.
When placing our reflection probes, we segment our scenes into cuboids. We can use anywhere from one to a dozen probes, depending on the size and shape of the space. The objects that best utilize reflection probes are, predictably, reflective materials such as glass and metal. Simply put, if a room was 100% carpet, a reflection probe would have little or no effect. In this case, light placement is what’s most important.
When we’re building out our scenes, there are often times when we don’t want certain objects included in our reflection probe’s cubemap, and we want it occluded. This can be achieved using the static tagging options. Much like the use of Lightmap Static in Unity 4, If an object is tagged as Reflection Probe Static, it will be factored into the baked cubemap.
Thanks to Unity 5’s lighting features and the freedom we’re enjoying on desktop, we’ve seen some huge improvements. This is a game we would have loved to see on mobile but couldn’t because of concerns about the readability our screen. The subtle gradations, the play of light and shadow...it’s been a satisfying venture for our artists. Now the question is: does it play as great as it looks in screenshots? That’s what we had to ask ourselves as we moved into the final stages of development, getting every last bit of performance we could out of République.
République Ships on Unity 5
We document our push to launch, and how we optimized and (fingers crossed) shipped a fantastic game.
Podcast: Episode #5Download (153MB)
Dev diary #5
Optimization and Release
The final stretch of development can be a painful one for many developers. Already exhausted by pouring your energy into content creation and raw creativity, you’re faced with tough choices: this feature has to go, that feature doesn’t perform well on our target platform. Camouflaj had a fair amount of runway to focus on polishing and iteration compared to our previous mobile releases, which were invariably a mad dash to the finish line. Still, sacrifices had to be made.
Our experience with Unity development gave us some insight into best practices as we approached release. However, there are some specific improvements to optimization analysis in Unity 5 that will prove helpful to you. Let’s take a look at what we were able to do to get République performing well.
Big Lighting Costs and Big Savings
Ah, those dynamic lights in our last entry sure do look great, don’t they? Unfortunately, when it came time to ship, we still had to make concessions. This is a natural course of development: artists draw up their dream world, then the technically minded folks come in and figure out how to make it function in-game as best as they can.
One of the most taxing areas in our game, performance-wise, was our library space. A large open room that serves as the hub for Episode 2 of République, there are very few opportunities to utilize traditional cost-saving measures like occlusion culling. Luckily, programmer Eric Bergen was able to create a custom tool that lowered some of our lighting costs - expensive shadow casting, most notably - in this massive environment.
Unity has a Quality Settings system built in, which uses a single setting to control everything from the quality of shadows to level-of-detail selection. Because of the great diversity in PC and Mac hardware, we wanted to give players the option to select individual categories of quality to adjust. To do this, we wrapped the Quality Settings with a more granular system that manages the default system as well as custom performance branching in our scripts.
We often use scripts that destroy or disable renderers and effects based on our quality categories. When players drop the particle detail setting from High to Low, we have expensive particle systems deactivate, or if the shadow settings are lowered we drop out some lights and some shadows. This is a quick and effective way to boost performance on older hardware, while while retaining graphical customization for more powerful hardware.
Unity’s Profiler isn’t new to Unity 5, but it’s extremely useful for investigating what’s taking up too much CPU or GPU time in your project. If you’re unsure about whether scripts, lighting, or geometry are responsible for performance strain, the Profiler will tell you.
The Profiler's timeline can help you visualize the performance of many areas of your game to see where resources are being allocated in each frame. This is particularly useful for observing the performance implications of asynchronous operations--operations which are run outside of the main and rendering threads--such as occlusion culling.
Debugging Our Frames
One thing that is new to Unity 5 is the Frame Debugger, which shows you what all the draw calls are in a given frame. This is helpful for figuring out where you’re losing frames. Here’s a look at the Cocoon and surrounding library space in Episode 2 with the Frame Debugger.
As you can see, we’ve got quite a lot of stuff rendering in this particular frame! One of the really handy features of the debugger comes from its integration with the Unity editor. Selecting any of the events in the debugger will take you to its instance in your scene. With this, you can more easily track down and modify any potential draw-call-intensive game objects. For projects where every single draw call matters, this tool is indispensable.
As I write this, we’re just a few days out from release. We’re pulling late nights at the office, pounding out the last few bugs and crashes on our PC and Mac builds before sending codes out to reviewers. We’ve revealed République Remastered to the world, but it won’t feel real until the game is in the public’s hands. And one of the greatest unexpected outcomes of this endeavour is the opportunity to speak to developers like you. We hope that relating our experience in Unity 5 has you feeling as inspired as we are. Thanks for reading!