Month: August 2013

Unity3D Unit Testing 1

| srivello

About RMC & Unity3D

Rivello Multimedia Consulting (RMC) provides consulting services for applications and games. RMC specializes in Unity3D development (see our work here). Please contact us today with any questions, comments, and project quotes.

As always, RivelloMultimediaConsulting.com/unity/ will be the central location for deep articles and tutorials, Facebook.com/RivelloMultimediaConsulting (like us!) will engage the growing RMC+Unity community, and for the latest opinions and cool links follow me at Twitter.com/srivello.

Update: December 18, 2013, Unity Technologies releases the official “Unity Test Tools.” Read the full post below, then jump over to “Unity3D Unit Testing 2: Unity Test Tools“.

Intro To Unit Testing

As the author of the Art Of Unit Testing begins, “I used to feel that a ‘unit’ was the smallest possible part of a code base (a method, really). But in the past couple of years I’ve changed my mind”. Unit Testing is a fundamental of Extreme Programming too. Here’s how I define a unit test;

A unit test is an automated piece of code that invokes a unit of work in the system and then checks a single assumption about the behavior of that unit of work.

A unit of work is a single logical functional use case in the system that can be invoked by some public interface (in most cases). A unit of work can span a single method, a whole class or multiple classes working together to achieve one single logical purpose that can be verified.

A good unit test is:

  • Fully automated
  • Consistently returns the same result
  • Tests a single logical concept in the system

Reasons To Write Unit Tests

Development experts urge us all to test. Here are some top reasons;

  • Tests Reduce Bugs in New Features
  • Tests Reduce Bugs in Existing Features – With well-tested code, introducing new features rarely breaks existing functionality.
  • Tests Are Good Documentation – A concise code example is better than many paragraphs of documentation.
  • Tests Allow Refactoring \ Reduce the Cost of Change – Be more nimble. Experiment with confidence.
  • Testing Forces You to Slow Down and Think
  • Tests Reduce Fear – Having a complete test suite allows programmers to remove the fear of making changes or adding new features.

Unit Testing in Unity

So in Unity, we have three levels that need to be continuously tested:

  • C# code
  • Prefabs and single configured objects
  • Scene definitions for the final game – also known as levels.

There are some challenges to each, however the 3rd is probably the most tricky since much of its setup is done in visually through the editor (vs in C#). One tip is to configure all Game-Objects as prefabs, so that the scenes only contain a minimum of manual definitions. Some greater thought on the subject is outlined in this post.

The Unity Technologies team (creators of the Unity IDE) announced in 2013 Unity’s Runtime API Test Framework. While this is a great sign of the teams dedication to TDD (see next section) and to create tested, durable code for us to use, this framework is not available for us developers to use. The team released this chart showing how much of the Unity code-base is currently covered by testing (see Figure 1).

Figure 1. (Updated August, 2013)
Figure 1. (Updated August, 2013)

Test Driven Development (TDD)

TDD is the practice of creating your tests while you implement (rather than after). Moreover, TDD developers create the test first (returning the simplest incorrect value), runs it to see the failed test result, then implements the correct, complete implementation. The unit testing leads or ‘drives’ the programming workflow. This requires discipline but leads to a codebase with (good) heavy test coverage.

To perform a test you need 3 things; a testing framework to run the tests, you need the tests, and then you need something to test. Here is an example of a test and something being tested.

Example Test (Pseudocode)

//ASSUMING YOU HAVE A CUSTOM 'DoIntAddition' METHOD TO TEST...

[Test]
public void DoIntAdditionTest() {

//SETUP
int value1_int = 10;
int value2_int = 1;

//TEST
int result_int = DoIntAddition (value1_int, value2_int);

//EVALUATE VIA ‘ASSERTION’
Assert.IsTrue(result_int == (value1_int + value2_int) );
}

Example Test (Summary)

  1. Most Unit Testing Frameworks have some meta-tags (attributes) such as [test] that is used to indicate which methods are ‘test methods’
  2. You do any setup needed to cleanly prepare for your test (optional)
  3. You perform the test (such as calling a custom method to be tested as shown)
  4. You evaluate using an assertion. An assertion is your way as the tester to suggest the expected behavior. In this example it seems almost too simple, almost silly to test this method. However, once you create the test, you can be forever confident that the ‘DoIntAddition’ method will perform as expected, even if you later change the implementation of the method. This is the confidence you gain from Unit Testing.

Challenges In Unit Testing

General

Unit testing requires discipline from the team. Team leaders and project stakeholders must allow time not only to add the user-facing features, but also to create non-functional features such as under-the-hood unit tests for the codebase. Each team member too must be responsible for his or her own code. Writing tests for your own code (beforehand as in TDD per above) or during development is best.

First and foremost, a game is different to many other types of software in that a good portion of the code is handling input, and visuals/graphics/UI. These are two notoriously “un-unit-testable” parts of a system.

To help these issues, you can start by separating your visuals from your logic. This will make the logic more testable.

Specific Challenges for Unity3D

A staple of Unity development is the heavy use of the MonoBehaviour class. There is always logic in MonoBehaviours that you simply cannot take out and test in isolation. Now if you try to simply instantiate a MonoBehaviour in your script you will see this error in the console: You are trying to create a MonoBehaviour using the ‘new’ keyword. This is not allowed. MonoBehaviours can only be added using AddComponent(). Alternatively, your script can inherit from ScriptableObject or no base class at all. Each Unit3D testing frameworks has a solution to this issue. See the documentation for the framework you choose.

Another problem is the tight coupling of scripts to UnityEngine.dll*, which is not testable; if code depends on UnityEngine directly I can’t give it a mock implementation under my control, ergo it is not testable. This post covers more about the issue.

Links

Unity3D Unit Testing Articles

Unity3D Unit Testing Frameworks (Packages)

Unity3D Dev Diary: uMOM 1

| srivello

About RMC & Unity3D

Rivello Multimedia Consulting (RMC) provides consulting services for applications and games. RMC specializes in Unity3D development (see our work here). Please contact us today with any questions, comments, and project quotes.

uMOM

Problem

Unity3D is a great gaming IDE and runtime. Like any tool I use regularly, I have many constructive criticisms as well. For example, there is no default Main.main() type ‘hook’ to centralize the entry into your runtime and no default way for UI and code to persist from scene to scene. I created a monolithic ‘SimpleManager’ class (see my HD training course video on architectures for more on that), however it is purposefully limited in scale. It is essentially a single manager packed with disparate responsibilities (sound, levels, object pooling, etc…). I knew there was a better solution.

Solution

Welcome to the Unity3D Manager of Managers (uMOM), pronounced as /you-mom/. I’m currently developing uMOM for the AssetStore.

The purpose of uMOM lean;

The uMOM package allows developers to add one or many reusable, custom manager classes to the project. Each ‘start’ automatically, have a predictable lifecycle (reset, add, update, remove), optionally receive ‘update()’ calls, optionally run during edit mode, and optionally persist between scenes. Managers operate as singletons and can easily address each other to collaborate.

Types of uMOM Managers

Really anything is possible, but here is the short list of use cases I will address first. Use some of mine or none of mine – then create your own. Most any existing mananger class you have can easily be converted for compatibility here. 

  • uEventManager -Streamline messaging between classes. Replace the need for Unity’s clunky ‘SendMessage’ and instead dispatch strongly typed objects. (The uEventManager will also be available separately).
  • Store references to audio clips (drag-n-drop) and control audio playback via C# from one place.
  • GUIManager – Layout UI which optionally persists between scenes. Centralize the controls to handle clicks, etc…
  • PoolManager – Optimize your run-time performance by persisting GameObject or prefab instances in RAM and displaying them as needed.
  • LevelManager – Queue up levels and perform transitions (e.g. fade-in-out) between them.
  • GameManager – While other managers are not project specific, this one likely is. Here you manage the core game mechanics, how the user wins and loses, store the score, etc…
  • SaveManager: Takes care of saving and loading user preferences and achievements.
  • MenuManager: Controls all menus, managing their animations, their contents, and their behaviors. Perhaps it heavily uses the GUIManager for layout.
  • Much, much, more…

The instructions of uMOM are concise;

  • 1. Select an existing uMOM manager class (from me, from the community) or create a new uMOM manager which subclasses BaseManager (which extends ScriptableObject). The class acts basically as an interface (because interfaces don’t serialize in Unity3D), allowing you maximum freedom in how you implement.
  • 2. Use the uMOM Unity Editor Window (See Figure 1) for all your needs. All compatible scripts in your project, regardless of folder location are shown here.
  • 3. Click ‘Convert’ (one time only) for the desired script. This efficiently creates a new ScriptableObject (e.g. MyManager.asset) for the original MonoScript (e.g. MyManager.cs)
  • 4. Click ‘Add’ (one time only) for the desired script. It is that step that actually puts your manager into use. Cool! You can click ‘Remove’ if you don’t want the manager to be in-use anymore.
  • 5. Edit your manager class source code however you like. No need to repeat the steps, your changes are hot-updated to the project. All this works in Edit Mode Or Play Mode.
Figure 1. (In Progress)
Figure 1. (In Progress)

 

Problems & BUGS!

During development I have hit many obstacles. All my issues continue to exist, even with the recent Unity 4.2.x release. Developing Unity Editor tools is a unique challenge compared to straight game development and it is unlike any other platforms I’ve used before. I’m sure all of these challenges can be solved.

I would LOVE some help resolving these issues. See this list (I’ll record a video soon if that would help you) and please comment below if you have confidence to solve any one area. I can share the code with you. I ask that any assistance be Work for Hire. Unfortunately on this particular project there is no partnership or co-ownership available. Once I have the project working, I plan to offer it as a premium package in the AssetStore.

1. Serialization

The uMOM class directly edits its public properties (as it should, right?). The many editor classes edit the SerializedObject and SerializedProperty of uMOM’s public properties (as it should right?). However in many places I see that I must STOP, change a value with a click in the uMOM Editor Window, then PLAY to have a change persist. Otherwise the change is lost next time I play/pause.

I have no workaround and I assume the issue is within my code. IDEAS?

2. Finding ‘All’ Assets

The uMOM Editor Window shows a list of all compatible MonoScripts (meaning all MonoScripts in Project window which extend BaseManager). That works. However, the programmatic listing of all compatible ScriptableObjects (meaning all ScriptableObjects in the Project window which have a type of a compatible MonoScript) is not so easy. Unity returns the ‘loaded’ (loose term) instances I want, but not ALL of them. The issue is well known (here, and here, and here, and more).

The common workaround I read online is to manually (with mouse) or programmatically (loop through all assets and call AssetDatabase.Load… [this crashes Unity IDE] or Selection.objects [This works inconsistently]… to ‘refresh’ the assets it can find). Manually = works but is not a long-term solution. Programmatically does not work.

Based on research I think this is a known Unity bug that Resources.FindObjectsOfTypeAll() and its deprecated equivalent Object.FindObjectsOfTypeAll, just do not really give you ‘all’. IDEAS?

Please Vote!

You can vote on this known bug (problem #2 above) here “Make FindObjectsOfTypeIncludingAssets find all just like the object picker” to encourage the Unity team to fix this issue. If you have nothing else to vote on, please cast your votes this way!