Dev Blog 12: Modular AI and Custom Editor Serialization

Overview

In one or two of my previous blog articles I've mentioned how I was implementing a custom artificial intelligence (AI) system.  A typical AI system usually boils down to a list of possible states that a game character or creature (actor) can be in and how or when they should transition to those states. This is known as a state machine.  My system is no exception because I really don't need anything fancy for what I'll be doing.  However I took it upon myself as a learning experience to create a modular based AI system so that I could make use of Unity's editor and quickly define and tweak my AI behavior as I develop.  I ran into a number of problems getting Unity to save that information, mostly due to my ignorance, but I also found it rather difficult to locate a cohesive amount of information on the subject so there was a lot of trial and error.  I also am not totally sure I'm doing things correctly, but it's been working great so far.

 

What is a state?

Let's clarify a little bit of our terms to make the big picture more understandable.  A 'state' is a term used to define an actors behavior.  It essentially describes what the actor is going to do, how they will do it, and keeps track of those things that are important for that information to be accomplished.  Some examples of a state would be things like attack, run away, or follow.  I even use them for simple things like playing a specific animation or doing nothing for a period of time.  Essentially I wanted to write some code for these things in a generic way so that I could take any actor and give them a list of states that they could perform.  I also wanted to be able to define the parameters of these states through the Unity Editor.  Some example parameters would be to define how aggressive an attack should be or how long should an actor follow their target.

 

How are states chosen?

Speaking code now, I called the main mono behavior component that keeps track and manages these AI behavior, ArtificialIntellgience. Simple no? It contains a decider class which is the logic that picks amongst the possible states. There will be various kinds and depending on how smart I want them to be they could pick a state at random, follow a pattern, or observe their surroundings for the best choice. Like all mono behaviors, the ArtificialIntelligence component updates every frame, asking the decider to pick a new state if the old one had completed or got canceled.  Moving back to states, I created a base class to handle the common infrastructure that would be required by all of my states and called that AIState. It looks something like this:

[Serializable]
    public abstract class AIState : ScriptableObject
    {
        protected ArtificialIntelligence intelligence;

        /// <summary>
        /// Initializes the particular state
        /// </summary>
        /// <param name="ai">The artificial intelligence</param>
        public virtual void Initialize(ArtificialIntelligence ai)
        {
            this.Intelligence = ai;
        }

        /// <summary>
        /// Clean up the state
        /// </summary>
        public virtual void OnDestroy()
        {
            this.Intelligence = null;
        }

        /// <summary>
        /// Reset the fields to a starting state 
        /// so the AIState can be reused.
        /// </summary>
        public virtual void ResetState()
        {
        }

        /// <summary>
        /// Triggered upon entering the state
        /// </summary>
        public abstract void Enter();

        /// <summary>
        /// Triggered upon leaving the state
        /// </summary>
        public virtual void Leave() { ResetState(); }

        /// <summary>
        /// Called once each frame to perform whatever actions the state defines
        /// </summary>
        public abstract void Update();

}

The ArtificialIntelligence component makes sure that the appropriate functions are run when transitioning between states. For example, when the old state has completed. The ArtificialIntelligence component will ask the decider for the next state, call Leave() on the current state and call Enter() on the new state.  The current state get's it's Update() function called every frame and the OnDestroy() function is only called when the ArtificialIntelligence component has it's own Mono Behavior OnDestroy() function called.  

Using this as a base, I went ahead and implemented the various states as I needed them. Here is an example of my most simple state

AIStateIdle

public float delay = 1.0f;
private float waitTimer;

public AIStateIdle()
{

}

public override void Initialize(ArtificialIntelligence ai)
{

    base.Initialize(ai);
}

public override void ResetState()
{
     this.waitTimer = 0.0f;
}

public override void Enter()
{

}

public override void Update()
{
    this.waitTimer += Time.deltaTime;

    // Enough time has passed
    if (this.waitTimer >= this.delay)
    {
        this.intelligence.StateComplete();
     }
}

Calling the ArtificialIntelligence StateComplete() function is what causes the state to be considered done and allows for the process of a new state to begin.  

Custom Editor and Serialization

All that work will be expanded upon and improved as I develop the game. None of it was really anything new for me and I spent most of my time trying to get it work in the Unity Editor and actually save correctly.  I wanted to be able to set the decider that the character would use and designate the states and their parameters using the editor.  To do this you first have to make an Editor folder at the root of your Assets directory.  Then inside there you need a special class to handle the editor UI. I chose to call my class EditorArtificialIntelligence. You have to denote that it is a CustomEditor for the ArtificialIntelligence class and it has to inherit the UnityEditor.Editor class.

[CustomEditor(typeof(ArtificialIntelligence))]
public class EditorArtificialIntelligence : UnityEditor.Editor

Inside the class are some boiler plate fields and functions to handle initial setup.

private GameObject gameObject;
private ArtificialIntelligence ai;

private void OnEnable()
{
    this.ai = (ArtificialIntelligence)target;
    this.gameObject = this.ai.gameObject;
}

The majority of my headaches where in this next portion.  I'll provide a large blob of code so you can see the overall function and how I did it.  But first let me explain what we're doing and at a high level what is going on.  Firstly we'll be telling the Editor what to put in the UI. It'll show the combo boxes and inputs. We also have to teach Unity how to serialize (save) the values that are input so that we don't lose them when starting the game or loading a prefab.  Unity provides a number of helper classes that we make use of here including PrefabUutility, AssetDatabase, and various serializing functions from UnityEditor. In order for Unity to know that is should save the game objects you have to mark your classes as [Serializable].  So my AIState classes (see above) and the AIStateEnum, Decider classes, and DeciderType enum all are marked as such.  If you have a private field, such as the decider instance in my ArtificialIntelligence class or the list of AIStateTypes and list of AIStates that my deciders holds that need to be serialized, you need to mark those with a  [SerializeField] designator.  And lastly, if your class is not a mono behavior, such as the AIStates and Deciders, they'll need to inherit the ScriptableObject class.

[Serializable]
public abstract class Decider : ScriptableObject
{
   protected ArtificialIntelligence intelligence;

   [SerializeField]
   protected List<AIStateType> aiStateTypes;

   [SerializeField]
   protected List<AIState> aiStates; 

   ...
}

So with all that, here is the main editor function.  I'll try to comment through the function to explain what's going on. 

public override void OnInspectorGUI()

{

        DrawDefaultInspector();
        if (this.ai.deciderType == DeciderType.NONE) return;

        // This is a custom utility function that I'll show later. It is
        // used to determine where we'll be saving the information of this game object.
        string assetPath = GetAssetPath(this.gameObject);

        // As we code out this UI, keep in mind that we're also writing the logic that handles
        // user interactions and serialization of the user's entered values.
        // This helps us tell the Editor that if any of the below UI get's modified by the user 
        // the object needs to be considered 'dirty'
        EditorGUI.BeginChangeCheck();

        // Load or create the decider, create a new one if it is out of date or doesn't even nexist.
        Decider decider = this.ai.Decider;
        // We have to handle the case when we have a fresh new ai component (with no decider as well as the times
        // when a user changes their decider type)
        bool generateNewDecider = (decider == null || this.ai.Decider.DeciderType != this.ai.deciderType);
        if (generateNewDecider)
        {
            if (String.IsNullOrEmpty(assetPath))
            {
                throw new Exception("No Decider set and asset path should be defined.");
            }

            // Attempt to load from the asset database
            if (!generateNewDecider)
            {
                decider = AssetDatabase.LoadAssetAtPath<Decider>(assetPath);
                if (decider == null)
                {
                    generateNewDecider = true;
                }
            }

            if (generateNewDecider)
            {
                // Remove the old decider asset, since it is being replaced
                UnityEngine.Object[] AiAssets = AssetDatabase.LoadAllAssetsAtPath(assetPath);
                foreach (UnityEngine.Object asset in AiAssets)
                {
                    if (asset is Decider)
                    {
                        UnityEngine.Object.DestroyImmediate(asset, true);
                    }
                }

                // This is another custom function, it is very simple and straight forward.
                // We pass in a deciderType enum (Random, Repeat, SimpleBoss, etc) and it runs creates a new instance and names it.
                // eg.   Decider decider = CreateInstance<DeciderRandom>();
                //       decider.Intelligence = this.ai;
                //       decider.name = decider.GetType().Name + ".asset";
                decider = CreateInstanceOfDecider(this.ai.deciderType);

                // Create the asset before serializing the decider
                if (!AssetDatabase.Contains(decider))
                {
                    AssetDatabase.AddObjectToAsset(decider, assetPath);
                    // Save our changes to the decider
                    AssetDatabase.SaveAssets();
                    // Force everything to sync up so we use these latest changes
                    AssetDatabase.Refresh();
                }
                    
                // Now serialize the decider, applying changes, and then keeping the object updated
                // with these latest modifications.
                this.serializedObject.FindProperty("decider").objectReferenceValue = decider;
                this.serializedObject.ApplyModifiedProperties();
                this.serializedObject.Update();
            }
        }

        UnityEngine.Object[] assets = AssetDatabase.LoadAllAssetsAtPath(assetPath);
        SerializedObject deciderSo = new SerializedObject(this.serializedObject.FindProperty("decider").objectReferenceValue);

        // Update the AIState type size based on user input.
        // We have to wipe out states if the count decreased or add new ones if the count went higher.
        int oldAiStateCount = deciderSo.FindProperty("aiStateTypes.Array.size").intValue;
        int aiStateCount = EditorGUILayout.IntField("Ai States", oldAiStateCount);
        if (aiStateCount != oldAiStateCount)
        {
            deciderSo.FindProperty("aiStateTypes.Array.size").intValue = aiStateCount;
            deciderSo.FindProperty("aiStates.Array.size").intValue = aiStateCount;

            // The user is increasing the number of states
            if (oldAiStateCount < aiStateCount)
            {
                // Initialize new AIStates to Unknown
                for (int i = oldAiStateCount; i < aiStateCount; ++i)
                {
                    deciderSo.FindProperty(string.Format("aiStateTypes.Array.data[{0}]", i)).enumValueIndex = (int)AIStateType.UNKNOWN;
                }
            }
            // The user is reducing the number of states
            else if (oldAiStateCount > aiStateCount)
            {
                // Remove old assets
                for (int i = aiStateCount; i < oldAiStateCount; ++i)
                {
                    foreach (UnityEngine.Object asset in assets)
                    {
                        if (asset is AIState && asset.name.StartsWith("State" + i))
                        {
                            UnityEngine.Object.DestroyImmediate(asset, true);
                            break;
                        }
                    }
                }
            }
            
            // Again we want to keep our serialized objects in sync so apply the cahnges and update everything    
            deciderSo.ApplyModifiedProperties();
            this.serializedObject.Update();
            deciderSo.Update();
        }

        // Allow the user to define what ai state types they want
        decider = (Decider)this.serializedObject.FindProperty("decider").objectReferenceValue;
        if (decider.AiStatesTypes != null)
        {
            List<AIStateType> oldAiStateTypes = new List<AIStateType>(decider.AiStatesTypes);
            for (int i = 0; i < aiStateCount; ++i)
            {
                //This will indent the labels so it's easier to see the heierarchy
                EditorGUI.indentLevel = 2;
                SerializedProperty prop = deciderSo.FindProperty(string.Format("aiStateTypes.Array.data[{0}]", i));
                EditorGUILayout.PropertyField(prop, new GUIContent("State" + i));
                    
                // If they change an aiState, replace the old one.
                AIState aiState;
                if (prop.enumValueIndex != (int)oldAiStateTypes[i])
                {
                    UtilityEditor.SetSerializedProperty(deciderSo,
                        string.Format("aiStateTypes.Array.data[{0}]", i),
                        prop.enumValueIndex);

                    // Remove the old AIState
                    foreach (UnityEngine.Object asset in assets)
                    {
                        if (asset is AIState && asset.name.StartsWith("State" + i))
                        {
                            UnityEngine.Object.DestroyImmediate(asset, true);
                        }
                    }
                        
                    // Generate a new AIState asset before we serialize
                    // This is just like the CreateInstanceOfDecider mentioned earlier, except with AIStates
                    // We pass in an AIStateType enum, (IDLE, WALK_RANDOM, etc)
                    // eg. AIState state = CreateInstance<AIStateIdle>();
                    //     state.name = state.GetType().Name + ".asset";
                    aiState = CreateInstanceOfAiState((AIStateType) prop.enumValueIndex);
                    if (aiState != null && !AssetDatabase.Contains(aiState))
                    {
                        aiState.name = "State" + i + "." + aiState.name;
                        AssetDatabase.AddObjectToAsset(aiState, assetPath);
                        AssetDatabase.SaveAssets();
                        AssetDatabase.Refresh();
                    }

                    // Now serialize it
                    UtilityEditor.SetSerializedProperty(deciderSo,
                        string.Format("aiStates.Array.data[{0}]", i),
                        aiState);
                }

                aiState = decider.AiStates[i];
                if (aiState != null)
                {
                    //More indents for visual ease
                    EditorGUI.indentLevel = 3;
                    aiState.OnInspectorGUI();
                }
            }

            deciderSo.ApplyModifiedProperties();
            this.serializedObject.ApplyModifiedProperties();
            deciderSo.Update();
        }
        
        // Now that we're at the end, we need a matching EditorGUI.EndChangeCheck() to go with the 
        // EditorGUI.BeginChangeCheck() that we had at the beginning. We now know if there were changes 
        // and if there were, set the componetn as dirty so that the user will remember to save their changes.
        if (EditorGUI.EndChangeCheck())
        {
            EditorUtility.SetDirty(ai);
        }
    }

 

And lastly, the utility helper that tells us where the assetpath of a gameobject is:

    private string GetAssetPath(GameObject gameObject)
    {
        PrefabType type = PrefabUtility.GetPrefabType(gameObject);
        if (type == PrefabType.PrefabInstance || type == PrefabType.DisconnectedPrefabInstance)
        {
            UnityEngine.Object gameObjectAsset = PrefabUtility.GetPrefabParent(gameObject);
            return AssetDatabase.GetAssetPath(gameObjectAsset);
        }
        if (type == PrefabType.Prefab)
        {
            UnityEngine.Object gameObjectAsset = PrefabUtility.GetPrefabObject(gameObject);
            return AssetDatabase.GetAssetPath(gameObjectAsset);
        }
        if (type == PrefabType.None)
        {
            // At runtime, we'll have no prefab type so we can't edit assets
            return String.Empty;
        }

        throw new NotImplementedException("EditorArtificialIntelligence::OnInspectorGUI() Unhandled PrefabType. Type:" + type);
    }

 

Now one thing that I didn't originally realize with all of this is that once you have this saved to your prefab, you'll have only one copy that all instances point at.  To avoid each actor from storing and conflicting with all other instances, you'll need to clone or duplicate in some manner all reference type variables, such as the Decider and AIStates in this example.  Look up the IClonable C# interface for further information. I did all my cloning in the Start function of my ArtificialIntelligence component and set a flag so I wouldn't do it again if the function happened to get called any additional times.

I wrote all this mostly from scratch. I did occasionally pull tiny bits from various stack overflow pages or the Unity forum, but no where was I able to find a complete Custom Editor example that showed how to implement serialization of custom objects correctly.  So I hope that this will help you out as well if that is something you've been struggling with. You may use and modify this code for your own game projects. Reproducing this blog post in any form, modified or unmodified is not permitted. Linking to it however is appreciated.