Cocos2D State Machine Wish List

I'm a big fan of Cocos2D for iOS and Android game development.  You only have to read the last 2-3 years of this blog to see that.  Cocos2D is tightly integrated with SpriteBuilder which is an excellent visual tool for creating levels, mapping out game objects, setting up animations and many parts of a game.  As of writing this it just keeps getting better with the advent of V4 road-map and SpriteBuilder's partnership with Apportable we see some great new features.

Right now you can easily integrate, animate and setup your assets in scenes, and then switch back to code in Swift or Objective-C to manipulate your game objects with a nice separation of responsibilities between the SpriteBuilder data files and the Cocos2D code.  

There is only one thing missing, a decent State Machine implementation.

PlayMaker by Hutong Games - http://www.hutonggames.com

UPDATE: I have started hacking around to get a sense of how much work there is and what sort of stuff there is in SpriteBuilder's code base to help.  On Github:

• Initial proof of concept for the state machine & bindings as a SpriteBuilder game
• Includes some performance testing
• Fork of the Cocos2D project with the state machine & bindings from the PoC
• Fork of SpriteBuilder with some new State Machine node types

Screenshot of the SpriteBuilder state machine fork

There's already some support for this from SpriteBuilder and Cocos2D folks which is awesome.  The executive summary is:

• we can have a better state machine that suits Cocos2D games
• we can have it visually designable in SpriteBuilder

My goal is to get something very humble that might not represent the best UI idiom up and working in the next 2-3 weeks, and this will go back to the community for some early feedback.

---

The gold standard for State Machine implementations at present in game development is surely Hutong Games PlayMaker for the Unity 3D game development system.  Of course PlayMaker is not available for Cocos2D, and even if it was PlayMaker's tight integration with Unity's component system would mean that Cocos2D would have to be re-architected to work with it.  There's no way that a "port" of PlayMaker to Cocos2D makes sense, as far as I can tell.  Nonetheless if you ask what is the premier experience for game developers using state machines in their work, I'd have to point out PlayMaker.

Unreal Engine's State Machine editing - http://www.unrealengine.com

Unreal Engine's editing tools also include a State Machine editor, but its very tightly bound to the skeletal animation system.  I have never built a game using Unreal but I suspect that despite the apparent power and sophistication of Unreal's editing tools you'd be hard pressed to use the State Machine to do the kind of complete visual game development that Hutong promises with PlayMaker. I know that realistically if you create a game with Unity and PlayMaker you are going to have to write some code, but if you can for example do great swathes of your game logic in PlayMaker that is very powerful as the less technically savvy members of your team can then get in and work on play without having to know C# or how to script & code Unity.

What Can We Learn?

So we don't have PlayMaker in Cocos2D, but we have an opportunity to learn from the state machine implementations of others, and make something great for 2D game development in our favourite toolkit.

Given a port of these tools is not the right thing, what can we take from PlayMaker?  Of the many things that PlayMaker gets right with respect to its visual editor would have to also be gotten right for a decent State Machine implementation for Cocos2D:

• tight integration with the visual level editor
• preview states and actions in the visual level editor
• states, events and transitions can be stored inside objects/components
• on publish, state machine data is integrated with the game object files
• tight integration with object animations to easily do common things
• can be used for many parts of gameplay, not just animations

Obviously in the case of Cocos2D the editor is the SpriteBuilder tool, which already handles animations and visual editing of many object properties.  We could in theory make a separate State Editor but it would have to be able to be docked in Sprite Builder in order to achieve these things.

What is there Already for Cocos2D?

At present Cocos2D has many of the things needed to profit from a great State Machine implementation, but there are no pro-grade State Machine tools around as far as I know.

There are a few open source implementations around for State Machines in code.  I used Blake Watter's Transition Kit in my game Space Bot Alpha.  Its a nice project, well set up with CocoaPods, unit-testing and so on, so as a piece of Software Engineering its nicely done.  But for game development for me it really failed on the count of making my job easier.  I wound up doing more work for it that it did for me.

A big part of why that was the case is that there is no tooling integration.  I cannot create a State Machine visually with Transition Kit, or any of the other current offerings - it all has to be done in code.  First off this has the problem that its very difficult to do this with a nice separation of concerns and a clean interface.  You wind up embedding the State Machine setup into the objects initialisers and then calling the transitions from various places scattered throughout the code.  Secondly its time consuming and confusing to layout the State Machine in code - with the best syntax and API in the world any useful state machine quickly becomes cumbersome and difficult to maintain.

A second problem is that there is no tight integration between the Cocos2D objects and the State Machines.  You would have to write a whole binding system, but that is work over and above writing the game I needed to write.

Think about Chipmunk as it is bound in Cocos2D right now.  Just turning on Physics and having Cocos2D node objects under a Physics node in SpriteBuilder enables it for physics.  Then once I have checked the enable physics box, the Chipmunk system can change the x and y position of my Cocos2D nodes in response to the physics simulation all without me having to write any code.  Nice!

That's how it should be for a State Machine.  I should be able to hook up a State Machine to my object and have transitions drive changes in the objects properties.  Even if that had to be done in code, having some sort of binding system would save a lot of work.  As it is the code to alter properties in response to state changes gets messy and its easy to circumvent the state machine by mistake and get unexpected results.

Another interesting project is hsm-statechart which is written in C, and is scriptable in Lua.  In theory this should be easy enough to integrate into a Cocos2D project.  However despite some attractive extra features (discussed below) relating to Hierarchical State Machines, and C performance, I tried Transition Kit first as it had an Objective-C API already.

Also worth understanding is the Apache SCXML project which provides an XML format for describing HSM's like those you can build with hsm-statechart.

Memory and Performance

Another problem I see with some State Machine implementations - and Transition Kit is not too bad here but it could be better - is object proliferation.  Its should be possible to create states and destroy them without doing a lot of allocation and destroying of objects off the heap.  Let's imagine we have a zombie game with 40 zombies, and a hero firing hundreds of bullets - we don't want to have an NSObject for every state in every machine for each of those.  But that's how some FSM's are set up.

Instead you need some sort of manager class that can create a unique state machine "pointer" record without duplicating the state objects for each entity using that machine.  Its like a set of CPU registers for a process or thread referencing a copy of a binary program file without making a copy of it for each process or thread.

Also for performance the management of updates on the machine would have to be done well - tightly integrated with the game objects own update mechanism where timers were needed; or designed so that it does not unnecessarily get updated.  String dispatch would be ideal, as long as it does not impact on performance, and it should be used where it can reduce coupling of the objects in the program - we don't want to create circular pointer chains that could result in ARC retain cycles.

Blending and Animation Integration

Check this video on setting up an animated door in Unity using PlayMaker.  Of particular interest is the blending of the open to the closed state & animation which comes for free.  Here you have an animation for the "closed" door, which specifies certain XYZ positions for the door model, and similarly XYZ positions for the "open" animation.  Creating a transition between open and closed gives you a smooth blending from one set of XYZ to the other, similarly as setting keyframes for a Cocos2D animation would.

In a game getting the State Machine to work well with the animation and actions system is vital.  If a whole lot of extra code has to be written to get this to happen the you wind up doing more work for the State Machine than it does for you.  Blending is part of the story - here you are getting basic animations for free just for using the State Machine.  But where you want to have an animation play as an object transitions from one state to the next you need extra care.

Sub-States: A Hierarchical State Machine

How do you handle the fact that during an animation from open to closed, you are not really in either of those states but in fact you're in some in-between animation state??

One option is to have a 4-state model for your door: open, closing, closed and opening and then you can have the animation complete handler for the closing animation cause the transition from closing to closed.  The problem with this is when you have some other game logic that depends on checking the state of the door.  Lets say you have some poison gas in the room - it can escape into the adjoining room if the state of the door is one of open, closing, and opening - but if an NPC character wants to walk through the door they can only do so in the open state.

A more complex State Machine implementation is to use State Charts to solve the problem.  These add hierarchies of states, and also variables & guards to the system.  Variables are like game objects values, except they only exist within the framework of the State Machine.  Heierarchies mean that you can take the animation states which are visual effects that depict and implement an object; and place them inside a semantic (or meaning related) concept of a state.  So you could have two high-level states of Closed and Open; with Open having as sub-states Opening, Closing and and Completely-Open animation states.  This way you can have the Open high-level state trigger the release of the gas, and the NPC only get access when the low-level state Completely-Open is triggered.

Effectively what you are doing is grouping your states.  You need to specify which of the sub-states is entered when the superstate is entered, and there are some other details, but basically they're an extension of the simpler FSM concept.

So here's the remainder of the items, which would apply to a State Machine implementation regardless of the game level editor question.  Here's part 2 on my wish list:

• the FSM must have simple bindings to objects (like Chipmunk)
• transitions can be configured to drive changes in objects properties
• blending of property changes should come for free (see video)
• load states, transitions and events by reading some kind of data file
• can also create states, transitions and events in code if needed (rarely)
• state machines can be serialised for use in save/load-game and other cases
• small memory footprint/CPU usage
• ability to specify variables and guards as for HSMs (nice to have)

When?  Why?

I think having a solid State Machine implementation with editor support would get new comers to game development much more excited, productive and involved in game development.  It could give existing experienced developers relief from some tedious, detail-focused coding tasks and make those things accessible to the level designers.  And it could make complex bug-prone parts of a game project into data-driven cleanly-seperated modules.  I think it would have huge value.

I'd like to see this happen, so I might be interested in pitching in on such a project.  But it would need broad adoption by the core SpriteBuilder/Cocos2D team.  Let's see if we can get them to have a think about it.  :-)

Cocos2D & CocosBuilder Version Decoder

Just now I got a great question from a viewer on my YouTube channel about Cocos2D & CocosBuilder versioning.  I'm going to post my answer here as it might help a few folks.

CocosBuilder for the Win!

Versions

Versions in Cocos2D can be really confusing - and its no fault of the Cocos2D guys as the stack is a collection of Open Source projects and as such the different components do not have nicely matching version numbers.

For example, right now, you need version 3.0alpha of CocosBuilder to work with version 2.1 of Cocos2D.  You read that right - v3.0 of CCB goes with v2.1 of Cocos2D.  They are different projects and although the guys are working hard to make coordinated releases now, its still confusing.

If you're using the stack I do, which is CocosBuilder and Cocos2D-iPhone then it's easy to get hold of the wrong software packages and get weird errors, with undeclared variables.  Another issue you can find is that at run-time you get an error saying that the CCB data files are from an incompatible version of CocosBuilder.

See my previous post on upgrading versions for a video about this issue and some tips about navigating the versions of Cocos2D.  Executive summary: use the Cocos2D-iphone blog posts to find the latest "coordinated release".

As I write this, for example, the versions you probably want are:

• Cocos2D v2.1
• CocosBuilder v3.0alpha5

These should work together just fine.  Obviously you'll want to use the latest "known good" combination (which might be later than the above depending on when you read this post!).

Release Quality

Another issue with versioning is that while Cocos2D v2.1 has been in "release candidate" for a little while, and we've seen version quality tags like v2.1rc1 or similar as a result: now v2.1 is out and we are back to a major dot-point release.  So nice work by Ric and Birkemose and the guys for getting that up to release quality.

But: we are still in alpha on CocosBuilder - if you're like me you want to work with more stable software.  However here we have to bite the bullet and work with 3.0alpha, at the time of writing, because that is the matching CCBReader for Cocos v2.1.  It is what it is and I guess Viktor Lidholt and other volunteers are working hard on CocosBuilder with what time they have spare to get it out of alpha.  I have not found any real issues so I suggest ploughing ahead with the 3.0alpha builds at this time.

Using the CocosBuilder Source

Note that the link to CocosBuilder above is to the "full source code", not to the "application".  Once you unzip the CocosBuilder full source code, drill down into the CocosBuilder directory with Finder to locate the CocosBuilder.xcodeproj file and double-click that to open it in XCode.  

Once open in XCode you can get your shiny new CocosBuilder binary by selecting the target "CocosBuilder" > "My Mac 64bit" from the Schemes drop-down (top-left in XCode, next to the run & stop buttons) and pressing "Run".  Once CocosBuilder is running, right-click on it's icon in the Dock and choose "Keep in Dock" so you can find it again easily.  

If you lose track of it, its stored under the "build" sub-directory of wherever you unzipped your sources.

Why go through all this if you can just download the binary?

I do it this way so I can be sure I have the matching sources.  If you're just building HTML/Javascript apps, and don't care about iPhone you don't need the sources and can just use the pre-built binary.  But to work with ObjC on the iPhone you need the sources.

Could you download the pre-built binary AND the sources?  Sure.  I don't have masses of room on the SSD on my laptop so I try to save a bit my just building out of the sources but you can use the pre-built binary if you like - just make sure you get the matching one for the sources!

I also like that under the CocosBuilder unzipped source tree you'll find a full documentation set; the Cocos Player application is there and also the full sources to the Example applications.

To get the correct CCBReader that will match your CocosBuilder, use the one from the Examples.  In the case of the above the folder you want is at this path:

$unzipped_source_tree/Examples/CocosBuilderExample/libs/CCBReader

Possible CocosBuilder v3.0alpha5 Issues

One final note, is that there seem to be a couple of new experimental features like the image previewer pane, on CocosBuilder v3.0alpha5, which seemed a bit clunky - hence I am actually currently using v3.0alpha4 which I find works just great for me.

I haven't reported those issues because I have just been too busy to spend time isolating them, but if you try v3.0alpha5 and see a problem take a screenshot and report it on the CocosBuilder GitHub.  You can probably revert back to 3.0alpha4 until the problem is fixed.

Upgrading a Project to a New Version of Cocos2D

A short video tutorial on this subject, since I just had to do this job myself and its not entirely obvious how to go about doing it, or how to solve some of the problems that can occur.

To check out the project used in this tutorial have a look at my earlier tutorials on Lua (linked in the panel on the right) and you'll find a reference to the GitHub repo where you can download the source.

Just to help Google searches finding this post:

• CTFontManagerRegisterFontsForURL link error?
• you need to add CoreText framework as this is a new dependency for Cocos
• CCFontDefinition symbols not found link error?
• you need to "add files" and re-add the cocos2D libs directory
• there are some new .m files which are in the distro but not added to your project

Link errors when upgrading Cocos2D are probably due to these issues - new files and dependencies.

See my video above for how to fix 'em!

Room to Room

This new progress update video shows a bit of a milestone with our animated door sliding open and our heroine able to move to a new room.

I did all the room background artwork for the apartment a few weeks ago and I have been coding like a demon since then ironing out all the issues with moving from room to room and persisting the game state across view and room changes.

For those of you who are coders you might be interested in this aspect of CocosBuilder and Cocos itself - since it is scene based, and your scene is effectively your entire user-interface, moving to a new room basically tears down the whole UI, and it has to be rebuilt again from the CCB file for the next room or view.

This makes for some fun and games when an item has been modified in one view and you need it to reflect those modifications in a new view on the "same item".  Like wise with items that have been added to the inventory, and which to not appear in the next view.

Lastly of course the actual opening of doors and the logic with the character having to wait for the door to slide open before walking through had to be done.

This is done with action queueing which also took quite a while to get working properly.  So for example the sequence to go through to the walk-in-wardrobe is

• player taps the door to the wardrobe and says "Go"
• queue a GoTo  action to the wardrobe room
• that requires the door to be opened first so...
• queue a GoTo the door
• queue a Open of the door
• when the door open occurs it plays an animation of it sliding open
• finally execute the action to go to the wardrobe

This is all now working.  Whew!

CocosBuilder and CGPaths for non-rect Tap Detection

This is just a short post to frame a video how-to on detecting taps for selection on touchscreen devices, using CGPaths - and how to create those CGPaths using CocosBuilder.

What this essentially does is makes CocosBuilder into a tool that you can use to create non-rectangular bounding paths for tap detection.  As a bonus, I show how you can use the CGPath information to create a blue selection sprite image to signify the active selection in the UI.


I reference Bob Ueland's excellent CGPath tutorial, and that does include some code that you can use for the actual hit test.

For the extraction of the CGPath data from the CocosBuilder file I am not posting my code for cutting-and-pasting but if you've followed my earlier tutorials and are building your own game in Cocos2D you should have enough knowledge to follow what I have done.  It is just simply a case of traversing the nodes and using the standard method to build up a path from the CGPoint information stored in the CCNodes.

My technique for creating a selection sprite from the CGPath data is also not tricky - just check the documentation.  Give it a try and if you get really stuck feel free to ask a question here or on G+.

Hope it helps, and enjoy selecting stuff!

CocosBuilder Revolution

A couple of months ago now I threw away all my code and started again.

Was that a bad thing?  Well, it was bad in the sense that I wish I had known back before I started writing all that code that there was a better way to do it, a way that was going to be so much more productive that even throwing away months of work was going to be sort of OK.

So what is that better way?  Using CocosBuilder.

Cocos2D is a powerful API for building your games on iOS, but it is just code, and when I started out on my game I quickly found it impossible to organise all the resources I needed and to size and position all the elements on the game screen.  As I mention in the video I had seen CocosBuilder last year but it seemed flakey and not up to doing enough of what I needed.

But the new 3.0 version is much improved and there are a number of tricks with CocosBuilder that allow you to do a lot more with it than appears at first blush.  It turns out to be well-suited to creating an adventure game like mine, if you can figure out how to game it to your ends.

To try to help anyone who is contemplating going down the path I went down, or anyone who needs a powerful game interface creation tool but is too afraid to ask, here is a video tutorial I put together looking at how to get started with CocosBuilder for iOS.  The tutorial also includes a bit of my thinking around the game editor journey for me.

The tutorial comes in 4 parts, each of around 10 minutes (due to YouTube constraints) and I have tried to make the divisions not too arbitrary.  I'd encourage watching the whole thing, but if you just want to dip into the bits you need I hope the text below will help choose the right parts to watch.

As well as watching my tutorial please also check out two other great tutorials (not videos but very good nonetheless) which I used myself during my CocosBuilder learning process:
http://code.zynga.com/2012/10/creating-a-game-with-cocosbuilder/
http://www.raywenderlich.com/23996/introduction-to-cocosbuilder

Here are download links to the software I used:

• cocos2d-iphone-2.1-rc1.tar.gz
• Download CocosBuilder Application

Part 1 - Installation and Setup of a Tutorial project for Cocos2D-iOS & CocosBuilder

Part 2 - CocosBuilder resources and first Run of content

Part 3 - Creating a HUD layer CCB

Part 4 - Animating the HUD & making the buttons work

Things I had to leave out

The tutorial was already up to 40 minutes so I had to leave some things out.  Maybe I need to learn how to talk faster.  :-)

Using the UI elements: I imported some UI elements in a plist file in part 1, but never got time to use them.  But if you experiment you should have no trouble in seeing how to use these.  It's pretty much a case of drag and drop.  In some cases you need to use the "Sprite frame" selector in the properties panel on the right-hand-side of CocosBuilder.

Making a layer use the targeted touch handler: I recorded some footage on this, but had to remove it to get the tut down to a reasonable size.  The trick here is that most tutorials and references tell you to implement registerWithTouchHandler in your CCLayer subclass in order to get the kCCTouchesOneAtATime behaviour.

With that behaviour you can use

- (BOOL)ccTouchBegan:(UITouch *)touch withEvent:(UIEvent *)event
{
    CCLOG(@"%@::%@ - %@", NSStringFromClass([self class]), NSStringFromSelector(_cmd), touch);
    CGPoint location = [touch locationInView: [touch view]];
    location = [[CCDirector sharedDirector] convertToGL:location];
    
    BOOL isHandled = NO;
    if (CGRectContainsPoint([self boundingBox], location))
    {
        isHandled = YES;
        // do stuff
    }
    return isHandled;
}

Now it turns out that you can get that by setting the following in CocosBuilder for your CCLayer subclass:

...and since these are set in didLoadFromCCB before the onEnter handler sets up the registration, this will cause the CCLayer's implementation of registerWithTouchHandler to set up your instance for one-at-a-time touches.  Now you can implement your touch handler (as above) and off you go.

Thanks for watching videos - I hope they're useful.  Please - if you spot any mistakes let me know so I can address them.

Happy Cocos building!