Gaming for the impaired, a big first step
Bounce is a simple game that does a lot. It is played with the MyoPro, a powered orthotic device made by Myomo and used by patients with impaired arm movement to regain control of their arm. I created the game as a simple implementation of the tools I made to use the MyoPro with the Unity game engine. To create these tools, I took Myomo's existing code base for serial communication with the Myopro, and worked in c++ and c# to create an easy to use Unity plugin that allows any experienced Unity developer to use the Myopro as a game controller.
Using the MyoPro in Unity
When I started on this project, Myomo had efficient c++ code for receiving streaming data from the MyoPro. I wanted to get this data into Unity so I could benefit from using a fully featured game engine. This kind of code interoperability is done with a Unity plugin. With a plugin, I wanted to create Unity-Developer facing code with an easy to use OO design that required as little knowledge about the MyoPro as possible, so that a new Unity developer could come in and start using the MyoPro as a controller without having to learn about bluetooth connections, serial communication, or command bytes.
I created the plugin so that for a Unity developer to connect to the MyoPro, they call a search function which populates a list of mac addresses, then pass that mac address to a connect function, which creates a new MyomoDevice object. Then, this object has a set of C# properties which will always return the latest values from the streaming data being handled in the background. The Unity developer does not need to maintain anything or parse any data; its all done by the plugin. Support for multiple connections in coming, and developers will just create a new MyomoDevice object corresponding to each new connection.
In the background, there is a lot of code that bas been abstracted out to make this possible. It starts with c++ classes in the Unity plugin that handle searching for and connecting to new devices. A new c++ object is created for parsing and storing the streaming data of each device. To expose these objects full of parsed streaming data to c# and Unity, pointers to these objects are passed to c# and wrapped in a c# object that a Unity developer can use. When Unity then asks for a property of this c# wrapper class, the c++ pointer is passed from c# back to c++, where functions can be called off the pointer to retrieve the latest streaming data. This data is then passed back up to the call to the c# property, and Unity gets the latest data from the device. Some properties even parse this data further to keep track of things that are not streamed back from the MyoPro, like the direction of movement of the arm.
developing Bounce for the myopro
The goal of Bounce, is to give MyoPro users an entertaining game that controls just like MyoPro, so that getting better at the game, means getting better at using the MyoPro, which in turn means helping users learn to use their arm again. In Bounce, the user raises their arm in the MyoPro to raise the paddle in the game, and lowers their arm in in the MyoPro to lower the paddle. Then using these controls they bounce a ball back and forth against an AI opponent.