Move a virtual rock, with your real muscles

myRock is a demo I created in Unity that uses Myomo's myoelectric sensors. These sensors allow you to use the strength of your real muscles to lift and throw a giant virtual rock with the feeling real world weight and inertia. This is my first attempt at virtual earthbending. I can't show the input device I am using, but it is essentially an armband that reads the muscle activity of one's biceps and triceps. I flex my bicep muscle to lift the rock on the screen, and then flex my tricep to throw it. Below is a little demo of this.


This demo is made in Unity, and the rock is a 3D object with mass and the force of gravity acting on it. When I lift the rock, my lift force is proportional to how hard I flex my biceps. Then when I throw the rock, the pushing force is proportional to how hard I flex my triceps. This means I can lift and throw a rock with the same muscles I would use in real life. And because I am exerting real effort with my muscles, the rock feels like it has real world weight.

Right now, the game is played from a first person perspective, and the rock is thrown wherever the player character is facing. I made some versions of the demo with targets to hit for fun, but accuracy is still difficult at this point, like I imagine it would be lifting and throwing a real boulder. For the future I want to extend this so that any 3D object with collision in unity can be picked up and thrown by the player, and of course, I want to try this in VR.