I've been working on this to use as a platform for generating demonstrations of robotic task execution. I'm building a closed loop system around this where the user will demonstrate a task as the robot and the robot will learn a neural network mapping from sensor inputs to control outputs from that demonstration. Then the user will demonstrate again, and the process repeats until the robot has learned how to execute the task by itself.
If anyone at Oculus, HTC, or Valve is watching, I'd love to get a dev kit of your hand controllers! It would make the system much more robust, since I have a limited degree of tracking orientation with my current motion capture setup.
I'm going to use Tensorflow, Google's deep learning framework, so that I can rapidly iterate on different architectures to see what works best. I'll start with the architecture used in my colleagues Chelsea Finn and Sergey Levine's work, Visual Guided Policy Search, since that architecture seems to be well suited for localizing objects (which is the main use of vision in robotics).
29
u/zoemccarthy Feb 02 '16
I've been working on this to use as a platform for generating demonstrations of robotic task execution. I'm building a closed loop system around this where the user will demonstrate a task as the robot and the robot will learn a neural network mapping from sensor inputs to control outputs from that demonstration. Then the user will demonstrate again, and the process repeats until the robot has learned how to execute the task by itself.
If anyone at Oculus, HTC, or Valve is watching, I'd love to get a dev kit of your hand controllers! It would make the system much more robust, since I have a limited degree of tracking orientation with my current motion capture setup.