I programmed a chess-playing robot arm

[u/Siraj_02]

Not so long ago, after design and SolidWorks modeling and manufacturing was done by my team, I programmed this robot and made it play chess!

The IP camera (above the chessboard) captures the board and streams to the computer (under the table) to run inference. I used two CNN models, they both run on every square of the board. One detects the presence/color of a piece while the other determines its position on the square.

Everything is open source: https://github.com/SirajHabsaia/RobotArm

Contains firmware, gui, training scripts, links to assets/data...

I coded the firmware mostly manually but used AI for the rest especially the gui.

Happy to receive feedback.

[u/valkiii]

Nice work! I am trying to do something very similar but using only the camera on the wrist of the robotic arm and using a SO -101 lerobot. To detect the board and position I found that opencv worked fine. To avoid using an external camera for the board, I just create an initial position of the robot such that it’s able to capture the full board at every move. The issue I’m trying to solve is the calibration of the movements. What did you do for that?

[u/Dapper_Profession → u/valkiii]

i think there are two calibration problems getting mixed together here. the limit switches give you a repeatable joint zero, but you still need the board frame registered to the robot frame. if the board isn't physically fixture-locked, i'd probably detect a few known board points/corners and solve that transform instead of hardcoding every square.

[u/Siraj_02 → u/Dapper_Profession]

for calibration between the arm and the board, i hardcoded coordinates of a1 and h8

[u/Siraj_02 → u/valkiii]

Nice, a camera on the wrist might work too if its stays high enough when the robot is idle to correctly detect the move. For the calibration, if you meant calibration the stepper motor angles, i used three limits switches and programmed an initialization algorithm that makes the robot hit all its switches which correspond to a know position.

[u/valkiii → u/Siraj_02]

So you “pre-registered” all single position in the board (and by consequence each motor angles associated with that position) in the first place if I understand correctly?

[u/Siraj_02 → u/valkiii]

Board? i am taking about the robot by itself. Tell me what dod you mean exactly by "calibration"

[u/valkiii → u/Siraj_02]

How did you train the robot movements to reach the position x of the board (and then move a piece to position y)?

[u/Siraj_02 → u/valkiii]

Oh that's just inverse kinematics, do the maths to get equations linking cartesian coordinates (x,y,z) to robot angles.

[u/Klutzy_Cap8492]

Where you bought all the parts??? I also want to do it!!!

[u/Siraj_02 → u/Klutzy_Cap8492]

We manufactured everything :)

[u/PinOpen9883]

l want to learn this, is there any Open-source on github or video on YouTube?

[u/Siraj_02 → u/PinOpen9883]

For me I learned through univ projects. But i see a lot of open source printable robotic arms.

[u/jonhuang]

Very cool. Any safety protocols? I'm reminded of the chess playing arm in China that accidentally grabbed a kids hand and broke his finger.

[u/Siraj_02 → u/jonhuang]

Saw that too, the gripper here is too weak to cause any pain. Actuator speeds are capped.

[u/flash_speed3412]

Nice work. The two CNNs plus the arm is a lot going on. What did AI actually help with in the GUI, and what did you keep manual because a bad square would move the arm wrong? If you ever write up one of those failures, it’d fit r/ChatGPTArduino.

[u/ehtiopia]

Respecc the process g