Project Background
The 3D Machine Vision Chinese Chess Robot was not only an image-recognition project. The system had to complete piece recognition, board-coordinate mapping, game decision, and robotic-arm move execution, finally forming a closed loop from visual input to physical action.
The main difficulty was that algorithm output had to land in the real world. YOLOv5 can output piece classes and image coordinates, but the robotic arm needs executable spatial positions. If any mapping step is unstable, the final piece placement will drift.
Core Problem
The core problem was error propagation. Detection-box position, board-grid mapping, arm movement control, and actual piece placement all introduce error. Each part may look acceptable alone, but after they are combined, the placement offset can become visible.
Another issue was loop verification. Proving that a model can recognize pieces is not enough. The recognition result must drive a decision, the decision must become a target coordinate, and that coordinate must be executed stably by the arm.
My Approach
I split the system into four nodes: visual recognition, coordinate mapping, game decision, and arm execution. The visual part used YOLOv5 for piece detection. The coordinate part used nine-point calibration to build the mapping from image coordinates to board coordinates. The decision part connected Alpha-Beta pruning. The execution part controlled the robotic arm with PWM.
To reduce placement error, I added curve-fitting compensation after coordinate mapping. This avoided directly trusting the center of the detection box. Calibration, real arm movement, and error correction were placed on the same verification path.
Debugging and Verification
During debugging, I did not only look at model inference. I verified the loop step by step. First, I checked whether piece class and position were stable. Second, I checked whether board coordinates after nine-point calibration were consistent. Third, I checked whether decision output could be converted into a target position. Finally, I observed whether the arm could repeatedly reach the same position.
When a placement offset appeared, the cause had to be separated into recognition error, mapping error, or execution error. If the detection box was stable but the landing point was offset, the problem was more likely in mapping or arm compensation. If recognition itself fluctuated, later control accuracy could not make the system stable.
Final Result
With nine-point calibration and curve-fitting compensation, the system reached +/-0.5 mm repeat positioning accuracy and controlled inference response within 300 ms. The project received national undergraduate innovation-program approval and an invention patent.
This result shows that the project was not only a model call. It connected vision, game logic, and mechanical execution into a practical engineering loop.
Review Takeaways
When a vision project lands in robotics, the model is only one part of the chain. The key is turning “recognized” into “executed reliably.” That requires every intermediate result to be observable and verifiable, instead of attributing all problems to model accuracy.
For similar projects, I would continue splitting the system into observable nodes: whether input is stable, mapping is reproducible, control has compensation, and final action meets expectation. This structure makes complex system debugging much more controllable.