Bot Chocolate

IMG_5932.MOV

Collaborators: James Oubre, Allan Garcia, Shantao Cao, Nick Marks

GitHub: https://github.com/daviddorf2023/BotChocolate

The purpose of this project was to create hot chocolate using the Franka Emika 7 DOF robot arm. To perceive the environment, the system utilizes an Intel D435i camera and AprilTags. Upon initial setup, a calibration process must be completed. After this, the process consisted of using AprilTags to locate a scoop of cocoa, a mug, a spoon, and a kettle relative to the robot. Next, using our custom MoveIt API for Python, movebot, the robot is able to perform path planning between all of these objects. It turns on the kettle, dumps the cocoa powder into the mug, pours the hot water over the power, and stirs the mixture it with the spoon.

Custom MoveIt API

The first step of the project involved creating an API that we could use for controlling the robot using the MoveIt 2 motion planner in ROS 2. We intercepted messages coming from the MoveIt interface in RViz to essentially reverse engineer them for a customizable program that could link motion plan requests into an entire trajectory sequence. Additionally, we implemented features that allow the user to select between generating trajectories around screw axes of the robot joints, or Cartesian trajectories that would keep the end effector orientation constant while moving to a specified XYZ location. Users can also choose between purely planning the robot trajectory for visualization in simulation, or planning and subsequently executing the motion plan generated by the program.

Computer Vision

For this project, we utilized fiducial markers - specifically AprilTags - to gather live data of the position of the objects in the robot's environment. This information was fed to our motion planning node to provide destinations for the end effector to go to, so that it could pick up the kettle, stirrer, and interact with other parts of our setup. The vision component of the project consists of two nodes: april_tf.py and calibration.py.

april_tf.py launches a node that listens to the transformations from the camera_link frame to the AprilTags of the kettle and jig. It then creates and broadcasts transformations to the kettle's button, the cocoa scooper, the cup, and the stirrer. Additionally, it publishes the positions of the scooper, kettle, kettle button, cup, stirrer, and jig to respective topics.

calibration.py file can be run while the april_tf node is running to calibrate the robot using a calibration tag to get the transformation from the camera_link to the base of the robot and save it in a .yaml file to be used when running the entire process later on.

Trajectory Planning

The trajectory package contains the trajectory node, which runs services that interact with the MoveIt API to command the Franka Emika robot's motion. The trajectory node allows the robot to move and manipulate objects in its surroundings to make hot chocolate, or any other hot beverage.

The robot utilizes both screw and Cartesian trajectories to achieve motions like pouring. Returning to predefined waypoints and the home position prevent the robot from reaching joint limits while it moves through a complex trajectory that takes it close to the edge of the robot's workspace. Additionally, obstacles were added into the motion planning simulation in RViz to prevent the robot from generating trajectories that could potentially collide with the table it was on, or other objects in its environment.

Page updated

Google Sites

Report abuse