How to Output your IMU/GPS Data on your Robot's POV Video Stream

Looking to recreate a Heads-Up-Display (HUD) effect on your robot’s POV video stream? Stephen Okay, Roboticist in Residence at Circuit Launch, did just that and provides simple step-by-step details on how you can do the same.

For starters, Stephen’s robot platform goes by the name Tenacity. Tenacity is mostly 3D-printed and was inspired by the Mars rovers Curiosity and Perseverance.

The camera setup that provides the POV visualization is an Orbbec Astra Pro with a structured-light sensor and RGB camera. For this project, Stephen only used the RGB camera. The text displayed in the HUD comes from live ROS topic data, and is generated by a Parker LORD 3DMGX5-GNSS/INS. The onscreen battery data comes from an Arduino-based battery monitor.

To get the information to create the actual HUD effect, all you need to do is subscribe to the ROS /camera/rgb/image_raw camera topic, the ros_mscl node to read the IMU messages, and ros_battery_monitor to read the battery monitor. Since the ROS image format isn’t ideal for us, images from the /camera/rgb/image_raw topic are run through the cv_bridge Python module to convert them from ROS Image format to OpenCV.

To get the useful information, there’s a bit of string processing that needs to be done on the data that comes from the subscribed topics. Once that’s done, send it to a series of OpenCV cv2.putText() commands which puts the topic data right over the image. This can then either be pushed out to another video stream for real-time heads-up POV monitoring of the rover’s activity or saved to an MP4 movie file for further analysis.

You can read Stephen’s original writeup here for more information. Learn more about how our ROS support can accelerate your success by visiting our ROS driver portal.

Related Products

Related Software

For more information, please contact our sales team