Detecting Objects from a Rosbag Recording using YOLO

A walkthrough on running a YOLO-trained object detector with resulting bounding boxes on a video file. Assumes video feed comes through a ROS topic.

Note: This is based on a pre-trained COCO dataset that includes 80 image classifications. A tutorial on custom training will be added soon

Note: This example uses camera/rgb/image_raw as the topic for the desired image data. Replace as needed

Requirements

• Ubuntu 16.0.4
• ROS
• Python 3+
• Install image_view:sudo apt-get install ros-<distro>-image-view
• Install mjpegtools: sudo apt-get install mjpegtools

Steps

Clone detector repo and setup image_view

1. export ROS_MASTER_URI=http://<your_server>:11311
2. roscore
3. git clone https://github.com/kalyco/yolo_detector.git
4. Grab the weight file here and place it in yolo-coco/

Prerecord rosbag to .bag file

• mkdir ~/rosbags

Run a script to obtain rosbag data:

rosbag record -b 0 --split --duration=2m -O camera_front.bag /camera/rgb/image_raw

Use image_view to export rosbag data to JPEGs

1. roscd image_view
2. rosmake image_view
3. Create an export.launch file: sudo vi export.launch
4. Paste the following:

<launch>
<node pkg="rosbag" type="play" name="rosbag" required="true" args="/home/<USER>/<camera_<N>>.bag"/>
<node name="extract" pkg="image_view" type="extract_images"respawn="false" required="true" output="screen" cwd="ROS_HOME">
  <remap from="image"to="/camera/rgb/image_raw"/>
</node>
</launch>

5. roslaunch export.launch

— this will dump the images named frame%04d.jpg into the ~/.ros dir

may take some time

Optional 1: move to test directory and add aliases

cd ~ && mkdir <dir_name> && mv ~/.ros/frame*.jpg <dir_name>/

Optional 2: create aliases for repeated use

# My aliases
alias newframes='cd ~ && rm -rf frames && mkdir frames'
alias mvframes='cd ~/.ros && mv frame*.jpg ../frames && cd ../frames'
alias mencode='mencoder "mf://*.jpg" -mf type=jpg:fps=15 -o output.mpg -speed 1 -ofps 30 -ovc lavc -lavcopts vcodec=mpeg2video:vbitrate=2500 -oac copy -of mpeg'

Optional 3: If the bagfile contains compressed images (or you are seeing a mismatch in the frame series), they will need to be decompressed before exporting. call:

rosrun image_transport republish compressed in:=camera/rgb/image_raw raw out:=camera_out/image

and change line 4 of the launch.export file to

<remap from=’image’ to=”/camera_out/image”/>

Generate MPGs directly from JPEGs

Note: -f is for frames per second (FPS), in this example we use 15

1. cd <~/.ros or ~/frames>
2. run:

mencoder “mf://*.jpg” -mf type=jpg:fps=15 -o output.mpg -speed 1 -ofps 30 -ovc lavc -lavcopts vcodec=mpeg2video:vbitrate=2500 -oac copy -of mpeg

Detect!

1. mv output.mpg <cloned_detector_path>/videos
2. cd <cloned_detector_path>
3. python yolo_video.py --input videos/output.mpg --output output/vid_output.avi --yolo yolo-coco

And that’s pretty much it! I recommend using VLC to check the video.

Troubleshooting

Error writing: Error opening file: camera_front_<n>.bag.active

An unfinished rosbag:

rosbag reindex camera_front_<N>.bag.active

May need to change the name for the recording if you are calling the command multiple times:

rosbag record /camera/depth/points -O Camera.bag

Problems with ros nodes already running?

This sometimes comes up during recording or conversion where you get errors about ros nodes running in the background:

Try any and/or all of the following before retrying your command

• rosnode kill -all
• sudo systemctl stop docker
• sudo service docker stop
• rosnode cleanup

Any other issues shoot me a message and I’ll try to help!

Related articles/links

Exporting rosbag to video

YOLO Detector