CISC849 S2016

From class_wiki
Jump to: navigation, search

Course information

Title CISC849 Autonomous Robot Vision
Shortened URL
Description Survey of color camera, depth camera (e.g., Kinect), and laser range-finder-based 2-D and 3-D sensing algorithms for mobile robot navigation and interaction. Focus applications will be humanoid robot perception for disaster response, driverless cars, and trail following
When Mondays and Wednesdays, 11 am-12 pm
Where Smith 341
Instructor Christopher Rasmussen, 446 Smith Hall,
Office hours Tuesdays and Thursdays, 3:30-4:15 pm
  • 20% Oral paper presentation (individual or pairs, 30 minutes)
  • 30% Two programming assignments (individual or pairs)
  • 50% Final project (teams of 1-3)
    • 10% = 2 page proposal, including planned methods, citations of relevant papers, data sources, and division of labor
    • 10% = Joint 20-minute presentation on final results, with accompanying slides
    • 30% = Actual results and estimated effort, factoring in difficulty of problem tackled
Academic policies Programming projects are due by midnight of the deadline day (with a grace period of a few hours afterward...after sunrise is definitely late). A late homework is a 0 without a valid prior excuse. To give you a little flexibility, you have 6 "late days" to use on homeworks to extend the deadline by one day each without penalty. No more than three late days may be used per assignment. Late days will automatically be subtracted, but as a courtesy please notify the instructor in an e-mail of your intention to use late days before the deadline. See submission instructions below.

Students can discuss problems with one another in general terms, but must work independently on programming assignments. This also applies to online and printed resources: you may consult them as references (as long as you cite them), but the words and source code you turn in must be yours alone. The University's policies on academic dishonesty are set forth in the student code of conduct here.

Homeworks Assignment submissions should consist of a directory containing all code (your .cpp files, makefile, etc.), any output data generated (e.g., images, movies, etc.), and an explanation of your approach, what worked and didn't work, etc. contained in a separate text or HTML file. Do not submit executables or .o files, please! The directory you submit for each assignment should be packaged by tar'ing and gzip'ing it or just zip'ing it. The resulting file should be submitted through Sakai.

You may develop your C/C++ code in any fashion that is convenient--that is, with any compiler and operating system that you want. However, we will be grading your homework on a Linux system with a makefile, and so you must avoid OS- and hardware-specific functions and provide a makefile for us that will work (like one of the templates above).

Possible Papers to Present (not a complete list)

  • "Vision and Learning for Deliberative Monocular Cluttered Flight", D. Dey, K. Shankar, et al., FSR 2015. UAV, obstacle avoidance
  • "Pushbroom Stereo for High-Speed Navigation in Cluttered Environments", A. Barry and R. Tedrake, ICRA 2015. UAV, obstacle avoidance
  • "Collaborative mapping of an earthquake-damaged building via ground and aerial robots", N. Michael et al., JFR 2012. UAV, UGV, disaster, mapping
  • "Vision Based Victim Detection from Unmanned Aerial Vehicles", M. Andriluka et al., IROS 2010. UAV, person detection
  • "Biped Navigation in Rough Environments using On-board Sensing", J. Chestnutt, Y. Takaoka, K. Suga, K. Nishiwaki, J. Kuffner, and S. Kagami, IROS 2009. Footstep planning, ladar, plane fitting
  • "Real-Time Navigation in 3D Environments Based on Depth Camera Data", D. Maier, A. Hornung, and M. Bennewitz, Humanoids 2012. RGB-D, localization, mapping, planning
  • "Robotic Grasping of Novel Objects using Vision", A. Saxena, J. Driemeyer, A. Ng, IJRR 2008. Grasping, learning
  • "Self-supervised Monocular Road Detection in Desert Terrain", H. Dahlkamp, A. Kaehler, D. Stavens, S. Thrun, and G. Bradski, 2006. DARPA GC, color similarity, segmentation
  • "Multi-Sensor Lane Finding in Urban Road Networks", A. Huang, D. Moore, M. Antone, E. Olson, S. Teller, RSS 2008. DARPA UC, edge detection, robust curve fitting, tracking
  • "High fidelity day/night stereo mapping with vegetation and negative obstacle detection for vision-in-the-loop walking", M. Bajracharya et al., IROS 2013. LS3, dense stereo depth, visual odometry
  • "Autonomous Door Opening and Plugging In with a Personal Robot", W. Meeussen et al., IROS 2010. PR2, grasping


Both of the following libraries are available for Linux, Windows, and Mac OS. For (relative) simplicity, we will not be installing ROS (which includes both)
  • PCL (>= 1.6, available for Linux, Windows, Mac OS X)
  • OpenCV (3.1)


Note: The blue squares in the "#" column below indicate Tuesdays.

# Date Topic Links/Readings/videos Assignments/slides
1 Feb. 9 Background slides
2 Feb. 11 Finish background; introduction to the DARPA Robotics Challenge "How South Korea's DRC-HUBO Robot Won the DARPA Robotics Challenge", IEEE Spectrum, June 9, 2015 slides
3 Feb. 16 DRC algorithm components slides
4 Feb. 18

Register/add deadline Feb. 24

PCL tutorial RANSAC background slides
5 Feb. 23 Plane/obstacle/object segmentation (3-D), clustering, ICP slides
HW #1
6 Feb. 25 NOVA "Rise of the Robots" Next generation Atlas
7 Mar. 1 Finish ICP/registration; trail-following overview slides
PCL registration
8 Mar. 3 Trail-following algorithmic components slides
HW #1 due
9 Mar. 8 Tracking, localization Thrun particle filtering slides slides
10 Mar. 10 Particle filtering OpenCV installation slides
HW #2
11 Mar. 15 Intro to OpenCV Mats, drawing, background subtraction, thresholding and erosion/dilation slides
12 Mar. 17 More on OpenCV; object recognition (sample paper) Connected components, bounding rect, tracker API; "Real-Time Human Pose Recognition in Parts from Single Depth Images", J. Shotton et al., CVPR 2011. Recognition, classification, RGB-D slides
13 Mar. 22 Finish Kinect body part recognition paper
14 Mar. 24 Survey paper choices slides
Paper presentation choice due Friday, March 25
HW #2 due
Mar. 29 NO CLASS
Spring break
Mar. 31 NO CLASS
Spring break
15 Apr. 5 Intro. to motion planning slides
16 Apr. 7 NO CLASS
Instructor away
Apr. 12 Student paper presentations C. Rasmussen, "Learning Locomotion over Rough Terrain using Terrain Templates", M. Kalakrishnan, J. Buchli, P. Pastor, and S. Schaal, IROS 2009

K. Eckenhoff, "Collaborative mapping of an earthquake-damaged building via ground and aerial robots"

Project proposal due

Rasmussen slides

17 Apr. 14 Student paper presentations P. Cao, "Vision and learning for Deliberative Monocular Cluttered Flight"
S. Veer, "Biped Navigation in Rough Environments using On-board Sensing"
18 Apr. 19

Withdraw deadline

Student paper presentations S. Khan, "Self-supervised monocular Road Detection in Desert Terrain"
M. Kaplan, "Linear Auto-Calibration..."
19 Apr. 21 Student paper presentations M. Zhou, "Pushbroom Stereo for High-Speed Navigation..."
W. Treible and K. Corder, "Real-Time Navigation in 3D Environments Based on Depth Camera Data"
20 Apr. 26 Mid-project review
21 Apr. 28 Mapping SLAM demos: Pirobot, MIT, Darmstadt "Hector" mapping
"Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels", N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007
DepthX slides

Thrun FastSLAM slides (grids from slide 29)
Accompanying Stachniss lecture

22 May 3 DARPA Urban Challenge Long video, highlights only, Stanford clips Team presentations from ICRA 2008 workshop
23 May 5 Final project review
24 May 10 "Bonus" material
25 May 12 NO CLASS
Finish your project!
26 May 17 Final project presentations -- class starts at 10 am Final project due