Difference between revisions of "CISC849 F2019"

From class_wiki
Jump to: navigation, search
(→‎Course information)
(→‎Schedule)
Line 112: Line 112:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|1
 
|style="background:rgb(102, 204, 255)"|1
−
|Feb. 6
+
|Aug. 27
 
|Background  
 
|Background  
 
|
 
|
Line 118: Line 118:
 
|-
 
|-
 
|2
 
|2
−
|Feb. 8
+
|Aug. 29
 
|Finish background
 
|Finish background
 
|
 
|
Line 124: Line 124:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|3
 
|style="background:rgb(102, 204, 255)"|3
−
|Feb. 13
+
|Sep. 3
 
|Introduction to DARPA Urban Challenge (UC), Robotics Challenge (DRC) and algorithm components
 
|Introduction to DARPA Urban Challenge (UC), Robotics Challenge (DRC) and algorithm components
 
|
 
|
Line 134: Line 134:
 
|-
 
|-
 
|4
 
|4
−
|Feb. 15
+
|Sep. 5
 
|Introduction to ROS; PCL tutorial
 
|Introduction to ROS; PCL tutorial
 
|
 
|
Line 143: Line 143:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|5
 
|style="background:rgb(102, 204, 255)"|5
−
|Feb. 20<br>''Register/add deadline Feb. 19''
+
|Sep. 10<br>''Register/add deadline''
 
|Plane/obstacle/object segmentation (3-D)
 
|Plane/obstacle/object segmentation (3-D)
 
|[https://en.wikipedia.org/wiki/Random_sample_consensus RANSAC background]
 
|[https://en.wikipedia.org/wiki/Random_sample_consensus RANSAC background]
Line 149: Line 149:
 
|-
 
|-
 
|6
 
|6
−
|Feb. 22
+
|Sep. 12
 
|Finish plane segmentation, clustering, ICP
 
|Finish plane segmentation, clustering, ICP
 
|<!--"Autonomous Door Opening and Plugging In with a Personal Robot", W. Meeussen ''et al.'', IROS 2010<br>"Biped Navigation in Rough Environments using On-board Sensing", J. Chestnutt, Y. Takaoka, K. Suga, K. Nishiwaki, J. Kuffner, and S. Kagami, IROS 2009-->
 
|<!--"Autonomous Door Opening and Plugging In with a Personal Robot", W. Meeussen ''et al.'', IROS 2010<br>"Biped Navigation in Rough Environments using On-board Sensing", J. Chestnutt, Y. Takaoka, K. Suga, K. Nishiwaki, J. Kuffner, and S. Kagami, IROS 2009-->
Line 156: Line 156:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|7
 
|style="background:rgb(102, 204, 255)"|7
−
|Feb. 27
+
|Sep. 17
 
|Image classification background<!--Clustering, normal estimation-->
 
|Image classification background<!--Clustering, normal estimation-->
 
|
 
|
Line 163: Line 163:
 
|-
 
|-
 
|8
 
|8
−
|Mar. 1
+
|Sep. 19
 
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Instructor away''<!--, Registration, features  -->
 
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Instructor away''<!--, Registration, features  -->
 
|
 
|
Line 169: Line 169:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|9
 
|style="background:rgb(102, 204, 255)"|9
−
|Mar. 6
+
|Sep. 24
 
|More classification background
 
|More classification background
 
|
 
|
Line 175: Line 175:
 
|-
 
|-
 
|10
 
|10
−
|Mar. 8
+
|Sep. 26
 
|<!--Object recognition (sample paper)-->Introduction to convolutional neural networks
 
|<!--Object recognition (sample paper)-->Introduction to convolutional neural networks
 
|<!--[http://research.microsoft.com/pubs/145347/bodypartrecognition.pdf "Real-Time Human Pose Recognition in Parts from Single Depth Images"], J. Shotton ''et al.'', CVPR 2011.  ''Recognition, classification, RGB-D''-->
 
|<!--[http://research.microsoft.com/pubs/145347/bodypartrecognition.pdf "Real-Time Human Pose Recognition in Parts from Single Depth Images"], J. Shotton ''et al.'', CVPR 2011.  ''Recognition, classification, RGB-D''-->
Line 181: Line 181:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|11
 
|style="background:rgb(102, 204, 255)"|11
−
|Mar. 13
+
|Oct. 1
 
|<!--Localization-->Finish detection & segmentation background, introduction to TensorFlow library
 
|<!--Localization-->Finish detection & segmentation background, introduction to TensorFlow library
 
|[https://arxiv.org/abs/1612.08242 YOLOv2, Redmon and Farhadi] (CVPR 2017)
 
|[https://arxiv.org/abs/1612.08242 YOLOv2, Redmon and Farhadi] (CVPR 2017)
Line 187: Line 187:
 
|-
 
|-
 
|12
 
|12
−
|Mar. 15
+
|Oct. 8
 
|<!--Particle filters and localization-->More on TensorFlow programming
 
|<!--Particle filters and localization-->More on TensorFlow programming
 
|<!--[http://www2.informatik.uni-freiburg.de/~wurm/papers/hornung10iros.pdf Humanoid Robot Localization in Complex Indoor Environments], A. Hornung, K. Wurm, M. Bennewitz, IROS 2010-->
 
|<!--[http://www2.informatik.uni-freiburg.de/~wurm/papers/hornung10iros.pdf Humanoid Robot Localization in Complex Indoor Environments], A. Hornung, K. Wurm, M. Bennewitz, IROS 2010-->
 
|<!--[http://robots.stanford.edu/probabilistic-robotics/ppt/particle-filters.ppt Thrun particle filtering slides]-->
 
|<!--[http://robots.stanford.edu/probabilistic-robotics/ppt/particle-filters.ppt Thrun particle filtering slides]-->
 
|-
 
|-
−
|style="background:rgb(102, 204, 255)"|13
+
|style="background:rgb(102, 204, 255)"|
−
|Mar. 20
+
|Oct. 10
 
|<!--Motion planning background-->Perception for stepping
 
|<!--Motion planning background-->Perception for stepping
 
|"Learning Locomotion over Rough Terrain using Terrain Templates", M. Kalakrishnan, J. Buchli, P. Pastor, and S. Schaal, IROS 2009
 
|"Learning Locomotion over Rough Terrain using Terrain Templates", M. Kalakrishnan, J. Buchli, P. Pastor, and S. Schaal, IROS 2009
Line 199: Line 199:
 
|-
 
|-
 
|14
 
|14
−
|Mar. 22
+
|Oct. 15
 
|style="background:rgb(255, 102, 0)"|NO CLASS<!--Mapping-->
 
|style="background:rgb(255, 102, 0)"|NO CLASS<!--Mapping-->
 
|<!--[http://wiki.ros.org/gmapping gmapping] demos: [http://www.youtube.com/watch?v=7iIDdvCXIFM Pirobot], [http://www.youtube.com/watch?v=_jwBKo0SXng MIT]<br>[http://www.youtube.com/watch?v=F8pdObV_df4 Darmstadt "Hector" mapping]<br>
 
|<!--[http://wiki.ros.org/gmapping gmapping] demos: [http://www.youtube.com/watch?v=7iIDdvCXIFM Pirobot], [http://www.youtube.com/watch?v=_jwBKo0SXng MIT]<br>[http://www.youtube.com/watch?v=F8pdObV_df4 Darmstadt "Hector" mapping]<br>
Line 206: Line 206:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|
 
|style="background:rgb(102, 204, 255)"|
−
|Mar. 27
+
|Oct. 17
−
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Spring break''
+
|
 
|
 
|
 
|
 
|
 
|-
 
|-
 
|
 
|
−
|Mar. 29
+
|Oct. 22<br>''Withdraw deadline''
−
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Spring break''
+
|
 
|
 
|
 
|
 
|
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|15
 
|style="background:rgb(102, 204, 255)"|15
−
|Apr. 3
+
|Oct. 24
 
|Imitation learning
 
|Imitation learning
 
|
 
|
Line 228: Line 228:
 
|-
 
|-
 
|16
 
|16
−
|Apr. 5
+
|Oct. 29
 
|Reinforcement learning
 
|Reinforcement learning
 
|
 
|
Line 234: Line 234:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|17
 
|style="background:rgb(102, 204, 255)"|17
−
|Apr. 10<br>
+
|Oct. 31
−
''Withdraw deadline Apr. 9''
 
 
|Student paper presentation
 
|Student paper presentation
 
|
 
|
−
* Peng Su, [https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Yeung_End-To-End_Learning_of_CVPR_2016_paper.pdf "End-to-end Learning of Action Detection from Frame Glimpses in Videos"]
 
 
|
 
|
 
|-
 
|-
 
|18
 
|18
−
|Apr. 12
+
|Nov. 5
 
|Student paper presentation
 
|Student paper presentation
 
|
 
|
−
* Patrick Geneva, [http://openaccess.thecvf.com/content_cvpr_2017/papers/Dong_Visual-Inertial-Semantic_Scene_Representation_CVPR_2017_paper.pdf "Visual-Inertial-Semantic Scene Representation for 3D Object Detection"]
 
 
|''Project proposal due Sunday, April 15''
 
|''Project proposal due Sunday, April 15''
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|19
 
|style="background:rgb(102, 204, 255)"|19
−
|Apr. 17<br>
+
|Nov. 7
 
|Student paper presentations
 
|Student paper presentations
 
|
 
|
−
* Zengxiang Lu, "Image Generation from Scene Graphs"
 
−
* Aishwarya Dang, [http://ieeexplore.ieee.org/document/7353481/ "VoxNet: A 3D Convolutional Neural Network for real-time object recognition"]
 
 
|
 
|
 
|-
 
|-
 
|20
 
|20
−
|Apr. 19
+
|Nov. 12
 
|Student paper presentations
 
|Student paper presentations
 
|
 
|
−
* Sumeet Gupta, [https://arxiv.org/abs/1704.05588 "Learning to Fly by Crashing"]
 
−
* Yulin Yang, [http://rpg.ifi.uzh.ch/docs/RAL18_Loquercio.pdf "DroNet: Learning to Fly by Driving"]
 
 
|
 
|
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|21
 
|style="background:rgb(102, 204, 255)"|21
−
|Apr. 24
+
|Nov. 14
 
|Student paper presentation
 
|Student paper presentation
 
|
 
|
−
Matt Schmittle, [https://arxiv.org/pdf/1509.06825.pdf Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours]
 
 
|
 
|
 
|-
 
|-
 
|22
 
|22
−
|Apr. 26
+
|Nov. 19
 
|Student paper presentations
 
|Student paper presentations
 
|
 
|
−
* Apoorva Patil, [http://www.cs.ubc.ca/~van/papers/2017-TOG-deepLoco/ DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning]
 
−
* Leighanne Hsu, [http://www2.informatik.uni-freiburg.de/~hornunga/pub/maier12humanoids.pdf Real-Time Navigation in 3D Environments Based on Depth Camera Data]
 
 
|<!--[https://docs.google.com/presentation/d/19Wm-H2ttW_m5pan_lCXDzmk468gLoud6Xug_2_i0mWI/edit?usp=sharing&authkey=CKmYmJYI slides]-->
 
|<!--[https://docs.google.com/presentation/d/19Wm-H2ttW_m5pan_lCXDzmk468gLoud6Xug_2_i0mWI/edit?usp=sharing&authkey=CKmYmJYI slides]-->
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|23
 
|style="background:rgb(102, 204, 255)"|23
−
|May 1
+
|Nov. 21
 
|Student paper presentation
 
|Student paper presentation
−
|Yiqun Jia, [https://arxiv.org/pdf/1603.03833.pdf From Virtual Demonstration to Real-World Manipulation Using LSTM and MDN]
+
|  
 
|
 
|
 
|-
 
|-
 
|24
 
|24
−
|May 3
+
|Nov. 25
 
|Instructor paper presentation
 
|Instructor paper presentation
 
|Rasmussen, [http://www.ri.cmu.edu/publication_view.html?pub_id=5590 "Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels"], N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007
 
|Rasmussen, [http://www.ri.cmu.edu/publication_view.html?pub_id=5590 "Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels"], N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007
Line 292: Line 282:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|25
 
|style="background:rgb(102, 204, 255)"|25
−
|May 8
+
|Nov. 28
 
|Final project review
 
|Final project review
 
|
 
|
Line 298: Line 288:
 
|-
 
|-
 
|
 
|
−
|May 10
+
|Dec. 3
 
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Work on projects...''
 
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Work on projects...''
 
|
 
|
Line 304: Line 294:
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|26
 
|style="background:rgb(102, 204, 255)"|26
−
|May 15
+
|Dec. 5
 
|Final project presentations part I
 
|Final project presentations part I
−
|
 
−
|
 
−
|-
 
−
|27
 
−
|May 16
 
−
|Final project presentations part II
 
−
|
 
−
|
 
−
|-
 
−
|28
 
−
|May 21
 
−
|Final project presentations part III
 
 
|
 
|
 
|
 
|
 
|}
 
|}

Revision as of 09:25, 27 August 2019

Course information

Title CISC849 Robot Vision and Learning
Description Survey of image-based 2-D and 3-D sensing algorithms for mobile robot navigation and interaction, including motion estimation, obstacle segmentation, terrain modeling, and object recognition, with a particular focus on deep learning techniques to dramatically improve performance.
When Tuesdays and Thursdays, 2-3:15 pm
Where Gore 317
Instructor Christopher Rasmussen, 446 Smith Hall, cer@cis.udel.edu
Office hours Mondays and Wednesdays, 2-3 pm
Grading
  • 20% Oral paper presentation (individual or pairs, 30 minutes)
  • 30% Two programming assignments (individual)
  • 50% Final project (teams of 1-3)
    • 10% = 2 page proposal, including planned methods, citations of relevant papers, data sources, and division of labor
    • 10% = Joint 15-minute presentation on final results, with accompanying slides
    • 30% = Actual results and estimated effort, factoring in difficulty of problem tackled
Academic policies Programming projects are due by midnight of the deadline day (with a grace period of a few hours afterward...after sunrise is definitely late). A late homework is a 0 without a valid prior excuse. To give you a little flexibility, you have 6 "late days" to use on homeworks to extend the deadline by one day each without penalty. No more than three late days may be used per assignment. Late days will automatically be subtracted, but as a courtesy please notify the instructor in an e-mail of your intention to use late days before the deadline. See submission instructions below.

Students can discuss problems with one another in general terms, but must work independently on programming assignments. This also applies to online and printed resources: you may consult them as references (as long as you cite them), but the words and source code you turn in must be yours alone. The University's policies on academic dishonesty are set forth in the student code of conduct here.

Homeworks Assignment submissions should consist of a directory containing all code (your .cpp files, makefile, etc.), any output data generated (e.g., images, movies, etc.), and an explanation of your approach, what worked and didn't work, etc. contained in a separate text or HTML file. Do not submit executables or .o files, please! The directory you submit for each assignment should be packaged by tar'ing and gzip'ing it or just zip'ing it. The resulting file should be submitted through Canvas.

You may develop your C/C++ code in any fashion that is convenient--that is, with any compiler and operating system that you want. However, we will be grading your homework on a Linux system with a makefile, and so you must avoid OS- and hardware-specific functions and provide a makefile for us that will work (like one of the templates above).

Possible Papers to Present (not a complete list)

Instructions for Homeworks

Software
Note
  • TensorFlow is available for multiple operating systems, but the processing demands can be high, especially if your computer does not have a new-ish Nvidia GPU
  • If you have significant problems installing/using PCL or TensorFlow on your own computer, I have a GPU workstation in my lab that can be used for selected training jobs. Contact me *after* you have determined that your own machine is insufficient

Schedule

Note: The blue squares in the "#" column below indicate Tuesdays.

# Date Topic Links/Readings/videos Assignments/slides
1 Aug. 27 Background slides
2 Aug. 29 Finish background slides
3 Sep. 3 Introduction to DARPA Urban Challenge (UC), Robotics Challenge (DRC) and algorithm components slides
4 Sep. 5 Introduction to ROS; PCL tutorial

ARGOS challenge overview (8:00)

slides
Sample PCL programs (these require ROS)
ETH ROS mini course (in particular: overview, RViz, TF), ETH case study "ANYmal at the ARGOS Challenge"

5 Sep. 10
Register/add deadline 		Plane/obstacle/object segmentation (3-D) RANSAC background HW #1
plane_fit.cpp (No ROS required)
6 Sep. 12 Finish plane segmentation, clustering, ICP slides
7 Sep. 17 Image classification background Stanford CS231n Image classification slides (6-61),
8 Sep. 19 NO CLASS
Instructor away
HW #1 due
9 Sep. 24 More classification background Loss/optimization slides (8-10, 37-77, finish with web demo), Intro to NN (83-88)
10 Sep. 26 Introduction to convolutional neural networks Stanford CS231n Convolutional Neural Networks slides (15-63, 71-78), CNN architectures (8-36), Detection and Segmentation slides (15-27)
11 Oct. 1 Finish detection & segmentation background, introduction to TensorFlow library YOLOv2, Redmon and Farhadi (CVPR 2017) Stanford CS231n Detection and Segmentation slides (44-49, 53-85), deep learning software slides (21-33, 40-59)
TensorFlow code sample from Stanford lecture
HW #2
12 Oct. 8 More on TensorFlow programming
Oct. 10 Perception for stepping "Learning Locomotion over Rough Terrain using Terrain Templates", M. Kalakrishnan, J. Buchli, P. Pastor, and S. Schaal, IROS 2009 slides
DeepLoco highlights video (6:23)
14 Oct. 15 NO CLASS HW #2 due Friday, March 23
Oct. 17
Oct. 22
Withdraw deadline
15 Oct. 24 Imitation learning Berkeley Levine course IL slides (DAgger defined on slide 14); CMU Fragkiadaki IL slides (1-29)
Paper presentation choice due
16 Oct. 29 Reinforcement learning UCL Silver course: RL slides (7-42), RL video (15:40-21:20, 1:02:13-1:04:40 contains movies missing from PDF); Stanford deep-Q lecture slides (23-62); original deep-Q paper
17 Oct. 31 Student paper presentation
18 Nov. 5 Student paper presentation Project proposal due Sunday, April 15
19 Nov. 7 Student paper presentations
20 Nov. 12 Student paper presentations
21 Nov. 14 Student paper presentation
22 Nov. 19 Student paper presentations
23 Nov. 21 Student paper presentation
24 Nov. 25 Instructor paper presentation Rasmussen, "Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels", N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007 DepthX slides
25 Nov. 28 Final project review
Dec. 3 NO CLASS
Work on projects...
26 Dec. 5 Final project presentations part I
Retrieved from "http://nameless.cis.udel.edu/class_wiki/index.php?title=CISC849_F2019&oldid=1593"