Difference between revisions of "CISC849 S2018"

From class_wiki
Jump to: navigation, search
(→‎Schedule)
(→‎Schedule)
 
(136 intermediate revisions by the same user not shown)
Line 15: Line 15:
 
|-
 
|-
 
|valign="top"|'''When'''
 
|valign="top"|'''When'''
−
|Tuesdays and Thursdays, 11-12:15 pm
+
|Tuesdays and Thursdays, 11 am-12:15 pm
 
|-
 
|-
 
|valign="top"|'''Where'''
 
|valign="top"|'''Where'''
Line 24: Line 24:
 
|-
 
|-
 
|valign="top"|'''Office hours'''
 
|valign="top"|'''Office hours'''
−
|Tuesdays and Thursdays, 3:30-4:15 pm  
+
|Wednesdays, 10 am -- 12 pm
 
|-
 
|-
 
|valign="top"|'''Grading'''  
 
|valign="top"|'''Grading'''  
Line 49: Line 49:
 
==Possible Papers to Present (not a complete list)==
 
==Possible Papers to Present (not a complete list)==
  
−
* "Collaborative mapping of an earthquake-damaged building via ground and aerial robots", N. Michael ''et al.'', JFR 2012. ''UAV, UGV, disaster, mapping''
+
* [https://arxiv.org/pdf/1509.06825.pdf Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours], Pinto and Gupta, ICRA 2016
−
* "Vision Based Victim Detection from Unmanned Aerial Vehicles", M. Andriluka ''et al.'', IROS 2010.  ''UAV, person detection''
+
* [http://www.cs.ucf.edu/~rrahmati/manipulationLearning.html Learning real manipulation tasks from virtual demonstrations using LSTM], Rahmatizadeh ''et al.'', AAAI 2018
−
* "Biped Navigation in Rough Environments using On-board Sensing", J. Chestnutt, Y. Takaoka, K. Suga, K. Nishiwaki, J. Kuffner, and S. Kagami, IROS 2009. ''Footstep planning, ladar, plane fitting''
+
* [https://arxiv.org/abs/1704.05588 Learning to Fly by Crashing], Gandhi, Pinto, and Gupta, IROS 2017
−
* "Real-Time Navigation in 3D Environments Based on Depth Camera Data", D. Maier, A. Hornung, and M. Bennewitz, Humanoids 2012. ''RGB-D, localization, mapping, planning''
+
<!--* "Collaborative mapping of an earthquake-damaged building via ground and aerial robots", N. Michael ''et al.'', JFR 2012. ''UAV, UGV, disaster, mapping''-->
−
* "Robotic Grasping of Novel Objects using Vision", A. Saxena, J. Driemeyer, A. Ng, IJRR 2008. ''Grasping, learning''
+
* [http://ieeexplore.ieee.org/document/7353481/ VoxNet: A 3D Convolutional Neural Network for real-time object recognition], Maturana and Scherer, IROS 2015
−
* "Self-supervised Monocular Road Detection in Desert Terrain", H. Dahlkamp, A. Kaehler, D. Stavens, S. Thrun, and G. Bradski, 2006. ''DARPA GC, color similarity, segmentation''
+
* [https://arxiv.org/abs/1612.00593 PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation], Qi, Su, Mo, and Guibas, CVPR 2017
−
* "Multi-Sensor Lane Finding in Urban Road Networks", A. Huang, D. Moore, M. Antone, E. Olson, S. Teller, RSS 2008.  ''DARPA UC, edge detection, robust curve fitting, tracking''
+
<!--* "Vision Based Victim Detection from Unmanned Aerial Vehicles", M. Andriluka ''et al.'', IROS 2010-->
 +
* [http://www2.informatik.uni-freiburg.de/~hornunga/pub/maier12humanoids.pdf Real-Time Navigation in 3D Environments Based on Depth Camera Data], D. Maier, A. Hornung, and M. Bennewitz, Humanoids 2012  
 +
* [https://www-cs.stanford.edu/~asaxena/learninggrasp/IJRR_saxena_etal_roboticgraspingofnovelobjects.pdf Robotic Grasping of Novel Objects using Vision], A. Saxena, J. Driemeyer, A. Ng, IJRR 2008  
 +
* [http://www.cs.ubc.ca/~van/papers/2017-TOG-deepLoco/ DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning], X. Peng, G. Berseth, K. Yin, and M. van de Panne, SIGGRAPH 2017
 +
<!--* "Self-supervised Monocular Road Detection in Desert Terrain", H. Dahlkamp, A. Kaehler, D. Stavens, S. Thrun, and G. Bradski, 2006. ''DARPA GC, color similarity, segmentation''-->
 +
<!--* "Multi-Sensor Lane Finding in Urban Road Networks", A. Huang, D. Moore, M. Antone, E. Olson, S. Teller, RSS 2008.  ''DARPA UC, edge detection, robust curve fitting, tracking''-->
 
<!--* "Visual odometry", D. Nister ''et al.'', CVPR 2004.  ''Feature detection, tracking, estimation, stereo''-->
 
<!--* "Visual odometry", D. Nister ''et al.'', CVPR 2004.  ''Feature detection, tracking, estimation, stereo''-->
−
* "High fidelity day/night stereo mapping with vegetation and negative obstacle detection for vision-in-the-loop walking", M. Bajracharya ''et al.'', IROS 2013. ''LS3, dense stereo depth, visual odometry''
+
* [http://ieeexplore.ieee.org/document/6696879/ High fidelity day/night stereo mapping with vegetation and negative obstacle detection for vision-in-the-loop walking], M. Bajracharya ''et al.'', IROS 2013   
−
* "Autonomous Door Opening and Plugging In with a Personal Robot", W. Meeussen ''et al.'', IROS 2010.  ''PR2, grasping
 
 
<!--* So many PR2 papers on grasping
 
<!--* So many PR2 papers on grasping
 
* Detection/recognition-->
 
* Detection/recognition-->
Line 66: Line 70:
 
!width="5%"|
 
!width="5%"|
 
!width="95%"|
 
!width="95%"|
−
|-
 
−
|valign="top"|'''Operating system'''
 
−
|
 
−
* If you don't have a Linux distribution running currently, it's not hard to add one to your machine.  The [http://www.ubuntu.com Ubuntu website] has comprehensive instructions on installing it from different sources (CD-ROM, USB stick, etc.) on a separate partition (aka "dual booting").  I recommend version [https://www.ubuntu.com/download/desktop 16.04.3 LTS]. 
 
−
* You will get an account on my lab workstation for the duration of the course.
 
 
|-
 
|-
 
|valign="top"|'''Software'''
 
|valign="top"|'''Software'''
 
|
 
|
−
* ROS
+
* [https://github.com/PointCloudLibrary/pcl Point Cloud Library (PCL)].  Latest version on GitHub seems to be 1.8.1.  Should be installable on Linux/Mac/Windows
−
**[http://www.ros.org/wiki/ROS/Installation Installation instructions] (Kinetic, Ubuntu 16.04, Desktop-Full Install).  This will take about 430 Mb of space, and it includes [http://pointclouds.org/ PCL] for 3-D point cloud processing and [http://opencv.org/ OpenCV] for computer vision/image processing.  We will mainly be using ROS for these included libraries and the visualization functionality of the rviz tool, so don't worry about "learning" ROS.  If you're curious, links to more information are below.   
+
** [http://www.pointclouds.org/documentation/tutorials/ Tutorials]
 +
** [http://www.pointclouds.org/documentation/ Documentation]
 +
** [https://larrylisky.com/2016/11/03/point-cloud-library-on-ubuntu-16-04-lts/ Installation on Ubuntu 16.04] (including prerequisite libraries)
 +
<!--
 +
* [http://www.ros.org/ ROS]
 +
**[http://www.ros.org/wiki/ROS/Installation Installation instructions] (Kinetic, Ubuntu 16.04, Desktop-Full Install).  It includes [http://pointclouds.org/ PCL] for 3-D point cloud processing and [http://opencv.org/ OpenCV] for computer vision/image processing.  We will mainly be using ROS for these included libraries and the visualization functionality of the rviz tool, so don't worry about "learning" ROS.  If you're curious, links to more information are below.   
 
** [http://clearpath.wpengine.netdna-cdn.com/wp-content/uploads/2014/01/ROS-Cheat-Sheet-Landscape-v2.pdf "Cheatsheet"]
 
** [http://clearpath.wpengine.netdna-cdn.com/wp-content/uploads/2014/01/ROS-Cheat-Sheet-Landscape-v2.pdf "Cheatsheet"]
 
** [http://www.ros.org/wiki/roblab-whge-ros-pkg Tutorial videos]
 
** [http://www.ros.org/wiki/roblab-whge-ros-pkg Tutorial videos]
 
** [http://www.ros.org/wiki/rviz/UserGuide Rviz user guide]
 
** [http://www.ros.org/wiki/rviz/UserGuide Rviz user guide]
 +
* [https://www.tensorflow.org/ TensorFlow]
 +
** [https://www.tensorflow.org/install/ Installation instructions]
 +
-->
 +
|-
 +
|valign="top"|'''Note'''
 +
|
 +
<!--
 +
* ROS works best with Linux.  If you don't have a Linux distribution running currently, it's not hard to add one to your machine.  The [http://www.ubuntu.com Ubuntu website] has comprehensive instructions on installing Ubuntu Linux from different sources (CD-ROM, USB stick, etc.) on a separate partition (aka "dual booting"), or you can use a virtual machine if you are comfortable with that.  I recommend version [https://www.ubuntu.com/download/desktop 16.04.3 LTS]. 
 +
-->
 +
* TensorFlow is available for multiple operating systems, but the processing demands can be high, especially if your computer does not have a new-ish Nvidia GPU
 +
* If you have significant problems installing/using PCL or TensorFlow on your own computer, I have a GPU workstation in my lab that can be used for selected training jobs.  Contact me *after* you have determined that your own machine is insufficient
 +
<!--
 
* PCL
 
* PCL
 
** Do not install separately--we will just use version (1.7) included in ROS Kinetic  
 
** Do not install separately--we will just use version (1.7) included in ROS Kinetic  
 
* OpenCV
 
* OpenCV
 
** Do not install separately--we will just use version (3) included in ROS Kinetic  
 
** Do not install separately--we will just use version (3) included in ROS Kinetic  
 +
-->
 
|}
 
|}
  
Line 99: Line 116:
 
|style="background:rgb(102, 204, 255)"|1
 
|style="background:rgb(102, 204, 255)"|1
 
|Feb. 6
 
|Feb. 6
−
|Background, introduction to DARPA Robotics Challenge
+
|Background  
 
|
 
|
−
|<!--[https://docs.google.com/presentation/d/1hXPhRiDdg8e-ZPG_zMT_X4-CQ7wYcJEzMeK3eOaBEQw/edit?usp=sharing slides]-->
+
|[https://docs.google.com/presentation/d/13tXZozuTBfFBte2vjSRkWXHlVJv0yBUDonySzBULUDI/edit?usp=sharing slides]
 
|-
 
|-
 
|2
 
|2
 
|Feb. 8
 
|Feb. 8
−
|DRC algorithm components
+
|Finish background
−
|[https://www.youtube.com/watch?v=kdx-DFI1VuA ARGOS challenge overview] (8:00)
 
 
|
 
|
 +
|[https://docs.google.com/presentation/d/1aEWKMVET3Hnn5eCg-vX9Li2hSMPD7kpgGZSAMNUh-cM/edit?usp=sharing slides]
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|3
 
|style="background:rgb(102, 204, 255)"|3
 
|Feb. 13
 
|Feb. 13
−
|Introduction to ROS
+
|Introduction to DARPA Urban Challenge (UC), Robotics Challenge (DRC) and algorithm components
 
|
 
|
−
|[http://www.rsl.ethz.ch/education-students/lectures/ros.html ETH ROS mini course] (in particular: overview, RViz, TF), [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_ANYmal_2016.pdf ETH case study "ANYmal at the ARGOS Challenge"]<!--[https://docs.google.com/presentation/d/1G_9xd6wtAH0tVbPceUwQGrlpDaPerPrhedRMde-yY78/edit?usp=sharing slides]-->
+
* [http://www.nytimes.com/2007/11/05/technology/05robot.html "Crashes and Traffic Jams in Military Test of Robotic Vehicles"] (NYTimes, Nov. 5, 2007)
 +
* [http://www.youtube.com/watch?v=P0NTV2mbJhA UC highlights] (6:06), [http://cs.stanford.edu/group/roadrunner/video.html Stanford UC clips]
 +
* [https://www.youtube.com/watch?v=8P9geWwi9e0 DRC highlights] (5:47)
 +
* [http://www.pbs.org/wgbh/nova/tech/rise-of-the-robots.html NOVA "Rise of the Robots"]
 +
|[https://docs.google.com/presentation/d/1gziYFt4rrb6tZ9fKm_oJ0tTHPHPX5LvKWcUo0k1mNBM/edit?usp=sharing slides]
 
|-
 
|-
 
|4
 
|4
 
|Feb. 15
 
|Feb. 15
−
|PCL tutorial
+
|Introduction to ROS; PCL tutorial
 +
|
 +
[https://www.youtube.com/watch?v=kdx-DFI1VuA ARGOS challenge overview] (8:00)
 
|
 
|
−
|<!--[https://docs.google.com/presentation/d/1iySLQMbwAve3ZgCJMzDP5nCcaJqyWAe4erj_fCPND6U/edit?usp=sharing slides]
+
[https://docs.google.com/presentation/d/1fTDJkK5XwvzuF3pPr3I7CHjG4lhqpDCOulPDamjcekM/edit?usp=sharing slides]<br>
−
[https://bitbucket.org/crasmuss/my_pcl_tutorial.git Sample PCL programs]-->
+
[https://bitbucket.org/crasmuss/my_pcl_tutorial.git Sample PCL programs] (these require ROS)<br>[http://www.rsl.ethz.ch/education-students/lectures/ros.html ETH ROS mini course] (in particular: overview, RViz, TF), [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_ANYmal_2016.pdf ETH case study "ANYmal at the ARGOS Challenge"]
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|5
 
|style="background:rgb(102, 204, 255)"|5
 
|Feb. 20<br>''Register/add deadline Feb. 19''
 
|Feb. 20<br>''Register/add deadline Feb. 19''
 
|Plane/obstacle/object segmentation (3-D)
 
|Plane/obstacle/object segmentation (3-D)
−
|
+
|[https://en.wikipedia.org/wiki/Random_sample_consensus RANSAC background]
−
|HW #1<!--[[CISC849_S2014_HW1|HW #1]]-->
+
|[[CISC849_S2018_HW1|HW #1]]<br>[https://drive.google.com/open?id=1NcSmS-plATqoWhuv4-BSjQ5xept3ajOR plane_fit.cpp] (No ROS required)
 
|-
 
|-
 
|6
 
|6
 
|Feb. 22
 
|Feb. 22
−
|HW #1 strategies
+
|Finish plane segmentation, clustering, ICP
−
|
+
|<!--"Autonomous Door Opening and Plugging In with a Personal Robot", W. Meeussen ''et al.'', IROS 2010<br>"Biped Navigation in Rough Environments using On-board Sensing", J. Chestnutt, Y. Takaoka, K. Suga, K. Nishiwaki, J. Kuffner, and S. Kagami, IROS 2009-->
−
|<!--
+
|[https://docs.google.com/presentation/d/17D8zMyRES-0sN7OeW28V0xFLqVOE4pFvjscs6eVEzL0/edit?usp=sharing slides]<!--
 
(Clustering example added [https://bitbucket.org/crasmuss/my_pcl_tutorial.git here])-->
 
(Clustering example added [https://bitbucket.org/crasmuss/my_pcl_tutorial.git here])-->
 
|-
 
|-
Line 139: Line 162:
 
|Image classification background<!--Clustering, normal estimation-->
 
|Image classification background<!--Clustering, normal estimation-->
 
|
 
|
−
|[http://cs231n.stanford.edu/syllabus.html Stanford CS231n] [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture2.pdf Image classification slides]<!--[https://docs.google.com/presentation/d/1r8SfI3s0FVIrDBYi3ab_h-kwCC7fIpFsMgsIXBSxBgc/edit?usp=sharing slides]
+
|[http://cs231n.stanford.edu/syllabus.html Stanford CS231n] [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture2.pdf Image classification slides] (6-61), <!--[https://docs.google.com/presentation/d/1r8SfI3s0FVIrDBYi3ab_h-kwCC7fIpFsMgsIXBSxBgc/edit?usp=sharing slides]
 
(Normals exampled added [https://bitbucket.org/crasmuss/my_pcl_tutorial.git here])-->
 
(Normals exampled added [https://bitbucket.org/crasmuss/my_pcl_tutorial.git here])-->
 
|-
 
|-
 
|8
 
|8
 
|Mar. 1
 
|Mar. 1
−
|Introduction to convolutional neural networks<!--, Registration, features  -->
+
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Instructor away''<!--, Registration, features  -->
 
|
 
|
−
|Stanford CS231n [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf Convolutional Neural Networks slides]<!--[http://www.pointclouds.org/assets/icra2013/registration.pdf PCL registration], [http://www.pointclouds.org/assets/uploads/cglibs13_features.pdf features]--><br>''HW #1 due''
+
|<!--[http://www.pointclouds.org/assets/icra2013/registration.pdf PCL registration], [http://www.pointclouds.org/assets/uploads/cglibs13_features.pdf features]--><br>''HW #1 due''
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|9
 
|style="background:rgb(102, 204, 255)"|9
 
|Mar. 6
 
|Mar. 6
−
|Introduction to OpenCV (in ROS), deep learning libraries, <br>detection & segmentation background
+
|More classification background
 
|
 
|
−
|Stanford CS231n [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture8.pdf Deep Learning software slides], [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture11.pdf Detection and Segmentation slides] <!--[https://docs.google.com/presentation/d/1MrC0XyhFf-Q0kHgy46hcWyMXI7mpEiJxqXMVlfiX2CU/edit?usp=sharing&authkey=COWqkvYK slides]-->
+
|[http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture3.pdf Loss/optimization slides] (8-10, 37-77, finish with web demo), [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture4.pdf Intro to NN] (83-88)<!--[https://docs.google.com/presentation/d/1MrC0XyhFf-Q0kHgy46hcWyMXI7mpEiJxqXMVlfiX2CU/edit?usp=sharing&authkey=COWqkvYK slides]-->
 
|-
 
|-
 
|10
 
|10
 
|Mar. 8
 
|Mar. 8
−
|More on detection, tracking<!--Object recognition (sample paper)-->
+
|<!--Object recognition (sample paper)-->Introduction to convolutional neural networks
−
|[https://arxiv.org/abs/1612.08242 YOLOv2, Redmon and Farhadi] (CVPR 2017)<!--[http://research.microsoft.com/pubs/145347/bodypartrecognition.pdf "Real-Time Human Pose Recognition in Parts from Single Depth Images"], J. Shotton ''et al.'', CVPR 2011.  ''Recognition, classification, RGB-D''-->
+
|<!--[http://research.microsoft.com/pubs/145347/bodypartrecognition.pdf "Real-Time Human Pose Recognition in Parts from Single Depth Images"], J. Shotton ''et al.'', CVPR 2011.  ''Recognition, classification, RGB-D''-->
−
|HW #2<!--[[CISC849_S2014_HW2|HW #2]]-->
+
|Stanford CS231n [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf Convolutional Neural Networks slides] (15-63, 71-78), [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture9.pdf CNN architectures] (8-36), [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture11.pdf Detection and Segmentation slides] (15-27)
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|11
 
|style="background:rgb(102, 204, 255)"|11
 
|Mar. 13
 
|Mar. 13
−
|Localization<!--Mapping (sample paper)-->
+
|<!--Localization-->Finish detection & segmentation background, introduction to TensorFlow library
−
|<!--[http://www.ri.cmu.edu/publication_view.html?pub_id=5590 "Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels"], N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007. ''Occupancy grids, SLAM''-->
+
|[https://arxiv.org/abs/1612.08242 YOLOv2, Redmon and Farhadi] (CVPR 2017)
−
|ETH localization lectures: [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_Localization_I_2017.pdf 1] [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_Localization_II_2017.pdf 2]<!--[https://docs.google.com/presentation/d/1CTGsRG7yvJVfZyBfaC1ZpBoUJ3ALbLaYnMeYZDM1wRA/edit?usp=sharing slides]-->
+
|<!--ETH localization lectures: [http://nameless.cis.udel.edAishwarya Dangu/class_data/849_s2018/ETH_Localization_I_2017.pdf 1] [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_Localization_II_2017.pdf 2]--><!--[https://docs.google.com/presentation/d/1CTGsRG7yvJVfZyBfaC1ZpBoUJ3ALbLaYnMeYZDM1wRA/edit?usp=sharing slides]-->Stanford CS231n [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture11.pdf Detection and Segmentation slides] (44-49, 53-85), [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture8.pdf deep learning software slides] (21-33, 40-59)<br>[https://drive.google.com/file/d/104GFBl4J6bseaTNBcqW2ub8hyXtL99d1/view?usp=sharing TensorFlow code sample from Stanford lecture]<br>[[CISC849_S2018_HW2|HW #2]]
 
|-
 
|-
 
|12
 
|12
 
|Mar. 15
 
|Mar. 15
−
|Particle filters, localization
+
|<!--Particle filters and localization-->More on TensorFlow programming
−
|[http://www2.informatik.uni-freiburg.de/~wurm/papers/hornung10iros.pdf Humanoid Robot Localization in Complex Indoor Environments], A. Hornung, K. Wurm, M. Bennewitz, IROS 2010.  ''Monte Carlo localization''
+
|<!--[http://www2.informatik.uni-freiburg.de/~wurm/papers/hornung10iros.pdf Humanoid Robot Localization in Complex Indoor Environments], A. Hornung, K. Wurm, M. Bennewitz, IROS 2010-->
−
|[http://robots.stanford.edu/probabilistic-robotics/ppt/particle-filters.ppt Thrun particle filtering slides]
+
|<!--[http://robots.stanford.edu/probabilistic-robotics/ppt/particle-filters.ppt Thrun particle filtering slides]-->
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|13
 
|style="background:rgb(102, 204, 255)"|13
 
|Mar. 20
 
|Mar. 20
−
|Motion planning background
+
|<!--Motion planning background-->Perception for stepping
−
|
+
|"Learning Locomotion over Rough Terrain using Terrain Templates", M. Kalakrishnan, J. Buchli, P. Pastor, and S. Schaal, IROS 2009
−
|''HW #2 due''
+
|[https://docs.google.com/presentation/d/1HQdeZ2JjATrWN7Y7BIgpeiFoIJa0J44P2d0bN_YSwRY/edit?usp=sharing slides]<br>[https://www.youtube.com/watch?v=G4lT9CLyCNw DeepLoco highlights video] (6:23)
 
|-
 
|-
 
|14
 
|14
 
|Mar. 22
 
|Mar. 22
−
|Perception for stepping<!--Mapping-->
+
|style="background:rgb(255, 102, 0)"|NO CLASS<!--Mapping-->
−
|"Learning Locomotion over Rough Terrain using Terrain Templates", M. Kalakrishnan, J. Buchli, P. Pastor, and S. Schaal, IROS 2009<!--[http://wiki.ros.org/gmapping gmapping] demos: [http://www.youtube.com/watch?v=7iIDdvCXIFM Pirobot], [http://www.youtube.com/watch?v=_jwBKo0SXng MIT]<br>[http://www.youtube.com/watch?v=F8pdObV_df4 Darmstadt "Hector" mapping]<br>
+
|<!--[http://wiki.ros.org/gmapping gmapping] demos: [http://www.youtube.com/watch?v=7iIDdvCXIFM Pirobot], [http://www.youtube.com/watch?v=_jwBKo0SXng MIT]<br>[http://www.youtube.com/watch?v=F8pdObV_df4 Darmstadt "Hector" mapping]<br>
 
Nao 3-D mapping and planning: [http://www.youtube.com/watch?v=srcx7lPoIfw 1], [http://www.youtube.com/watch?v=g2NZ_EasJv0 2]-->
 
Nao 3-D mapping and planning: [http://www.youtube.com/watch?v=srcx7lPoIfw 1], [http://www.youtube.com/watch?v=g2NZ_EasJv0 2]-->
−
|<!--ETH SLAM lectures: [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_SLAM_I_2017.pdf 1] [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_SLAM_II_2017.pdf 2]<!--[http://nameless.cis.udel.edu/class_data/cisc829/oct18/thrun_fastslam.pdf Thrun FastSLAM slides] (grids from slide 29)<br>[http://www.youtube.com/watch?v=3Yl2aq28LFQ Accompanying Stachniss lecture]--><br>-->''Paper presentation choice due Friday, March 23''
+
|<!--ETH SLAM lectures: [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_SLAM_I_2017.pdf 1] [http://nameless.cis.udel.edu/class_data/849_s2018/ETH_SLAM_II_2017.pdf 2]<!--[http://nameless.cis.udel.edu/class_data/cisc829/oct18/thrun_fastslam.pdf Thrun FastSLAM slides] (grids from slide 29)<br>[http://www.youtube.com/watch?v=3Yl2aq28LFQ Accompanying Stachniss lecture]-->''HW #2 due Friday, March 23''
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|
 
|style="background:rgb(102, 204, 255)"|
Line 199: Line 222:
 
|style="background:rgb(102, 204, 255)"|15
 
|style="background:rgb(102, 204, 255)"|15
 
|Apr. 3
 
|Apr. 3
−
|DARPA Urban Challenge; project kick-off
+
|Imitation learning
−
|[http://www.youtube.com/watch?v=P0NTV2mbJhA Urban Challenge highlights], [http://cs.stanford.edu/group/roadrunner/video.html Stanford clips]
 
 
|
 
|
 +
* [http://rpg.ifi.uzh.ch/docs/RAL16_Giusti.pdf A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots], Giusti ''et al.'', ''IEEE Robotics and Automation Letters'', 2016
 +
* [https://arxiv.org/pdf/1604.07316.pdf End to End Learning for Self-Driving cars], Bojarski ''et al.'', 2016
 +
* [https://www.youtube.com/watch?v=NJU9ULQUwng Dave-2 driving a Lincoln], [https://www.youtube.com/watch?v=umRdt3zGgpU Quadcopter Navigation in the Forest...], [https://www.youtube.com/watch?v=hNsP6-K3Hn4 DAgger for drones]
 +
* [https://www.youtube.com/watch?v=ZMhO1FO_j0o&t=566s Hal Daume DAgger explanation] (9:26-12:43)
 +
|[http://rll.berkeley.edu/deeprlcourse/ Berkeley Levine course] [http://rll.berkeley.edu/deeprlcourse/f17docs/lecture_2_behavior_cloning.pdf IL slides] (DAgger defined on slide 14); [https://katefvision.github.io/katefSlides/immitation_learning_I_katef.pdf CMU Fragkiadaki IL slides] (1-29)<br>''Paper presentation choice due''
 
|-
 
|-
 
|16
 
|16
 
|Apr. 5
 
|Apr. 5
−
|Miscellaneous
+
|Reinforcement learning
−
|
 
 
|
 
|
 +
|[http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html UCL Silver course]: [http://nameless.cis.udel.edu/class_data/849_s2018/intro_RL.pdf RL slides] (7-42), [https://www.youtube.com/watch?v=2pWv7GOvuf0 RL video] (15:40-21:20, 1:02:13-1:04:40 contains movies missing from PDF); [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture14.pdf Stanford deep-Q lecture slides] (23-62); [https://www.nature.com/articles/nature14236.pdf original deep-Q paper]
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|17
 
|style="background:rgb(102, 204, 255)"|17
 
|Apr. 10<br>
 
|Apr. 10<br>
 
''Withdraw deadline Apr. 9''
 
''Withdraw deadline Apr. 9''
−
|Student paper presentations
+
|Student paper presentation
 +
|
 +
* Peng Su, [https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Yeung_End-To-End_Learning_of_CVPR_2016_paper.pdf "End-to-end Learning of Action Detection from Frame Glimpses in Videos"]
 
|
 
|
−
|''Project proposal due Monday, April 9''
 
 
|-
 
|-
 
|18
 
|18
 
|Apr. 12
 
|Apr. 12
−
|Student paper presentations
+
|Student paper presentation
−
|
 
 
|
 
|
 +
* Patrick Geneva, [http://openaccess.thecvf.com/content_cvpr_2017/papers/Dong_Visual-Inertial-Semantic_Scene_Representation_CVPR_2017_paper.pdf "Visual-Inertial-Semantic Scene Representation for 3D Object Detection"]
 +
|''Project proposal due Sunday, April 15''
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|19
 
|style="background:rgb(102, 204, 255)"|19
Line 226: Line 255:
 
|Student paper presentations
 
|Student paper presentations
 
|
 
|
 +
* Zengxiang Lu, "Image Generation from Scene Graphs"
 +
* Aishwarya Dang, [http://ieeexplore.ieee.org/document/7353481/ "VoxNet: A 3D Convolutional Neural Network for real-time object recognition"]
 
|
 
|
 
|-
 
|-
 
|20
 
|20
 
|Apr. 19
 
|Apr. 19
−
|Mid-project review; student paper presentation
+
|Student paper presentations
 
|
 
|
 +
* Sumeet Gupta, [https://arxiv.org/abs/1704.05588 "Learning to Fly by Crashing"]
 +
* Yulin Yang, [http://rpg.ifi.uzh.ch/docs/RAL18_Loquercio.pdf "DroNet: Learning to Fly by Driving"]
 
|
 
|
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|21
 
|style="background:rgb(102, 204, 255)"|21
 
|Apr. 24
 
|Apr. 24
−
|Student paper presentations
+
|Student paper presentation
 
|
 
|
 +
Matt Schmittle, [https://arxiv.org/pdf/1509.06825.pdf Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours]
 
|
 
|
 
|-
 
|-
 
|22
 
|22
 
|Apr. 26
 
|Apr. 26
−
|Motion planning background
+
|Student paper presentations
 
|
 
|
 +
* Apoorva Patil, [http://www.cs.ubc.ca/~van/papers/2017-TOG-deepLoco/ DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning]
 +
* Leighanne Hsu, [http://www2.informatik.uni-freiburg.de/~hornunga/pub/maier12humanoids.pdf Real-Time Navigation in 3D Environments Based on Depth Camera Data]
 
|<!--[https://docs.google.com/presentation/d/19Wm-H2ttW_m5pan_lCXDzmk468gLoud6Xug_2_i0mWI/edit?usp=sharing&authkey=CKmYmJYI slides]-->
 
|<!--[https://docs.google.com/presentation/d/19Wm-H2ttW_m5pan_lCXDzmk468gLoud6Xug_2_i0mWI/edit?usp=sharing&authkey=CKmYmJYI slides]-->
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|23
 
|style="background:rgb(102, 204, 255)"|23
 
|May 1
 
|May 1
−
|Miscellaneous
+
|Student paper presentation
−
|
+
|Yiqun Jia, [https://arxiv.org/pdf/1603.03833.pdf From Virtual Demonstration to Real-World Manipulation Using LSTM and MDN]
 
|
 
|
 
|-
 
|-
 
|24
 
|24
 
|May 3
 
|May 3
−
|Final project review
+
|Instructor paper presentation
−
|
+
|Rasmussen, [http://www.ri.cmu.edu/publication_view.html?pub_id=5590 "Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels"], N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007
−
|
+
|[https://docs.google.com/presentation/d/1CTGsRG7yvJVfZyBfaC1ZpBoUJ3ALbLaYnMeYZDM1wRA/edit#slide=id.i31 DepthX slides]
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|25
 
|style="background:rgb(102, 204, 255)"|25
 
|May 8
 
|May 8
−
|"Bonus" material
+
|Final project review
 
|
 
|
 
|
 
|
 
|-
 
|-
−
|26
+
|
 
|May 10
 
|May 10
−
|Miscellaneous
+
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Work on projects...''
 
|
 
|
 
|
 
|
 
|-
 
|-
−
|style="background:rgb(102, 204, 255)"|27
+
|style="background:rgb(102, 204, 255)"|26
 
|May 15
 
|May 15
−
|Final project presentations
+
|Final project presentations part I
 +
|
 +
|
 +
|-
 +
|27
 +
|May 16
 +
|Final project presentations part II
 +
|
 +
|
 +
|-
 +
|28
 +
|May 21
 +
|Final project presentations part III
 +
|
 
|
 
|
−
|''Final project due''
 
 
|}
 
|}

Latest revision as of 10:53, 10 May 2018

Course information

Title CISC849 Robot Vision and Learning
Shortened URL https://goo.gl/ektJij
Description Survey of image-based 2-D and 3-D sensing algorithms for mobile robot navigation and interaction, including motion estimation, obstacle segmentation, terrain modeling, and object recognition, with a particular focus on deep learning techniques to dramatically improve performance.
When Tuesdays and Thursdays, 11 am-12:15 pm
Where Smith 102A
Instructor Christopher Rasmussen, 446 Smith Hall, cer@cis.udel.edu
Office hours Wednesdays, 10 am -- 12 pm
Grading
  • 20% Oral paper presentation (individual or pairs, 30 minutes)
  • 30% Two programming assignments (individual)
  • 50% Final project (teams of 1-3)
    • 10% = 2 page proposal, including planned methods, citations of relevant papers, data sources, and division of labor
    • 10% = Joint 15-minute presentation on final results, with accompanying slides
    • 30% = Actual results and estimated effort, factoring in difficulty of problem tackled
Academic policies Programming projects are due by midnight of the deadline day (with a grace period of a few hours afterward...after sunrise is definitely late). A late homework is a 0 without a valid prior excuse. To give you a little flexibility, you have 6 "late days" to use on homeworks to extend the deadline by one day each without penalty. No more than three late days may be used per assignment. Late days will automatically be subtracted, but as a courtesy please notify the instructor in an e-mail of your intention to use late days before the deadline. See submission instructions below.

Students can discuss problems with one another in general terms, but must work independently on programming assignments. This also applies to online and printed resources: you may consult them as references (as long as you cite them), but the words and source code you turn in must be yours alone. The University's policies on academic dishonesty are set forth in the student code of conduct here.

Homeworks Assignment submissions should consist of a directory containing all code (your .cpp files, makefile, etc.), any output data generated (e.g., images, movies, etc.), and an explanation of your approach, what worked and didn't work, etc. contained in a separate text or HTML file. Do not submit executables or .o files, please! The directory you submit for each assignment should be packaged by tar'ing and gzip'ing it or just zip'ing it. The resulting file should be submitted through Canvas.

You may develop your C/C++ code in any fashion that is convenient--that is, with any compiler and operating system that you want. However, we will be grading your homework on a Linux system with a makefile, and so you must avoid OS- and hardware-specific functions and provide a makefile for us that will work (like one of the templates above).

Possible Papers to Present (not a complete list)

Instructions for Homeworks

Software
Note
  • TensorFlow is available for multiple operating systems, but the processing demands can be high, especially if your computer does not have a new-ish Nvidia GPU
  • If you have significant problems installing/using PCL or TensorFlow on your own computer, I have a GPU workstation in my lab that can be used for selected training jobs. Contact me *after* you have determined that your own machine is insufficient

Schedule

Note: The blue squares in the "#" column below indicate Tuesdays.

# Date Topic Links/Readings/videos Assignments/slides
1 Feb. 6 Background slides
2 Feb. 8 Finish background slides
3 Feb. 13 Introduction to DARPA Urban Challenge (UC), Robotics Challenge (DRC) and algorithm components slides
4 Feb. 15 Introduction to ROS; PCL tutorial

ARGOS challenge overview (8:00)

slides
Sample PCL programs (these require ROS)
ETH ROS mini course (in particular: overview, RViz, TF), ETH case study "ANYmal at the ARGOS Challenge"

5 Feb. 20
Register/add deadline Feb. 19 		Plane/obstacle/object segmentation (3-D) RANSAC background HW #1
plane_fit.cpp (No ROS required)
6 Feb. 22 Finish plane segmentation, clustering, ICP slides
7 Feb. 27 Image classification background Stanford CS231n Image classification slides (6-61),
8 Mar. 1 NO CLASS
Instructor away
HW #1 due
9 Mar. 6 More classification background Loss/optimization slides (8-10, 37-77, finish with web demo), Intro to NN (83-88)
10 Mar. 8 Introduction to convolutional neural networks Stanford CS231n Convolutional Neural Networks slides (15-63, 71-78), CNN architectures (8-36), Detection and Segmentation slides (15-27)
11 Mar. 13 Finish detection & segmentation background, introduction to TensorFlow library YOLOv2, Redmon and Farhadi (CVPR 2017) Stanford CS231n Detection and Segmentation slides (44-49, 53-85), deep learning software slides (21-33, 40-59)
TensorFlow code sample from Stanford lecture
HW #2
12 Mar. 15 More on TensorFlow programming
13 Mar. 20 Perception for stepping "Learning Locomotion over Rough Terrain using Terrain Templates", M. Kalakrishnan, J. Buchli, P. Pastor, and S. Schaal, IROS 2009 slides
DeepLoco highlights video (6:23)
14 Mar. 22 NO CLASS HW #2 due Friday, March 23
Mar. 27 NO CLASS
Spring break
Mar. 29 NO CLASS
Spring break
15 Apr. 3 Imitation learning Berkeley Levine course IL slides (DAgger defined on slide 14); CMU Fragkiadaki IL slides (1-29)
Paper presentation choice due
16 Apr. 5 Reinforcement learning UCL Silver course: RL slides (7-42), RL video (15:40-21:20, 1:02:13-1:04:40 contains movies missing from PDF); Stanford deep-Q lecture slides (23-62); original deep-Q paper
17 Apr. 10

Withdraw deadline Apr. 9

Student paper presentation
18 Apr. 12 Student paper presentation Project proposal due Sunday, April 15
19 Apr. 17
 Student paper presentations
20 Apr. 19 Student paper presentations
21 Apr. 24 Student paper presentation

Matt Schmittle, Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours

22 Apr. 26 Student paper presentations
23 May 1 Student paper presentation Yiqun Jia, From Virtual Demonstration to Real-World Manipulation Using LSTM and MDN
24 May 3 Instructor paper presentation Rasmussen, "Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels", N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007 DepthX slides
25 May 8 Final project review
May 10 NO CLASS
Work on projects...
26 May 15 Final project presentations part I
27 May 16 Final project presentations part II
28 May 21 Final project presentations part III
Retrieved from "http://nameless.cis.udel.edu/class_wiki/index.php?title=CISC849_S2018&oldid=1159"