Difference between revisions of "CISC849 F2019"

From class_wiki
Jump to: navigation, search
(→‎Schedule)
(→‎Schedule)
 
(21 intermediate revisions by the same user not shown)
Line 26: Line 26:
 
|
 
|
 
* 25% Oral paper presentation (individual, 30 minutes)
 
* 25% Oral paper presentation (individual, 30 minutes)
−
* 25% Programming/learning assignment (individual)
+
* 25% Programming/learning assignment (1-3 students)
−
* 50% Final project (1 or 2 students)
+
* 50% Final project (1-3 students)
 
** 10% = 2 page proposal, including planned methods, citations of relevant papers, data sources, and division of labor
 
** 10% = 2 page proposal, including planned methods, citations of relevant papers, data sources, and division of labor
 
** 10% = Joint 15-minute presentation on final results, with accompanying slides
 
** 10% = Joint 15-minute presentation on final results, with accompanying slides
Line 193: Line 193:
 
|style="background:rgb(102, 204, 255)"|13
 
|style="background:rgb(102, 204, 255)"|13
 
|Oct. 15
 
|Oct. 15
 +
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Instructor away''
 +
|
 +
|
 +
|-
 +
|14
 +
|Oct. 17
 
|Imitation learning
 
|Imitation learning
−
|* [http://rpg.ifi.uzh.ch/docs/RAL16_Giusti.pdf A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots], Giusti ''et al.'', ''IEEE Robotics and Automation Letters'', 2016
+
|
 +
* [https://www.youtube.com/watch?v=hUoDNeZS4so Agile Autonomous Driving using End-to-End Deep Imitation Learning], Pan ''et al.'', RSS 2018
 +
* [http://rpg.ifi.uzh.ch/docs/RAL16_Giusti.pdf A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots], Giusti ''et al.'', ''IEEE Robotics and Automation Letters'', 2016
 
* [https://arxiv.org/pdf/1604.07316.pdf End to End Learning for Self-Driving cars], Bojarski ''et al.'', 2016
 
* [https://arxiv.org/pdf/1604.07316.pdf End to End Learning for Self-Driving cars], Bojarski ''et al.'', 2016
−
* [https://www.youtube.com/watch?v=NJU9ULQUwng Dave-2 driving a Lincoln], [https://www.youtube.com/watch?v=umRdt3zGgpU Quadcopter Navigation in the Forest...], [https://www.youtube.com/watch?v=hNsP6-K3Hn4 DAgger for drones]
+
* <!--[https://www.youtube.com/watch?v=NJU9ULQUwng Dave-2 driving a Lincoln], -->[https://www.youtube.com/watch?v=umRdt3zGgpU Quadcopter Navigation in the Forest...], [https://www.youtube.com/watch?v=hNsP6-K3Hn4 DAgger for drones]
 
* [https://www.youtube.com/watch?v=ZMhO1FO_j0o&t=566s Hal Daume DAgger explanation] (9:26-12:43)
 
* [https://www.youtube.com/watch?v=ZMhO1FO_j0o&t=566s Hal Daume DAgger explanation] (9:26-12:43)
 
|[http://rll.berkeley.edu/deeprlcourse/ Berkeley Levine course] [http://rll.berkeley.edu/deeprlcourse/f17docs/lecture_2_behavior_cloning.pdf IL slides] (DAgger defined on slide 14); [https://katefvision.github.io/katefSlides/immitation_learning_I_katef.pdf CMU Fragkiadaki IL slides] (1-29)
 
|[http://rll.berkeley.edu/deeprlcourse/ Berkeley Levine course] [http://rll.berkeley.edu/deeprlcourse/f17docs/lecture_2_behavior_cloning.pdf IL slides] (DAgger defined on slide 14); [https://katefvision.github.io/katefSlides/immitation_learning_I_katef.pdf CMU Fragkiadaki IL slides] (1-29)
−
|-
 
−
|14
 
−
|Oct. 17
 
−
|Reinforcement learning
 
−
|
 
−
|[http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html UCL Silver course]: [http://nameless.cis.udel.edu/class_data/849_s2018/intro_RL.pdf RL slides] (7-42), [https://www.youtube.com/watch?v=2pWv7GOvuf0 RL video] (15:40-21:20, 1:02:13-1:04:40 contains movies missing from PDF); [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture14.pdf Stanford deep-Q lecture slides] (23-62); [https://www.nature.com/articles/nature14236.pdf original deep-Q paper]
 
 
<br>''Homework due''
 
<br>''Homework due''
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|15
 
|style="background:rgb(102, 204, 255)"|15
 
|Oct. 22<br>''Withdraw deadline''
 
|Oct. 22<br>''Withdraw deadline''
−
|RL for locomotion
+
|Reinforcement learning
 +
|[http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html UCL Silver course]: [http://nameless.cis.udel.edu/class_data/849_s2018/intro_RL.pdf RL slides] (7-42), [https://www.youtube.com/watch?v=2pWv7GOvuf0 RL video] (15:40-21:20, 1:02:13-1:04:40 contains movies missing from PDF); [http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture14.pdf Stanford deep-Q lecture slides] (23-62); [https://www.nature.com/articles/nature14236.pdf original deep-Q paper]
 
|
 
|
−
|[https://www.youtube.com/watch?v=G4lT9CLyCNw DeepLoco highlights video] (6:23)
 
 
<!--<br>''Paper presentation choice due''-->
 
<!--<br>''Paper presentation choice due''-->
 
|-
 
|-
 
|16
 
|16
 
|Oct. 24
 
|Oct. 24
−
|Instructor paper presentation
+
|style="background:rgb(255, 102, 0)"|NO CLASS<br>''Break before presentations start''
 
|
 
|
 
|
 
|
Line 223: Line 225:
 
|Oct. 29
 
|Oct. 29
 
|Student paper presentation
 
|Student paper presentation
−
|
+
|Maxim Bazik, "PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud"
 
|
 
|
 
|-
 
|-
Line 229: Line 231:
 
|Oct. 31
 
|Oct. 31
 
|Student paper presentation
 
|Student paper presentation
−
|
+
|Woosik Lee, "Learning Wheel Odometry and IMU Errors for Localization"
 
|<!--''Project proposal due Sunday, April 15''-->
 
|<!--''Project proposal due Sunday, April 15''-->
 
|-
 
|-
Line 241: Line 243:
 
|Nov. 7
 
|Nov. 7
 
|Student paper presentations
 
|Student paper presentations
−
|
+
|Chuchu Chen, "Deep Learning for 2D Scan Matching and Loop Closure"<br>
 +
Chirantan Ghosh, "Video Action Transformer Network"
 
|
 
|
 
|-
 
|-
Line 247: Line 250:
 
|Nov. 12
 
|Nov. 12
 
|Student paper presentation
 
|Student paper presentation
−
|
+
|Sha Liu, "Robots that can adapt like animals"<br>
 +
Seyedalireza Khoshsirat, "PCN: Point Completion Network"
 
|
 
|
 
|-
 
|-
Line 253: Line 257:
 
|Nov. 14
 
|Nov. 14
 
|Student paper presentations
 
|Student paper presentations
−
|
+
|Shengye Li, "Learning Common and Specific Features for RGB-D Semantic Segmentaion with Deconvolutional Networks"<br>
 +
Eric Wright, "Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight"
 
|<!--[https://docs.google.com/presentation/d/19Wm-H2ttW_m5pan_lCXDzmk468gLoud6Xug_2_i0mWI/edit?usp=sharing&authkey=CKmYmJYI slides]-->
 
|<!--[https://docs.google.com/presentation/d/19Wm-H2ttW_m5pan_lCXDzmk468gLoud6Xug_2_i0mWI/edit?usp=sharing&authkey=CKmYmJYI slides]-->
 
|-
 
|-
Line 259: Line 264:
 
|Nov. 19
 
|Nov. 19
 
|Student paper presentation
 
|Student paper presentation
−
|Shivanand, "PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows"  
+
|Shivanand Sheshappanavar, "PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows"<br>Zhang Guo, "Believe It or Not, We Know What You Are Looking at!"
 
|
 
|
 
|-
 
|-
 
|24
 
|24
 
|Nov. 21
 
|Nov. 21
−
|Instructor paper presentation
+
|Student paper presentations<!--Instructor paper presentation: RL for locomotion-->
−
|<!--Rasmussen, [http://www.ri.cmu.edu/publication_view.html?pub_id=5590 "Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels"], N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007-->
+
|Nikhil Gothankar, "Fast Online Object Tracking and Segmentation: A Unifying Approach"<br>
−
|<!--[https://docs.google.com/presentation/d/1CTGsRG7yvJVfZyBfaC1ZpBoUJ3ALbLaYnMeYZDM1wRA/edit#slide=id.i31 DepthX slides]-->
+
Vineet Singh, "Spatial Transformer Networks"<!--Rasmussen, [http://www.ri.cmu.edu/publication_view.html?pub_id=5590 "Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels"], N. Fairfield, G. Kantor, and D. Wettergreen, JFR 2007-->
 +
|<!--[https://www.youtube.com/watch?v=G4lT9CLyCNw DeepLoco highlights video] (6:23)--><!--[https://docs.google.com/presentation/d/1CTGsRG7yvJVfZyBfaC1ZpBoUJ3ALbLaYnMeYZDM1wRA/edit#slide=id.i31 DepthX slides]-->
 
|-
 
|-
 
|style="background:rgb(102, 204, 255)"|
 
|style="background:rgb(102, 204, 255)"|
Line 282: Line 288:
 
|style="background:rgb(102, 204, 255)"|25
 
|style="background:rgb(102, 204, 255)"|25
 
|Dec. 3
 
|Dec. 3
−
|Final project presentations part I
+
|Student paper presentation
−
|
+
|Md Mottalib, "An Empirical Evaluation of Deep Learning on Highway Driving"
 
|
 
|
 
|-
 
|-
 
|26
 
|26
−
|Dec. 5
+
|Dec. 5, 6
−
|Final project presentations part II
+
|Final project presentations
 
|
 
|
 
|
 
|
 
|}
 
|}

Latest revision as of 20:59, 2 December 2019

Course information

Title CISC849 Robot Vision and Learning
Description Survey of image-based 2-D and 3-D sensing algorithms for mobile robot navigation and interaction, including motion estimation, obstacle segmentation, terrain modeling, and object recognition, with a particular focus on deep learning techniques to dramatically improve performance.
When Tuesdays and Thursdays, 2-3:15 pm
Where Gore 317
Instructor Christopher Rasmussen, 446 Smith Hall, cer@cis.udel.edu
Office hours Mondays and Wednesdays, 2-3 pm
Grading
  • 25% Oral paper presentation (individual, 30 minutes)
  • 25% Programming/learning assignment (1-3 students)
  • 50% Final project (1-3 students)
    • 10% = 2 page proposal, including planned methods, citations of relevant papers, data sources, and division of labor
    • 10% = Joint 15-minute presentation on final results, with accompanying slides
    • 30% = Actual results and estimated effort, factoring in difficulty of problem tackled
Academic policies Programming projects are due by midnight of the deadline day (with a grace period of a few hours afterward...after sunrise is definitely late). A late homework is a 0 without a valid prior excuse. To give you a little flexibility, you have 6 "late days" to use on homeworks to extend the deadline by one day each without penalty. No more than three late days may be used per assignment. Late days will automatically be subtracted, but as a courtesy please notify the instructor in an e-mail of your intention to use late days before the deadline. See submission instructions below.

Students can discuss problems with one another in general terms, but must work independently on programming assignments. This also applies to online and printed resources: you may consult them as references (as long as you cite them), but the words and source code you turn in must be yours alone. The University's policies on academic dishonesty are set forth in the student code of conduct here.

Homeworks Assignment submissions should consist of a directory containing all code (your .cpp files, makefile, etc.), any output data generated (e.g., images, movies, etc.), and an explanation of your approach, what worked and didn't work, etc. contained in a separate text or HTML file. Do not submit executables or .o files, please! The directory you submit for each assignment should be packaged by tar'ing and gzip'ing it or just zip'ing it. The resulting file should be submitted through Canvas.

You may develop your C/C++ code in any fashion that is convenient--that is, with any compiler and operating system that you want. However, we will be grading your homework on a Linux system with a makefile, and so you must avoid OS- and hardware-specific functions and provide a makefile for us that will work (like one of the templates above).

Possible Papers to Present (not a complete list)


Schedule

Note: The blue squares in the "#" column below indicate Tuesdays.

# Date Topic Links/Readings/videos Assignments/slides
1 Aug. 27 Background slides
2 Aug. 29 Introduction to DARPA Urban Challenge (UC), Robotics Challenge (DRC) and algorithm components slides
3 Sep. 3 More about Grand Challenges, trail-following slides
4 Sep. 5 Traditional 2-D/3-D shape/object segmentation, part I

slides

5 Sep. 10
Register/add deadline 		NO CLASS
6 Sep. 12 Traditional 2-D/3-D shape/object segmentation, part II slides
7 Sep. 17 Image classification background Stanford CS231n Image classification slides (6-61),
8 Sep. 19 NO CLASS
9 Sep. 24 More classification background; Introduction to convolutional neural networks Stanford CS231n: Loss/optimization slides (8-10, 37-77, finish with web demo), Intro to NN (83-88), Convolutional Neural Networks slides (15-63, 71-78), CNN architectures (8-36)
10 Sep. 26 Detection & segmentation background YOLOv2, Redmon and Farhadi (CVPR 2017) Stanford CS231n Detection and Segmentation slides (15-27, 44-85, 89-94)
11 Oct. 1 TensorFlow programming and homework Deep learning software slides (21-33, 40-59)
my slides
Homework assigned
Oct. 3 NO CLASS
Instructor away
Oct. 8 NO CLASS
Instructor away
12 Oct. 10 Perception for stepping "Learning Locomotion over Rough Terrain using Terrain Templates", M. Kalakrishnan, J. Buchli, P. Pastor, and S. Schaal, IROS 2009 ANYmal video, ANYmal slides, ANYmal "Where Should I Walk?" video, Kalakrishnan slides
13 Oct. 15 NO CLASS
Instructor away
14 Oct. 17 Imitation learning Berkeley Levine course IL slides (DAgger defined on slide 14); CMU Fragkiadaki IL slides (1-29)


Homework due

15 Oct. 22
Withdraw deadline 		Reinforcement learning UCL Silver course: RL slides (7-42), RL video (15:40-21:20, 1:02:13-1:04:40 contains movies missing from PDF); Stanford deep-Q lecture slides (23-62); original deep-Q paper
16 Oct. 24 NO CLASS
Break before presentations start
17 Oct. 29 Student paper presentation Maxim Bazik, "PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud"
18 Oct. 31 Student paper presentation Woosik Lee, "Learning Wheel Odometry and IMU Errors for Localization"
19 Nov. 5 Student paper presentations Yi Liu, "Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation"
20 Nov. 7 Student paper presentations Chuchu Chen, "Deep Learning for 2D Scan Matching and Loop Closure"

Chirantan Ghosh, "Video Action Transformer Network"

21 Nov. 12 Student paper presentation Sha Liu, "Robots that can adapt like animals"

Seyedalireza Khoshsirat, "PCN: Point Completion Network"

22 Nov. 14 Student paper presentations Shengye Li, "Learning Common and Specific Features for RGB-D Semantic Segmentaion with Deconvolutional Networks"

Eric Wright, "Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight"

23 Nov. 19 Student paper presentation Shivanand Sheshappanavar, "PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows"
Zhang Guo, "Believe It or Not, We Know What You Are Looking at!"
24 Nov. 21 Student paper presentations Nikhil Gothankar, "Fast Online Object Tracking and Segmentation: A Unifying Approach"

Vineet Singh, "Spatial Transformer Networks"

Nov. 26 NO CLASS
Thanksgiving break
Nov. 28 NO CLASS
Thanksgiving break
25 Dec. 3 Student paper presentation Md Mottalib, "An Empirical Evaluation of Deep Learning on Highway Driving"
26 Dec. 5, 6 Final project presentations
Retrieved from "http://nameless.cis.udel.edu/class_wiki/index.php?title=CISC849_F2019&oldid=1737"