CS 591: Intelligent Assistive Robotics

Spring 2018 | Fridays | 10:45am - 1:15pm | BYAC 260

Instructor Siddharth Srivastava

Office hours Tuesdays and Wednesdays 2-3pm, BYENG 592

Contact details BYENG 592 | siddharths@asu.edu | 480-727-7451 (email is strongly preferred)

Join us on Piazza!

The focus of this course is on techniques for making robots intelligent enough to be able to assist humans. This topic constitutes a very active area of research in academia as well as industry, with multiple academic and commercial organizations jumping in to explore and develop the potential of robots that help humans. We will learn advanced tools for robotics that empower autonomous systems such as self-driving cars and study how they can be extended to enable versatile assitive robots.


This course covers some of the recent advances in intelligent robotics, particularly focusing on aspects relating to the theory and practice of getting robots to reason about and accomplish assistive tasks in indoor environments. Students will have the opportunity to learn how to use robotics simulators, and to gain an understanding of this active research area including topics from motion planning, perception, learning and reasoning under uncertainty with a special reference to mobile manipulation. Final projects will be used to apply the learned knowledge and test new ideas in simulation or on a physical robot.

Course format

The course is designed as a seminar class that will start with a brief recap of the fundamentals and move on to discussion and evaluation of prominent research papers. Students will play a key role in the selection of papers in the later half of the course. Weekly meetings will be organized as instructor and student led discussions of the selected papers. A detailed schedule of presentations will be developed with the students at the beginning of the semester.

Projects Students will propose and implement final projects. Given the challenging nature of the topic, students will have flexibility in the design and nature of projects, going from novel algorithmic solutions for subproblems to implementations of existing techniques on an application designed by the team.

Project development will be an essential component and students will be guided from early stages of their projects. Project proposals will be due 6 weeks from the start of the course. Most projects are expected to be demonstrated using robotics simulators; exceptionally performing teams will get the opportunity to demonstrate their work on a physical robot towards the end of the semester.

Grading and expectations

There will be no midterm or final exams. Each team will be expected to produce a final report with clear indications of each team member’s role in the project. Each student will submit critiques for a subset (to be determined) of the selected research papers and lead the discussion of at least one of the selected papers. Grades will be assigned based on students' critiques and discussion-leads (30%), innovation in the final project’s design and implementation (35%) and final report (35%).


471 or 571 or Instructor’s Approval

Tentative Schedule

To be refined during the first few classes.

Date Topic(s) Papers
1/12 Introduction, recap of task planning and motion planning
1/19 Motion planning and Robot simulators
1/26 Going beyond motion planners
  • Gravot, Fabien, et al. "aSyMov: a planner that deals with intricate symbolic and geometric problems." ISRR 2005.
  • Plaku, Erion, et al. "Sampling-based motion and symbolic action planning with geometric and differential constraints." ICRA 2010
  • 2/2 Constraint based TMP solvers
  • Garrett, Caelan, et al. "Sample-based methods for factored task and motion planning." RSS 2017
  • Dantam, Neil T., et al. "Incremental task and motion planning: a constraint-based approach." RSS 2016.
  • 2/9 Hierarchical SDM
  • Marthi, Bhaskara, et al. "Angelic semantics for high-level actions." 2007.
  • Srivastava, Siddharth, et al. "Metaphysics of planning domain descriptions." AAAI 2016.
  • 2/16 TMP, continued
    Project proposals due
  • Srivastava, Siddharth, et al. "Combined task and motion planning through an extensible planner-independent interface layer." ICRA 2014.
  • Konidaris, George, et al. "Constructing symbolic representations for high-level planning." AAAI. 2014.
  • 2/23 Learning and using hierarchies for SDM
  • Hofmann, Till, et al. "Initial results on generating macro actions from a plan database for planning on autonomous mobile robots" ICAPS 2017.
  • Nan, Rong, et al. "MDPs with unawareness in robotics." UAI 2016.
  • 3/2 Perception for navigation
  • Durrant-Whyte, Hugh, et al. "Simultaneous localization and mapping (SLAM): Part I." IEEE RAS magazine,13(2), 99-110.
  • Bailey, Tim, et al. "Simultaneous localization and mapping (SLAM): Part II." IEEE RAS Magazine, 13(3), 108-117.
  • Devin, Coline, et al. "Learning modular neural network policies for multi-task and multi-robot transfer." ICRA 2017 [Learning for manipulation]
  • 3/16 Perception for manipulation
  • Xiang, Yu, et al. "Objectnet3d: A large scale database for 3d object recognition." ECCV 2016.
  • Mahler, Jeffrey, "Dex-net 1.0: A cloud-based network of 3d objects for robust grasp planning using a multi-armed bandit model with correlated rewards." ICRA 2016
  • 3/23 Learning for manipulation
  • Vicente, Pedro, et al. "Towards markerless visual servoing of grasping tasks for humanoid robots." ICRA 2017
  • Fern, Alan et al. "A Decision-Theoretic Model of Assistance", JAIR 2014 [Shared Autonomy]
  • 3/30 Shared autonomy
  • Javdani, Shervin, et al. "Shared autonomy via hindsight optimization". RSS 2015.
  • Milliken, Lauren, et al. "Modeling user expertise for choosing levels of shared autonomy." ICRA 2017.
  • 4/6 Explainable behavior
  • Ribeiro, Marco Tulio, et al. "Why should I trust you?: Explaining the predictions of any classifier". KDD 2016.
  • Hayes, Bradely, et al. "Improving robot controller transparency through autonomous policy explanation." HRI 2017.
  • 4/13 User-friendly robot control
  • Coronado, Enrique, et al. "Gesture-based robot control: Design challenges and evaluation with humans." ICRA 2017.
  • Huang, Justin, et al. "Code3: A system for end-to-end programming of mobile manipulator robots for novices and experts." ICRA 2017.
  • 4/20 Communication and assistive robotics
  • Huang, Sandy, et al. "Enabling robots to communicate their objectives." RSS 2017.
  • Broad, Alexander, et al. "Path planning under interface-based constraints for assistive robotics." ICAPS 2016.
  • 4/27 Project reports due Project presentations

    Absence and Make-up Policies

    If a student is unable to present on a scheduled date due to extenuating circumstances, an arrangement will be made to reschedule their presentation. Accommodations will be made for religious observances provided that students notify the instructor at the beginning of the semester concerning those dates. Students who expect to miss class due to officially university-sanctioned activities should inform the instructor early in the semester. Alternative arrangements will generally be made for any examinations and other graded in-class work affected by such absences. The preceding policies are based on ACD 304–04, “Accommodation for Religious Practices” and ACD 304–02, “Missed Classes Due to University-Sanctioned Activities.”

    Classroom Behavior

    Cell phones and pagers must be turned off during class to avoid causing distractions, unless instructed by the presenters (e.g., for online polling software). The use of recording devices is not permitted during class. Any violent or threatening conduct by an ASU student in this class will be reported to the ASU Police Department and the Office of the Dean of Students.

    Academic Integrity

    All students in this class are subject to ASU’s Academic Integrity Policy (available at http://provost.asu.edu/academicintegrity) and should acquaint themselves with its content and requirements, including a strict prohibition against plagiarism. All violations will be reported to the Dean’s office, who maintain records of all offenses. Students are expected to abide by the FSE Honor Code (http://engineering.asu.edu/integrity/).

    Disability Accommodations

    Suitable accommodations will be made for students having disabilities and students should notify the instructor as early as possible if they will require same. Such students must be registered with the Disability Resource Center and provide documentation to that effect.

    Sexual Discrimination

    Title IX is a federal law that provides that no person be excluded on the basis of sex from participation in, be denied benefits of, or be subjected to discrimination under any education program or activity. Both Title IX and university policy make clear that sexual violence and harassment based on sex is prohibited. An individual who believes they have been subjected to sexual violence or harassed on the basis of sex can seek support, including counseling and academic support, from the university. If you or someone you know has been harassed on the basis of sex or sexually assaulted, you can find information and resources at https://sexualviolenceprevention.asu.edu/faqs. As a mandated reporter, I am obligated to report any information I become aware of regarding alleged acts of sexual discrimination, including sexual violence and dating violence. ASU Counseling Services, https://eoss.asu.edu/counseling, is available if you wish discuss any concerns confidentially and privately.

    Notice: Any information in this syllabus (other than grading and absence policies) may be subject to change with reasonable advance notice.

    Notice: All contents of these lectures, including written materials distributed to the class, are under copyright protection. Notes based on these materials may not be sold or commercialized without the express permission of the instructor. [Note: Based on ACD 304-06.]