Welcome to the Humanoids 2023 Workshop:

Generalizable and Robust Decision Making, Planning, and Control for Humanoid Loco-Manipulation

9:00am – 5:30pm, Tuesday, Dec 12th, 2023, Autstin, TX

Room 1.516 (Level 1), Engineering Education and Research Center (EER)

Online Webinar: https://gatech.zoom.us/j/98644386372

Objective and Scope

Humanoid robots capable of simultaneous execution of locomotion and manipulation tasks can aid in a wide range of critical real-life applications such as search and rescue, emergency response, home assistance, manufacturing, and delivery and courier services. However, traditional algorithms of decision making, planning, and control, either model-based or data-driven, lack robustness and generalizability in the face of the broad spectrum of real-world uncertainties such as unknown contact modes and changing operating conditions. This workshop is intended to cover the key challenges, advances, and future directions of various aspects of humanoid loco-manipulation, such as decision making, planning, learning, control, sensing, and mechatronic design. The workshop will solicit posters and videos through the submission of extended abstracts for an interactive session for all the participants. Furthermore, with the authorization of the authors, contributed extended abstract, posters, and videos (including the recording of the workshop presentations) will be included on the official workshop website and Youtube after the event. Summarizing, the objectives of the workshop are:
(1) to create awareness about this growing field of research;
(2) present a selection of the state-of-the-art and ongoing research activities on loco-manipulation;
(3) engage and solicit additional contributions from the robotic community;
(4) provide an opportunity for a community discussion and reflection on loco-manipulation.

List of Organizers

Yan Gu (IEEE RAS member), Purdue University, yangu@purdue.edu (Primary Contact Person)

Ye Zhao (IEEE RAS Senior Member), Georgia Institute of Technology, ye.zhao@me.gatech.edu

Xuan Lin (IEEE RAS member), University of California, Los Angeles, maynight@ucla.edu

Zhaoyuan Gu (IEEE RAS student member), Georgia Institute of Technology, zgu78@gatech.edu

List of Topics

Invited Speakers

Here is a list of invitees who have agreed to present:

Title: State Estimation and Control of Underactuated Humanoid Walking on a Nonstationary Surface

Details

Abstract: Legged robots have the potential to assist humans with a wide range of real-world tasks in dynamic, unstructured environments, such as search and rescue on disaster sites, monitoring of natural resources, space exploration, home assistance, and delivery and courier. While legged locomotion on stationary (regular or irregular) surfaces has been extensively studied, legged locomotion on dynamic surfaces, which include surfaces that globally move in the inertial frame (e.g., ships, aircraft, and trains), remains a new robot functionality that has not been tackled. This new functionality will empower legged robots to perform critical, high-risk tasks such as shipboard firefighting and fire suppression as well as disinfection on moving public transportation vehicles to help contain the spread of infectious diseases. Yet, enabling reliable locomotion on dynamic surfaces presents substantial fundamental challenges in legged robot control due to the high complexity of the hybrid, time-varying physical interaction between the robot and the surface. In this talk, Dr. Gu will present the current progress from her research group in creating new methods of modeling, state estimation, and control of legged robots that achieve provably stable locomotion on dynamic surfaces by explicitly addressing the associated hybrid, time-varying robot dynamics.

Bio: Dr. Yan Gu received her B.S. degree in Mechanical Engineering from Zhejiang University (China) in 2011 and her Ph.D. degree in Mechanical Engineering from Purdue University in 2017. She was an Assistant Professor in the Department of Mechanical Engineering at the University of Massachusetts Lowell (UML) as an Assistant Professor between 2017 and 2022. She has been with the faculty of the School of Mechanical Engineering at Purdue University since Fall 2022. Her research focuses on nonlinear control and hybrid systems with applications to legged locomotion, including bipedal and quadrupedal robot locomotion and exoskeleton-assisted human walking. Her long-term research goal is to realize provably safe and autonomous legged locomotion in dynamic, unstructured environments. Towards reaching this goal, her research draws upon nonlinear control theory, theory of hybrid systems, dynamics, and optimization to create new methods of modeling, state estimation, and control of legged locomotion that explicitly address the complex physical interaction between the robot and the environment. She received the ONR Young Investigator Program Award in 2023, the NSF CAREER Award in 2021, and the Verizon’s 5G Robotics Challenge Award in 2019. Her research on legged locomotion has been funded by NSF, ONR, ARL, and Verizon’s 5G Lab and reported by various media such as Boston Globe, CNET, Robotics Business Review, and NPR’s WBUR.

Title: Towards fast mixed-integer quadratic programming algorithms for real-time loco-manipulation control. slides.

Details

Abstract: Conventionally, mixed-integer quadratic programmings (MIQPs) are too slow due to their NP-hard nature, limiting their applications to offline motion planning. In this talk, we present recent progress on an old technique – Generalized Benders Decomposition – that allows our MIQPs algorithms to match the solving speed of the commercially-adopted Branch and Bound technique on mixed-logic dynamic control problems. The controller learns discrete control policies fast and data-efficiently, allowing quick adaptation to new problems. We will also discuss the applications of fast MIQP algorithms such as humanoid loco-manipulation control.

Bio: Xuan Lin is a postdoc researcher at Robotics and Mechanisms Laboratory, University of California, Los Angeles (UCLA). He received M. Sc. in 2019, and his Ph.D. in 2023, both in Mechanical Engineering from UCLA. Xuan Lin’s research interests range from robot hardware to control algorithms. He has previously developed several wall-climbing robots. His current interest is developing fast optimization algorithms for loco-manipulation planning and control.

Title: BRUCE – A kid-size humanoid robot open-platform for research and education

Details

Abstract: Significantly faster development in quadruped robots than humanoid robots has been witnessed recently in terms of locomotion capability. Compared to quadruped robots, humanoid robots are typically more mechanically complex, requiring more powerful actuators and more degrees of freedom, as well as being more intrinsically unstable. This poses two critical challenges in the study of humanoid robots. First, accessibility to the hardware platform is limited as either it takes too much effort to develop a humanoid robot platform independently or the commercially available ones, if any, are too expensive to afford. As we know, one of the biggest issues in developing robotics is the difference between the simulation and the real world, i.e., what works in the simulation will likely not be reflected exactly in the real world. It is therefore necessary to perform meticulous testing on the physical robot in order to adjust the system to function as desired. Second, as the humanoid robot system is highly nonlinear and complex, nominally underactuated and unstable, multi-input and multi-output, as well as time-variant and hybrid (considerably more challenging than the quadruped robot system), more advanced and efficient control algorithms are essential especially for robust bipedal locomotion. To address some of the problems in these challenges, we have been developing a next-generation kid-size humanoid robot capable of dynamic behaviors for general research purposes. The robot is named BRUCE – Bipedal Robot Unit with Compliance Enhanced. BRUCE is open-source (both hardware and software), safe to test, easy to maintain, with an affordable price for the robotics community.

Bio: Junjie Shen earned his Ph.D. and M.S. in Mechanical Engineering from University of California, Los Angeles (UCLA) in 2022, and his B.S. in Mechanical Engineering from Shanghai Jiao Tong University in 2016. He worked in the Robotics and Mechanisms Laboratory (RoMeLa) at UCLA from 2016 to 2022. His research interests focus on optimization-based motion planning and control of legged robots. He (and his coauthors) won IROS Best Paper Award in 2019. Currently, he is working with humanoid robots at Westwood Robotics.

Title: Key Components of Expeditious Legged Robots: Balancing Control, Energy Efficiency, and Power Amplification

Details

Abstract: Legged robots, especially humanoid robots, are showing great potential in solving societal problems. Their hybrid high-dimensional dynamics present the first challenges in the feedback controls for balancing, solving which provides a first step feasibility evaluation of the applications of loco-manipulation towards general purpose legged robots. The next steps must involve (re)looking at energy efficiency and power amplification on legged robots, so that they can be expeditious, performing a wide range of tasks with different power requirements and for a long duration of operation. In this talk, I will first present a step-level balancing control framework for engineering versatile, adaptive, and robust bipedal walking behaviors. Then, I will focus on two recent preliminary work from my lab, looking at using novel legged design to promote energy efficiency and power amplification with realizations of highly efficient/dynamic behaviors that cannot be realized by the original robotic systems. Finally, I will touch upon a humanoid robot I am building with some personal perspectives on legged robotic control, efficiency, and power.

Bio: Xiaobin Xiong is currently an assistant professor in the Department of Mechanical Engineering at the University of Wisconsin-Madison. His research centers around robotic legged locomotion, aiming to provide novel system solutions with rigorous control methodologies for realizing expeditious legged locomotion. Broadly speaking, his interests are on the enhancement of the physical intelligence of legged robots to move and work in human society. He received his B.S. from Tongji University, China in 2013, M.S. from Northwestern University in 2015, and Ph.D. from Caltech in 2021, all in mechanical engineering.

Title: Three new perspectives for legged navigation: temporal-logic-guided robustness, safety, and social intelligence

Details

Abstract: While legged robots have made remarkable progress in dynamic balancing and mobility, there remains substantial room for improvement in terms of navigation and decision-making capabilities. One major challenge stems from the difficulty of designing safe, resilient, and real-time planning and decision-making frameworks for these complex legged machines navigating unstructured environments. Symbolic task planning and formal control methods offer promising yet underexplored solutions. This talk will introduce three perspectives on enhancing safety and resilience in task and motion planning (TAMP) for agile legged locomotion and navigation. First, we’ll discuss hierarchically integrated TAMP for dynamic locomotion in environments susceptible to perturbations, focusing on robust recovery behaviors. Next, we’ll cover our recent work on safe and socially acceptable legged navigation planning in environments that are partially observable and crowded with humans. Finally, we will briefly discuss our recent effort in rigid-soft triple-arm locomotion and manipulation.

Details

Bio: Jonathan Hurst is Chief Robot Officer and co-founder of Agility Robotics, and Professor and co-founder of the Oregon State University Robotics Institute. He holds a B.S. in mechanical engineering and an M.S. and Ph.D. in robotics, all from Carnegie Mellon University. Hurst’s research focuses on understanding the fundamental science and engineering best practices for robotic legged locomotion and physical interaction. Agility Robotics is bringing this new robotic mobile manipulation capability to market, solving problems for customers, working towards a day when robots can go where people go, generate greater productivity across the economy, and improve quality of life for all.

Title: Next iterations of quadratic programming for adaptive and robust motion control

Details

Abstract: Convex quadratic programming (QP) has become a major item in the robotics toolbox, with well-known applications including whole-body control, model predictive control (MPC), contact planning and state estimation. Current challenges when solving QP-formulated problems include feasibility (ensuring that a solution exists, e.g. when some problem parameters come from measurements), recursive feasibility (in MPC: ensuring the system does not steer towards unfeasible problems) and real-time performance. In this talk, we will review how next-generation QP solvers have become able to handle non-convex and unfeasible problems, returning principled solutions in any case, and how this enables their inclusion as differentiable functions in end-to-end trainable control pipelines.

Bio: Stéphane is a research scientist at the Inria Paris Robotics Lab. He received his M.Sc. in Computer Science from the École Normale Supérieure (ENS Paris) in 2012, and his Ph.D. in Mechano-informatics from the University of Tokyo in 2016. After graduation, Stéphane has worked at CNRS as tenured researcher and at ANYbotics AG as locomotion team lead before joining the robotics lab at Inria Paris where he is currently (having a blast) doing research at the interface between motion control, machine learning and computer vision. Stéphane is a proponent of open source robotics and contributes to projects like Upkie wheeled bipeds, robot_descriptions.py or the qpsolvers benchmark.

Title: Advancing Agility: From a single-legged hopper to quadrupeds and humanoids with actuated toes

Details

Abstract: In this talk, I delve into the pursuit of achieving human- and animal-like agility in robots. A comprehensive understanding of robot hardware, real-time control systems, dynamics, perception, and motion planning is fundamental in this quest. I will explore the intricacies of control architecture, emphasizing the harmonious integration of hardware and software. The talk will address challenges in classical control, such as feedback control bandwidth, uncertainty handling, and robustness, alongside high-level planning issues including step planning, perception integration, and trajectory optimization. I will present my approaches to these challenges, supported by experimental results from various legged robots. These include point-foot bipeds (e.g., Hume, Mercury, and Pat),  a robot with liquid-cooling viscoelastic actuators (Draco), the quadruped with proprioceptive actuators (Mini-Cheetah), and the recent progress in the hopping robot, StaccaToe, featuring an actuated toe mechanism. Central to our discussion is the relationship between hardware limitations and controller design, highlighting the importance of a systematic approach in mastering dynamic locomotion control in robotics.

Bio: Donghyun is an Assistant Professor at the University of Massachusetts Amherst. Prior to joining UMass, he was a postdoctoral research associate at MIT’s Biomimetic Robotics Lab from 2019 to 2020, and at the University of Texas at Austin’s Human-Centered Robotics Lab in 2018, where he also completed his Ph.D. in 2017. Donghyun’s research primarily focuses on developing control architectures for dynamic legged robots with a strong emphasis on the experimental validation of his formulations. He designs control systems that integrate sensing, planning, and feedback controllers, accounting for the dynamics, real-time constraints, and hardware limitations. Currently, Donghyun is broadening his research horizons to include perception and machine intelligence, aiming to increase the adaptability and versatility of robotic systems across diverse terrains. His work has been recognized with a best paper award in Transactions on Mechatronics in 2020. His 2022 ICRA paper was nominated as a finalist for the outstanding dynamics and control paper, and his 2016 paper in Transactions on Robotics was selected as a finalist for the best whole-body control paper and video.

Title: Towards general loco-manipulation control of the 1X Androids

Details

Abstract: Advanced solutions for locomotion or loco-manipulation, encompassing closed-form solutions using simplified models, online optimization, or reinforcement learning that utilizes simulation, are fundamentally dependent on the ability to accurately model the dynamics of the problem. In this talk we will present our recent work on robot control and motion planning and provide an overview how it enables our Androids to collect data to perform autonomous manipulation tasks. In particular, we will highlight how we use different model structures, including template models, centroidal and whole-body dynamics, for Model Predictive Control of out high DoF humanoid robots and discuss how we address the arising challenges including energy efficient locomotion, natural gaits and computational complexity.

Bio: Manuel Yves Galliker is the team lead controls and embedded at 1x Technologies. He holds a B.Sc. and M.Sc. in mechanical engineering with a focus on robotics, systems and controls from ETH Zurich. Manuel’s research interests range from mechantronics to optimal and data-driven control. During his academic tenure, he dedicated his efforts to researching Model Predictive Control for legged robots, contributing at ETH’s Robotics Systems Lab and as a visiting researcher at Caltech’s AMBER lab. His work culminated in a publication presented at the previous year’s Humanoids conference, where it was distinguished as a finalist for the Best Paper Award. In his current role he is leading the R&D efforts on controls and embedded for 1X new bipedal Android NEO.

Program

Time (CDT)TalkSpeakerComments
8:45 – 9:00Welcome & Introduction
9:00 – 9:30Next iterations of quadratic programming for adaptive and robust motion controlStéphane Caron (Remote)Talk 1
9:30 – 10:00Advancing Agility: From a single-legged hopper to quadrupeds and humanoids with actuated toesDonghyun KimTalk 2
10:00 – 10:30State Estimation and Control of Underactuated Humanoid Walking on a Nonstationary SurfaceYan GuTalk 3
10:30 – 11:00Coffee break
11:00 – 11:30Jonathan W. HurstTalk 4
11:30 – 12:00Key Components of Expeditious Legged Robots: Balancing Control, Energy Efficiency, and Power AmplificationXiaobin XiongTalk 5
12:00 – 12:30Panel Session: Open challenges and future directions in humanoid loco-manipulationAll Morning SpeakersPanel Discussion 1
12:30 – 14:00Lunch
14:00 – 14:15Human-in-the-loop Learning for HumanoidsKyutae Sim, Hye-Young ChungShort Talk 1
14:15 – 14:30SCALER: Versatile Multi-Limbed Robot for Free-Climbing in Extreme
Terrains
Robot demo
14:30 – 15:00Towards fast mixed-integer quadratic programming algorithms for real-time loco-manipulation controlXuan LinTalk 6
15:00 – 15:30Three new perspectives for legged navigation: temporal-logic-guided robustness, safety, and social intelligenceYe ZhaoTalk 7
15:30 – 16:00Coffee break
16:00 – 16:30BRUCE – A kid-size humanoid robot open-platform for research and educationJunjie ShenTalk 8
16:30 – 17:00On controls and motion planning for automating physical labor in human spacesManuel Yves GallikerTalk 9
17:00 – 17:30Panel Session: Open challenges and future directions in humanoid loco-manipulationAll Afternoon SpeakersPanel Discussion 2
17:30End

Blog at WordPress.com.

Design a site like this with WordPress.com
Get started