Large Language Models (LLMs, or n-gram models on steroids) that have been trained originally to generate text by repeatedly predicting the next word in the context of a window of previous words, have captured the attention of the AI (and the world) community. Part of the reason for this is their ability to produce meaningful completions for prompts relating to almost any area of human intellectual endeavors. This sheer versatility has also led to claims that these predictive text completion systems may be capable of abstract reasoning and planning. In this tutorial we take a critical look at the ability of LLMs to help in planning tasks–either in autonomous modes, or in assistive modes. We are particularly interested in characterizing these abilities–if any–in the context of problems and frameworks widely studied in the AI planning community.
The tutorial will both point out the fundamental limitations of LLMs in generating plans that will normally require resolving subgoal interactions with combinatorial search, and also show constructive uses of LLMs as complementary technologies to the sound planners that are developed in the AI Planning community. In addition to presenting our own work in this area, we provide a critical survey of many related efforts, including by researchers outside of the planning community.
Subbarao Kambhampati is a professor of computer science at Arizona State University. Kambhampati studies fundamental problems in planning and decision making, motivated in particular by the challenges of human-aware AI systems. He is a fellow of Association for the Advancement of Artificial Intelligence, American Association for the Advancement of Science, and Association for Computing machinery, and was an NSF Young Investigator. He served as the president of the Association for the Advancement of Artificial Intelligence, a trustee of the International Joint Conference on Artificial Intelligence, the chair of AAAS Section T (Information, Communication and Computation), and a founding board member of Partnership on AI. Kambhampati’s research as well as his views on the progress and societal impacts of AI have been featured in multiple national and international media outlets. He can be followed on Twitter @rao2z.
Home
Karthik Valmeekam is a third-year Ph.D. student at Arizona State University working at the Yochan Lab under the guidance of Prof. Subbarao Kambhampati. His research primarily focuses on Large Language Models (LLMs) and reasoning, with a special emphasis on exploring the planning abilities of LLMs. This includes understanding the various roles that LLMs can play in planning and reasoning about actions and change. He has also made contributions in areas like Human Aware AI Planning and Preference Based Reinforcement Learning. His research has been recognized at major AI conferences such as NeurIPS, ICLR, and ICAPS.
Home
Lin Guan is a fifth-year PhD student at Arizona State University under the supervision of Prof. Subbarao Kambhampati. His research primarily focuses on building intelligent decision-making agents through methods such as reinforcement learning from human feedback (i.e., RLHF) and plan generation with large language models (i.e., LLM-based AI agents). His research has been recognized at top-tire AI conferences such as NeurIPS, ICLR, and ICML.
HomeS. Kambhampati, K. Valmeekam, M. Marquez & L. Guan. (2023, July). On the Role of Large Language Models in Planning. Tutorial presented at the International Conference on Automated Planning and Scheduling (ICAPS), Prague. https://yochan-lab.github.io/tutorial/ICAPS-2023/.
@misc{kambhampati2023role,
author = {Kambhampati, Subbarao. and Valmeekam, Karthik. and Marquez, Matthew. and Guan, Lin.},
title = {On the Role of Large Language Models in Planning},
year = {2023},
month = {July},
note = {Tutorial presented at the International Conference on Automated Planning and Scheduling (ICAPS), Prague},
url = {https://yochan-lab.github.io/tutorial/ICAPS-2023/}
}