Using artificial intelligence to program autonomous robots: Natural language instructions to concrete robot actions

Abstract

Automation and robotics have reached a significant position in today’s society by increasing efficiency, safety and quality of life. As technology develops, the number of application areas for robots and the level of automation are constantly on the rise. Already, the tasks for autonomous robots are diverse and working domains range from vast logistic centers and harbors to grocery stores and individual households. As the amount of application areas increase and robots evolve, the importance of robots’ ability to function and adapt to new and complex environments grows evermore. This paper is a literature review based on previous research of the area and presents two central problems. Implementation of new functionality on a robot requires significant amount of expertise as well as time and other resources, especially funds. In addition, communication and information sharing between humans and robots in shared working environments must be enhanced for cooperation to be as natural and fluent as possible. To reach these goals, new operating models and adequate tools must be developed. In this review, ways to utilize artificial intelligence to enhance human-robot communication and bridge the gap between end-users and the implementation of robot actions are explored. The main goal of the review is to combine results and good practices from previous research and offer a look into techniques that enable the instructing of robots using natural language. The paper answers the questions: “What phases are required when transforming natural language instructions into robot action plans using large language models” and “What data is required to construct robotic action plans and how to gather and present this data?” The main contribution of this paper is a pipeline based on previous research that describes the necessary steps to convert natural language instructions to concrete robot actions. Seven separate phases were identified which included task division into subtasks, construction of natural language motion plan, generation of concrete motion plan, task execution, failure detection, explanation forming and replanning.