数字创新中心

Center for Digital Innovation

Cocobo: Exploring Large Language Models as the Engine for End-User Robot Programming

Yate Ge; Yi Dai; Run Shan; Kechun Li; Yuanda Hu; Xiaohua Sun

2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

摘要/Abstract

用户端开发允许普通用户根据自己的需求定制服务机器人或应用程序。一种用户友好的方法是自然语言编程。然而,它遇到了诸如用户表达空间庞大和调试与编辑支持有限等挑战,这限制了其在用户端编程中的应用。大型语言模型(LLMs)的出现为人类语言指令与机器人执行的代码之间的翻译和解释提供了有希望的途径,但它们在用户端编程系统中的应用需要进一步研究。我们介绍了Cocobo,这是一种由LLMs驱动的具有交互式图表的自然语言编程系统。Cocobo利用LLMs理解用户的创作意图,生成并解释机器人程序,并促进可执行代码与流程图表示之间的转换。我们的用户研究显示,Cocobo具有较低的学习曲线,即使是没有编程经验的用户也能成功定制机器人程序。

End-user development allows everyday users to tailor service robots or applications to their needs. One user-friendly approach is natural language programming. However, it encounters challenges such as an expansive user expression space and limited support for debugging and editing, which restrict its application in end-user programming. The emergence of large language models (LLMs) offers promising avenues for the translation and interpretation between human language instructions and the code executed by robots, but their application in end-user programming systems requires further study. We introduce Cocobo, a natural language programming system with interactive diagrams powered by LLMs. Cocobo employs LLMs to understand users’ authoring intentions, generate and explain robot programs, and facilitate the conversion between executable code and flowchart representations. Our user study shows that Cocobo has a low learning curve, enabling even users with zero coding experience to customize robot programs successfully.

相关信息/Info

作者/Authors

链接/Link

Yate Ge; Yi Dai; Run Shan; Kechun Li; Yuanda Hu; Xiaohua Sun

https://ieeexplore.ieee.org/document/10714576

图片/Figures