Human inclinations and desires diverge, particularly in relation to the pursuit of cleanliness and order. In an ideal world, domestic automatons, particularly those intended to aid humans with household chores, should possess the capacity to execute tasks in a manner that accommodates these idiosyncratic preferences.

In recent times, esteemed scholars from Princeton University and Stanford University embarked upon an ambitious endeavor to personalize the assistance rendered by domestic automatons, harnessing the potential of large language models (LLMs). These artificial intelligence models have experienced a surge in popularity following the advent of ChatGPT. A research paper, which has been pre-published on arrive, outlines their method, which was initially tested on a mobile robot named Tidy Bot, specially engineered to restore order in indoor settings.


"In order for a robot to deliver personalized physical assistance with utmost efficacy, it must acquire knowledge of user preferences that can be consistently applied to future scenarios," articulated Jimmy Wu, Rika Antonova, and their fellow researchers in their publication. "In this undertaking, we explore the personalization of household cleaning activities through the utilization of robots capable of tidying up rooms by collecting and organizing objects."

The researchers proposed an innovative approach that capitalizes on the well-documented summarization capabilities inherent in LLMs such as ChatGPT. These models can condense information or offer general guidance after being trained on diminutive datasets or illustrative examples.

Within the context of their investigation, Wu, Antonova, and their peers employed an LLM to fabricate "summaries" of a user's proclivities when it pertains to tidying up, extrapolating from a few input prompts provided by the users themselves. For instance, a user might provide textual input such as "Garments of a crimson hue should be deposited within the drawer, while those of a white hue ought to be placed within the closet." Subsequently, the model generates generalized preferences that can serve as a guiding principle for a robot's actions.

"A fundamental quandary lies in determining the optimal placement for each object, as preferences can vary significantly depending on individual tastes and cultural backgrounds," elucidated Wu, Antonova, and their colleagues in their academic discourse.

"For instance, one individual might exhibit a predilection for storing shirts within a drawer, whereas another might prefer them adorning a shelf. Our objective entails constructing systems that can acquire such preferences from a mere handful of examples through prior interactions with a given individual. We illustrate that robots can amalgamate language-based planning and perception with the few-shot summarization capabilities of LLMs to discern generalized user preferences that possess broad applicability to future interactions."

To appraise the viability of their approach, the researchers undertook a series of experiments, evaluating both the generalized preferences elicited when feeding the model with data from text-based datasets, as well as its impact on the ability of an actual robot to personalize tidying activities. They specifically applied their methodology to Tidy Bot, an autonomous robot they designed that adeptly cleans floors while simultaneously collecting random objects from its vicinity and placing them in designated locations.

"This approach facilitates swift adaptation and achieves a remarkable accuracy rate of 91.2% when dealing with previously unseen objects within our benchmark dataset," asseverated Wu, Antonova, and their fellow researchers. "Moreover, we demonstrate the effectiveness of our approach on a real-world mobile manipulator dubbed Tidy Bot, which successfully organizes 85.0% of objects in real-world test scenarios."

The recent endeavors undertaken by this esteemed group of researchers underscore the immense potential harbored within LLMs. These models are not solely confined to assisting users with written tasks or answering inquiries but can also amplify the capabilities of robotic systems. In the times to come, this groundbreaking research may ignite a flame of inspiration within other teams, spurring them to explore the potential of employing these models within the realm of robotics.

The researchers' innovative LLM-based approach and the Tidy Bot robot they developed may soon contribute to the creation of advanced domestic automatons that possess the capability to execute chores and maintain orderly environments in perfect alignment with their users' preferences. Further studies may refine and enhance this methodology, enabling it to excel in excessively cluttered environments, for instance.

"Our implementation of the real-world system incorporates certain simplifications, such as the utilization of manually crafted manipulation techniques, the adoption of top-down grasping methods, and the presumption of pre-determined receptacle locations," the researchers expounded.

"These limitations could be overcome by integrating more sophisticated primitives into our system and expanding the capabilities of the perception system. Furthermore, due to the inability of mobile robots to traverse obstacles, the system would not function optimally in heavily cluttered settings. It would be intriguing to incorporate advanced high-level planning mechanisms, enabling the robot to reason not only about picking up the closest object but also about clearing a pathway through the clutter."