Did you participate in iRobot's data collection efforts? We'd love to hear from you. Please reach out at [email protected]. Ultimately, though, this set of images represents something bigger than any one individual company’s actions. They speak to the widespread, and growing, practice of sharing potentially sensitive data to train algorithms, as well as the surprising, globe-spanning journey that a single image can take—in this case, from homes in North America, Europe, and Asia to the servers of Massachusetts-based iRobot, from there to San Francisco–based Scale AI, and finally to Scale’s contracted data workers around the world (including, in this instance, Venezuelan gig workers who posted the images to private groups on Facebook, Discord, and elsewhere). Together, the images reveal a whole data supply chain—and new points where personal information could leak out—that few consumers are even aware of. iRobot says that sharing images in social media groups violates Scale’s agreements with it, and Scale says that contract workers sharing these images breached their own agreements. Alarm bells are sounding around Artificial Intelligence (AI) and the people behind the technology once again. Images of a young woman sitting on a toilet seat, taken by a cleaning robot, and leaked to closed social media groups on Facebook and Discord, are the epicentre of the newest controversy against a technology that sci-fi films have long poked around. The image leak, first reported by the MIT Tech Review, was taken as a test version of the Roomba. The woman pictured was not a customer, but either a volunteer or an employee of iRobot, the manufacturer of the Roomba robot vacuum cleaner.

The 15 images shared with MIT Technology Review are just a tiny slice of a sweeping data ecosystem. iRobot has said that it has shared over 2 million images with Scale AI and an unknown quantity more with other data annotation platforms; the company has confirmed that Scale is just one of the data annotators it has used. You have to assume that people … ask each other for help. The policy always says that you’re not supposed to, but it’s very hard to control.” It’s not expected that human beings are going to be reviewing the raw footage,” emphasizes Justin Brookman, director of tech policy at Consumer Reports and former policy director of the Federal Trade Commission’s Office of Technology Research and Investigation. iRobot would not say whether data collectors were aware that humans, in particular, would be viewing these images, though the company said the consent form made clear that “service providers” would be. Most robot vacuum companies MIT Technology Review spoke with explicitly said they don’t use customer data to train their machine-learning algorithms. Samsung did not respond to questions about how it sources its data (though it wrote that it does not use Scale AI for data annotation), while Ecovacs calls the source of its training data “confidential.” LG and Bosch did not respond to requests for comment. The remaining training data comes from what iRobot calls “staged data collection,” in which the company builds models that it then records.

This data is then used to build smarter robots whose purpose may one day go far beyond vacuuming. But to make these data sets useful for machine learning, individual humans must first view, categorize, label, and otherwise add context to each bit of data. This process is called data annotation. In iRobot’s case, over 95% of its image data set comes from real homes, whose residents are either iRobot employees or volunteers recruited by third-party data vendors (which iRobot declined to identify). People using development devices agree to allow iRobot to collect data, including video streams, as the devices are running, often in exchange for “incentives for participation,” according to a statement from iRobot. The company declined to specify what these incentives were, saying only that they varied “based on the length and complexity of the data collection.” When I ask Kevin Guo, the CEO of Hive, a Scale competitor that also depends on contract workers, if he is aware of data labelers sharing content on social media, he is blunt. “These are distributed workers,” he says. “You have to assume that people … ask each other for help. The policy always says that you’re not supposed to, but it’s very hard to control.” So, what exactly was iRobot doing recording its unreleased product’s samplers? It was sending the data to another party- San Francisco-based Scale AI. From there, the data was going to Scale’s contracted data workers, who were sitting in Venezuela. Scale AI’s contractors, or data labellers, were working on a project for iRobot to tag photos. All the photos the Roomba took, would label to teach the machine to recognize objects in their surroundings better.