Lab teaches robot to put down objects properly

Cornell University researchers, using a 3D camera, have taught a robot how to put down simple household objects accurately in office and home scenes.


After scanning a room, a robot points to the keyboard it was asked to locate. It uses context to identify objects, such as the fact that a keyboard is usually in front of a monitor. Courtesy: Cornell University Personal Robotics LabFor robots, picking up objects is easy. Putting them down – in proper context – is not so simple.

Researchers at Cornell’s Personal Robotics Laboratory, led by Ashutosh Saxena, assistant professor of computer science, has found that placing objects for robots is harder because there are many options. Drinking cups, for example, go upright when placed on a table, but must be placed upside down in a dishwasher.

In early tests, the researchers placed a plate, mug, martini glass, bowl, candy cane, disc, spoon and tuning fork on a flat surface, on a hook, in a stemware holder, in a pen holder and on several different dish racks.

The research robot surveyed its environment with a 3D camera, then it randomly tested space as suitable locations for placement. For some objects the robot tests for “caging” – the presence of vertical supports that would hold an object upright. It also gives priority to “preferred” locations: A plate goes flat on a table, but upright in a dishwasher.

After training, the Saxena lab’s robot placed objects correctly 98 percent of the time when it had seen the objects and environments previously. When working with new objects in new environments, the robots placed the objects correctly 92 percent of the time.

But first, the robot has to find the dish rack. Just as humans assess a room when walking in, the Cornell researchers (Saxena and colleague Thorsten Joachims, Cornell associate professor of computer science) have developed a system that enables a robot to scan a room and identify its objects.

Pictures from the robot’s camera are stitched together to form a 3-D image of the entire room, which is then divided into segments.

The researchers trained a robot by giving it 24 office scenes and 28 home scenes in which they had labeled most objects. The computer examines such features as color, texture and what is nearby and decides what characteristics all objects with the same label have in common. In a new environment, it compares each segment of its scan with the objects in its memory and chooses the best fit.

In tests, the robot correctly identified objects about 73 percent of the time in home scenes and 84 percent in offices. In a final test, the robot successfully located a keyboard in an unfamiliar room. Again, context gives this robot an advantage. The keyboard only shows up as a few pixels in the image, but the monitor is easily found, and the robot uses that information to locate the keyboard. Saxena will present some of this research at the Neural Information Processing Systems conference in Granada, Spain, in December 2011.

Cornell University

- Edited by Chris Vavra, Control Engineering, 

No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Choosing controllers: PLCs, PACs, IPCs, DCS? What's best for your application?; Wireless trends; Design, integration; Manufacturing Day; Product Exclusive
Variable speed drives: Smooth, efficient, electrically quite motion control; Process control upgrades; Mobile intelligence; Product finalists: Vote now; Product Exclusives
Machine design tips: Pneumatic or electric; Software upgrades; Ethernet advantages; Additive manufacturing; Engineering Leaders; Product exclusives: PLC, HMI, IO
This article collection contains the 5 most referenced articles on improving the use of PID.
Learn how Industry 4.0 adds supply chain efficiency, optimizes pricing, improves quality, and more.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Cyber security cost-efficient for industrial control systems; Extracting full value from operational data; Managing cyber security risks
Drilling for Big Data: Managing the flow of information; Big data drilldown series: Challenge and opportunity; OT to IT: Creating a circle of improvement; Industry loses best workers, again
Pipeline vulnerabilities? Securing hydrocarbon transit; Predictive analytics hit the mainstream; Dirty pipelines decrease flow, production—pig your line; Ensuring pipeline physical and cyber security