Smarter robot grasping with sensors, software, and the cloud
In robotics, end effectors are where the rubber meets the road. The robot “hand” is the ultimate touchpoint for every product or part that goes out the door. Smart factories and warehouses can only achieve the agile, connected, and collaborative environments envisioned by Industrie 4.0 if all the systems are intelligent and add value to the overall enterprise. As a result, robot grippers need to be smarter than average.
Hardware and software advances for robot grippers help enable safer, closer human-robot collaboration, ease of use, and flexibility for handling a wide variety of shapes and sizes. Plug-and-play features make them easier to integrate and implement, especially for small and midsize enterprises (SMEs) with day-to-day changeovers. Integrated sensors put 3-D vision, tactile, and force sensing in the palm of our hand. Algorithms orchestrate this into an intelligent solution that learns. Cloud-sharing, AI-enabled robots are teaching themselves how to grasp smarter and better.
Smart collaborative grippers
One of the ways robot grippers are getting smarter is by learning to play nice with humans. Automation has become less about replacing humans. Now, it’s more about humans sharing production responsibilities with their robot coworkers. Collaborative robots are at the center of the movement. This is focusing more attention on collaborative gripping solutions.
Schunk GmbH & Co. KG has been designing and producing robot grippers since the early 1980s. They see great potential for collaborative grippers that enable direct interaction and communication with humans. Co-act, which stands for collaborative actuator, is a family of collaborative grippers made by Schunk. The series is based on the company’s gripping technology with modifications to limit force and prevent other potential hazards when working closely with people. The company took its standard, electric 2-finger parallel gripper and built a protective housing around it with rounded corners to eliminate sharp edges and pinch points.
The Co-act EGP gripper is force limited to comply with Technical Specification ISO/TS 15066:2016 Robots and Robotic Devices – Collaborative Robots. Released in 2016, the technical spec provides data-driven guidelines for designers, integrators and users of human-robot collaborative systems to evaluate and mitigate risks. Annex A of ISO/TS 15066 contains data from a study on pain thresholds for different parts of the human body, including hands and fingers. It provides thresholds for maximum permissible pressure and force.
“The biggest difference between a standard EGP gripper and the EGP we use on the collaborative robot is that we safely limited the force to 140 N,” said Markus Walderich, automation group manager at Schunk Inc. in Morrisville, N.C. “We also made sure that if something should go wrong with the power supply, there’s no way the peak force could ever surpass 140 N.”
The Co-act gripper series is compatible with a variety of collaborative robots on the market.
“Our grippers are plug and play,” Walderich said. “You don’t need any adapter plate and the electrical connection is already provided so you can connect it right at the wrist. You don’t have to run a cable along the arm to wire it into the controller.”
Schunk stocks the Co-act grippers with different mechanical and electrical connections, making them plug-and-play ready for the different robot brands. Walderich said the gripper is easy to integrate and control.
“We use discrete signals to control it, simple 24 V signals,” he said, explaining that by using discrete signals, there’s no need for software drivers. “You only need one signal to open the gripper and one signal to close. Every robot already comes with a digital output that can open and close our gripper.”
The Co-act EGP-C also features an integrated LED status ring that will provide visual indication of the gripper state. Different colors will indicate a proper grip or an error state.
“Visually, you will be able to see right away if something is wrong,” Walderich said. “There’s integrated sensor feedback that tells you if the gripper is open or closed.”
The Co-act grippers are best suited for material handling, machine tending, and simple assembly tasks. The manufacturer is using them in its own factory to increase capacity and avoid worker injuries. In the largely manual assembly process for its trademark grippers, Schunk is using a collaborative robot equipped with a Co-act gripper to scrape components across a sharp-edged extraction plate to remove residual sealant material.
Walderich said the gripper’s main limitation is the force-limited operation. But what if you eliminate the likelihood of accidental human-gripper contact in the first place? That opens up a whole world of possibilities.
More sensors, more collaboration
Sophisticated sensors will bring a new level of smarts to the Schunk Co-act gripper family. The Co-act JL1 prototype is a “technology carrier” for demonstrating features that may be used in future collaborative grippers. The prototype won the Hermes Award for innovative industrial technology at the Hannover Messe exhibition in 2017.
“We put many possible technologies that we are aware of today in this prototype that makes it a safe, collaborative gripper,” says Walderich. “We will use what we’ve learned to derive future Co-act grippers with higher payloads. The next evolution of collaborative grippers will be higher force grippers that detect if there’s a human finger or hand in the gripping area and then it will not apply force higher than 140 N.”
The Co-act JL1 prototype gripper has a suite of sensors designed to track the proximity of humans and trigger evasive movements to avoid direct human contact. A capacitive sensor creates an electric field around the gripper to detect when anything containing a lot of water enters this field. That way, it can distinguish between a workpiece and a human body part. And it can do it within a narrow radius of 20 cm. If a human hand comes within proximity, the gripper automatically switches into safe operating mode.
A force-moment sensor detects unexpected force effects, such as a collision or malfunction. It also allows for manual guidance, positioning and teaching. Tactile sensors in the fingertips give the gripper a sense of touch. Then it can determine the exact gripping force acting on an object, allowing it to apply the appropriate amount of force for fragile items.
The prototype gripper has built-in 3-D cameras to help detect workpieces. An on-board touchscreen provides direct communication with the gripper for teaching or switching operating modes. Two different gripping types, parallel and angular, allow the JL1 to handle objects with varied geometries.
These advanced capabilities will help enable the agile manufacturing environments required for Industrie 4.0 and beyond, where humans and robots work collaboratively.
Ease of use and flexibility for SMEs
On Robot grippers were designed with collaborative applications in mind. The company says its grippers are built for “plug-and-produce” automation. This is especially advantageous for small and midsize enterprises with low-volume, high-mix production that needs to stay agile as needs change.
The electric 2-finger parallel grippers have on-board smarts enabled by software. This not only limits the force for use in human-robot collaborative applications, but also makes the servo grippers easy to integrate and implement.
“Beyond safety, there’s also the ease of use,” said Kristian Hulgard, VP of sales – North America for On Robot A/S, which is headquartered in Odense, Denmark. “When we go out and demo the product, it’s ready to pick and place items in 5 to 10 minutes. We cut a lot of engineering and programming hours out of the installation. That’s a huge part of what makes it collaborative.”
Hulgard said the computer numerical control (CNC) machining space is a huge market for collaborative robots, where collaborative robots are often used to load and unload the machines. Most of these companies are small to midsize mom-and-pop shops with production schedules that often change daily.
“They are running 200 parts one day and then 300 parts the next day,” Hulgard said. “With our smart gripper, the flexibility comes with being able to grab different sizes of objects with different force. You simply enter the size of the item you want to grab and how much force you want to apply, and away you go. The option to change the gripper’s functionality is a game changer. The return on investment is about 3 to 4 months. It’s easy to see the value in that.”
Software-enabled smart gripping
“You can mount our gripper on any robot you want and it will work. But where we see the value is how you control the gripper in the software,” Hulgard said. “Right now, the software is only for Universal Robots, but we will introduce that software for other robot brands in the future.”
Operators use the touchscreen tablet that comes with the collaborative robots to enter commands.
“When you install our software into the robot, you cannot tell that it’s third-party software,” Hulgard said. “It becomes an integrated part of the Universal Robots PolyScope software. The same way that you teach the collaborative robot, you can teach the gripper. It will do much of the work for you. Once you show the gripper the part to be picked by pushing the ‘close’ button on the screen, then it measures that part. With a click of a button, you then apply that size into your program. In the same programming process, you insert the force you want to apply to the part.”
Hulgard explained how you can set different force values depending on the nature of the items you’re handling. If it’s a CNC machine tending application, you might apply full force to ensure you have a strong grip on the metal part. For a packing application with a fragile item where you need to be careful not to crush the product, you would apply a small force.
The electrical connection is directly made to the tool flange so there’s no cable running the length of the robot arm to the controller. The direct connection also makes it possible for the UR collaborative robot to make endless rotations without getting entwined in the cable.
A dual gripper configuration is available for both payload models. With a dual gripper, a part can be unloaded from the CNC machine in the same pass that a new part is loaded into the machine for processing. This increases productivity by reducing cycle time.
“We go from a project time of about 29 seconds in that same cycle with a single gripper, down to about 17 seconds with the dual gripper. It’s almost half the time,” Hulgard said. “If we’re talking about huge batches where time is money, for the extra investment in the dual setup, it makes a lot of sense.”
By mounting the collaborative robot on a mobile base, the customer can easily move the machine tending station from one CNC machine to another and handle a wide range of components of varying shapes and sizes. It’s important to note no machine vision was used in this CNC machine tending application. The On Robot gripper is able to determine whether it’s gripping part A, B, or C by detecting the different widths of the parts.
Hulgard said the goal is to continue to improve based on feedback related to their hardware, software, or any other aspect related to their product.
“We added depth compensation in response to customer’s suggestions. Since our fingers grip in an arcing pattern, we need to compensate for the height that the fingers are arcing. If you have to grip something very flat to a table, like a coin, then you need to move the robot upward while you’re gripping, so the fingers don’t hit the table.”
With a lot of development and a little tweaking, software is making grippers smarter and more adept.
Not far from the Ivy League halls of Harvard University, another startup may get noticed for their innovative grippers. Once again, though, it’s the brain behind the brawn that’s noteworthy.
RightHand Robotics draws from several advanced technologies to automate individual item picking in warehouses and e-commerce fulfillment centers. They offer a hardware-software solution combining innovative grasping, advanced sensors, and artificial intelligence to ramp up the range and reliability of automated “piece-picking” in intralogistics.
The RightPick solution is a 3-finger robotic hand with a suction cup protruding from its palm was picking items from bins in the Honeywell Intelligrated booth with speed and accuracy. As shown in the video below, the RightPick was picking everything from bottles, tubes and even cans of soup, to boxes, bags and shrink-wrapped multipacks.
Courtesy: RightHand Robotics, Inc.
At the MODEX supply chain event in April, RightPick workcells operating in five exhibitor booths picked and placed 131,072 items over the duration of the show. The RightPick systems achieved pick rates of up to 1,000 units per hour across an assortment of items, which included products the system had never seen before.
It’s about reliability at rates nearly unimaginable just a few years ago.
“Over the course of the trade show, we can run as many picks as you would in a small warehouse over the course of a day,” said Leif Jentoft, one of the cofounders of RightHand Robotics, Inc. in Somerville, Mass. “For us, this was really about the reliability of the system. Our systems are ready for primetime.”
Clever grasping via the cloud
The RightPick piece-picking solution relies on a host of intelligent hardware and software technologies. A compliant, rubber-jointed fingered hand with a suction cup grasps various items with the aid of 3-D depth cameras and other sensors. The fingers help stabilize an item, so you can achieve a faster cycle time and pick heavier items. Computer vision helps the system figure out how to grasp the item. Artificial intelligence, specifically machine learning, is applied to improve the grasps over time. Data is shared with other robots via the cloud.
“Our core product is the RightPick.AI software and then the gripper working together. The brain and the hand,” Jentoft said. “The brain gets feedback from the gripper about what works and what doesn’t work, and is able to get better. We’re at the stage now where we have systems on the ground in warehouses in Japan, Europe, Canada and the U.S. and we’re using all of those to pool into the dataset, through the cloud.
“That’s really where the magic is. It’s about the datasets. We don’t think the winning proposition is training a robot on each different item. We think the value is in training a more general set of capabilities. We can take something we’ve never seen before and based on our experience, pick that up effectively.”
To understand how far this technology has come, it helps to take a look back. As a company, RightHand Robotics has been working on this challenge since its inception in 2014. The cofounders’ research, however, dates back much further. A closer look provides insight into some of the lesser-touted innovations behind the solution.
The RightHand Robotics’ technology team hails from the Harvard Biorobotics Lab, the GRAB Lab at Yale University, and the Massachusetts Institute of Technology. Many of them have doctorates in mechanical engineering, computer science, or robotics. Jentoft met his fellow cofounders, Yaro Tenzer and Lael Odhner, while the three were still in grad school.
“Yaro and I were at the Harvard Lab, and Lael was at the Yale lab, trying to build better robotic hands to handle things outside of carefully controlled factories,” Jentoft said. “In the factory you have the same part presented the same way a million times. We were trying to understand how you do something where you don’t have that predictability.”
The three university researchers ended up working on a joint project with iRobot Corporation to develop an end effector for the DARPA Robotics Challenge. That work led to the iRobot iHY under-actuated hand, which later won the competition.
“We had these soft, compliant fingers with mechanisms that make it easier to grab items. We had been studying those since the early 2000s in Rob Howe’s lab at Harvard,” Jentoft said. “We were using tactile sensing. Every time you pick an item, you get sensor feedback as to what works and what didn’t work.”
Jentoft and Tenzer also cofounded TakkTile LLC, which makes tactile sensors. Their patent-pending technology uses microelectromechanical systems (MEMS) to provide inexpensive gram-level sensing in a robust form factor.
Multimodal intralogistics solutions
Since their postdoc days, Jentoft and his cohorts have redesigned the gripper mechanism to make it more industrial. They’ve also stripped it down to make it more reproducible and affordable.
“It’s important to note that we’re a hardware-enabled software company,” Jentoft said. “You need the right hardware to do the grasping even if you’re building the brain. In the last five years, with 3-D printing and all of these off-the-shelf sensors that have been developed for the cell phone industry, it’s become much less expensive to build production-grade hardware. Hardware is always hard. But it’s getting easier.”
It’s the same, Jentoft said, for vision technology. “In the last few years, we’ve had good depth sensors out on the market. We use off-the-shelf sensors but the whole vision stack is internal. We’re able to use those 3-D images to figure out how to pick up items that we’ve never seen before. When I started grad school that was a $10,000 problem. When the Kinect came out,” referring to Microsoft’s depth sensor originally released as an add-on for the Xbox 360 video game system in 2010, “it became a $150 problem.”
Jentoft stressed the importance of finding a balance between precision and speed. Their goal is not to be perfect at the expense of cycle time. They call it the 3Rs – range, rate and reliability.
“With Amazon breathing down everyone’s neck, it’s not just about did you do the right thing? It’s did you do it quickly?” Jentoft said. “Can you do it scaled up? And can you do it with enormous labor shortages in the market? Stories of 20% absenteeism are very common. We were talking to a warehouse that has 300% annual turnover, also not unusual.”
RightHand Robotics is focused on providing their RightPick solution for integration into existing warehouse technologies and workflows, including automated storage and retrieval system (AS/RS) tending, sorter induction, autobagger induction, and kitting. They partner with other intralogistics and e-commerce systems providers to deliver the whole solution. Potential end users include e-commerce warehouses, retailers’ warehouses, and third-party logistics providers.
RightPick.AI software is robot agnostic. Although we usually see the RightHand Robotics gripper teamed up with a UR collaborative robot, the hardware-software solution can be used with other collaborative robots or traditional industrial robots. The gripping system is designed to grasp items 2 kg or less, which is typical of the types of products handled in these e-commerce and intralogistics applications.
Grasping our collaborative future
Smart robot grasping has become a multidisciplinary endeavor. Solutions are coming from all corners of the engineer’s toolbox. Mechatronics, soft robotics, sensor technology, intuitive software, and now AI and cloud robotics – all are having an impact. The future will be collaborative at every level.
Tanya M. Anandan is contributing editor for the Robotic Industries Association (RIA) and Robotics Online. RIA is a not-for-profit trade association dedicated to improving the regional, national, and global competitiveness of the North American manufacturing and service sectors through robotics and related automation. This article originally appeared on the RIA website. The RIA is a part of the Association for Advancing Automation (A3), a CFE Media content partner. Edited by Chris Vavra, production editor, Control Engineering, CFE Media, firstname.lastname@example.org.