OSARO SightWorks advances AI-powered robotic picking and packing

Warehouse automation increasingly requires robots to do more than identify and retrieve products. They must recognize unfamiliar objects, adapt to changing inventory, determine how to grasp different shapes and materials, and understand where and how items should be placed to complete an order.

OSARO is addressing these challenges with SightWorks, its AI-powered perception software designed to help robotic arms operate in high-variability fulfillment environments.

Deep learning eliminates traditional SKU onboarding

Built around deep learning and machine learning, the platform enables robots to recognize objects, identify grasping opportunities and execute complex pick-and-place tasks without extensive SKU onboarding or manual programming.

According to Derik Pridmore, co-founder and CEO of OSARO, SightWorks takes a different approach from traditional machine vision systems.

"The main difference is that it's deep learning first, so it's an AI-first implementation," Pridmore said. "Because we're deep learning first and because we have piece-picking geared models, we are already providing models that have been trained on millions of items."

Traditional machine vision systems often rely on predefined parameters, edge detection and shape recognition to identify products. Those approaches can struggle in warehouses handling thousands of SKUs with different packaging, dimensions and orientations.

Instead, SightWorks uses pretrained deep learning models to identify objects and potential grasping points. Its perception software generates confidence maps using RGB and 3D data, allowing robots to identify objects without relying on individual SKU databases or extensive product registration. Another differentiator, according to Pridmore, is the platform's ability to account for the robot's end effector - or gripping tool - when determining how to handle an object.

"If I was picking something up with a suction cup, I have to grab it from the side, right? I wouldn't be able to grab it from the rim,” he said. But if I'm using an actuated end effector, all of a sudden I can grab from the top.”

Factors such as suction-cup positioning, flexibility and air pressure can affect grasping performance. Deep learning helps SightWorks account for these variables and adapt its grasping strategies to different objects and equipment configurations.

The platform also incorporates an automated learning process. Each deployment includes a GPU-equipped computer, camera and pretrained perception models. Images are processed locally, with results communicated through APIs to robot controllers and programmable logic controllers (PLCs).

When customers enable cloud connectivity, operational data and pick results can periodically be sent to OSARO for model retraining. OSARO said that updated models are delivered automatically when the company's system determines that retraining will improve performance.

"Sometimes you get the data, you retrain, you see the model would actually be worse, and so you leave the model alone," Pridmore said. "The user doesn't have to do anything."

Connecting robotic picking with packing

OSARO's approach recognizes that picking and packing are closely connected automation challenges.

"There's no such thing as just an independent picking problem because you're always doing something with the item," Pridmore said.

Complexity increases when applications require precise placement, specific orientations, or coordinated packing strategies.

For example, OSARO uses SightWorks in piece-picking applications that feed vertical automated bagging equipment. Robots must retrieve individual products and place them within relatively small areas while accounting for orientation.

Kitting introduces additional requirements. OSARO has deployed its technology in applications involving products such as dog toys for BarkBox and custom iPhone cases for Case History. These workflows require robots to select multiple items and place them into the correct kits or containers, sometimes with specific orientations and space constraints.

"That actually forced us to develop very fast inference times for the object modeling piece," Pridmore explained. Robots must quickly estimate an item's dimensions and shape to determine how it should fit within its destination.

When multiple cameras are used, SightWorks can generate 3D models of objects as the robot moves. This helps the system estimate the dimensions and geometry of items, including reflective or flexible products that can change shape during manipulation.

Foundation models point to the next advancements

SightWorks has continued to improve as OSARO incorporates operational data from its growing deployment base. Pridmore said that newer models are increasingly accurate and capable of handling wider varieties of SKUs, which reduces the amount of learning required for new installations.

Looking ahead, Pridmore expects smaller foundation models to expand robotic perception beyond object recognition and measurement toward basic robot-control capabilities.

"We will see smaller foundation models," he said. "You'll be able to get not just measurements of the items and recognition of the items, but probably in the next couple of years be able to do some basic control from deep learning models."

He also anticipates improvements in scene understanding, giving robots the ability to recognize more conditions and objects that could affect an application, expanding capabilities such as error detection and safety monitoring.

OSARO has already experimented with larger foundation models internally, but Pridmore said their computational requirements must become more practical before those capabilities can be commercially offered to integrators.

Ultimately, the goal is to make increasingly sophisticated robotic applications easier to deploy and operate.

"It's a relatively easy-to-use product,” he said. “I think that especially the automated learning piece should hopefully give people some confidence that they can tackle new applications.”