How to read a pick rate: we audited eight vendor claims and none are comparable

Here are three real claims, read in August 2026 from three robot vendors' own home pages. One promises two to three times the productivity. One promises up to 600 containers an hour. One reports 533 pieces per person-hour. Now answer the question any buyer would ask: which of these is fastest?

You cannot answer it. Not because the work is hard, but because the question has no answer. The three numbers measure different things, over different periods, under conditions none of the three vendors published. Putting them side by side looks like a comparison. It is closer to comparing a distance, a speed and a price.

So we read every performance claim these vendors publish on their own live pages. Here is what we found, and what to ask instead.

The audit

We collected every performance claim we could read on four vendors' own sites, plus one vendor who publishes none. We then asked one question of each claim: does the vendor say how it was measured? Not in a whitepaper behind a form. On the page where the number appears.

Seven numbers. Not one says how it was measured. One of them, AutoStore's 63-second order fulfillment time, at least names the customer site it came from, which is more than the rest offer. But naming a site is provenance, not method. It still leaves out the order profile, the measurement window, and whether the clock ran through exceptions. A claim without conditions is not a small measurement problem. It is an unfalsifiable statement.

We should be precise about what we checked and what we did not. We read what these vendors publish openly. Some of them will have detailed measurement data they share under a mutual non-disclosure agreement once you are in a sales process, and if you are buying, you should ask for exactly that. Our point is narrower and still holds: the numbers doing the marketing work, the ones that reach a buyer before any conversation starts, carry no conditions at all.

What each number leaves out

Take them one at a time, because each hides something different.

"Increase your warehouse productivity by 2-3x." Productivity of what, against what baseline? A site picking badly by hand has more room to triple than a well-run one. The multiple describes the old process as much as the new robots. Worse, the better your operation already is, the smaller your gain, which is the opposite of how the claim reads.

"Pick from up to 600 containers per hour at each station." Three problems in nine words. "Up to" makes it a ceiling, not an expectation. "At each station" invites you to multiply, but only if every station can be fed at once, which depends on the robot fleet behind them. And a container is not a pick. If an order line needs three containers, your line rate is a third of the container rate.

"Picking productivity jumped from 149 to 533 pieces per person-hour." This one has the most detail and is still the most slippery, because of the words "per person". Pieces per person-hour can rise while the site gets slower, if you remove people faster than you lose throughput. It is a labour productivity number, not a throughput number. Those are different purchases.

"Maintain 99.9% uptime during peak seasons." Uptime of what, and measured how? If a fleet of 100 robots has one down all week, is that 99% or 100%? Both readings are defensible, which means the number is not.

The vendor who publishes nothing

One of the companies we checked, Berkshire Grey, publishes no performance figure on its landing page at all. Its pages talk about reliability and scale without a number attached.

This is not worse than the others, and it may be more honest. A vendor who declines to publish a number they cannot qualify is behaving reasonably. But it leaves a buyer with nothing to work from, and it means the loudest number in your shortlist is not the best system. It is the vendor most willing to publish an unconditioned figure.

That is the real risk here. Unconditioned claims do not just mislead individually. They sort the market the wrong way round, and they punish the vendors being careful.

What you can check the claims against, and what you cannot

In most industries you would resolve this by pointing at a standard. A tested load rating or a fuel economy figure means something because a published method says how to measure it, and everyone uses the same one.

Correction, added after publication. We first wrote that no standards body publishes anything here. That was too strong, and a reader following our own sourcing rules would have caught it. Here is the accurate version.

On the American side, we could find no ANSI or ASTM standard test method for picking rate. We enumerated the active standards of ASTM Committee F45, which covers robotics and autonomous systems. They address navigation, docking, obstacle detection, communication and terminology. None counts a pick. A cycle-time test method for robot grippers is a proposed work item, not a published standard [1].

Europe is different, and this is the useful part. FEM, the European materials handling federation, has published rules for storage and retrieval machines since 1981. FEM 9.221 defines throughput as the number of storage and retrieval operations per unit of time and gives a method for deriving it [2]. FEM 9.222 sets out how to run acceptance, performance and availability tests on an installation [3]. Both are free downloads.

Two honest limits on that. Those rules cover crane-based storage and retrieval machines, not a person or robot picking units from a tote, so they are not a picks-per-hour test method either. And FEM 9.222 does not tell you what number to hit. It tells you to agree one first: it says the availability of the installation should be specified, at the latest, before signing the contract [3].

That is still a lever worth having. You can ask a vendor to accept a FEM 9.222 style performance and availability test written into the contract. And the claim we can make without qualification stands: not one of the eight claims above cites any standard, method or test protocol, because we read all eight ourselves.

The six questions

Send these to every vendor on your shortlist, about every number in their deck. They are short, and none of them is hostile.

The replies are worth as much as the answers. A vendor who has measured their system properly will answer in a day, because they already know. A vendor who has not will send a case study instead, or offer a call. Treat that as an answer too.

What to do with the number once you have it

Put it in the contract, with its conditions attached. A rate in a sales deck costs the vendor nothing. The same rate in a pilot agreement, defined as picks per hour measured over a full shift on your own order profile, with the clock running through exceptions, is a commitment.

Write down the pass mark before the pilot starts. We have made the opposite mistake ourselves, and published the correction. In our own first evaluation we chose a looser bar than the benchmark's own. That alone turned a dataset where nothing clears the standard into a 48.1% headline. Nothing about the robots changed. Only the bar did. A number set after everyone has watched the system run will bend toward whoever is holding the pen.

And measure the thing you are actually buying. If you are buying throughput, do not accept a per-person figure. If you are buying labour savings, do not accept a peak-hour ceiling. The gap between those two is where most disappointing pilots live.

Sources

All vendor claims were read directly from the companies' live public pages in August 2026 and are quoted verbatim in Figure 1, with the page named for each. We deliberately excluded two further claims that we could not verify at first hand, one attributed to a named customer site and one to an academic simulation study, because we could not reach the original source to confirm the wording. If a vendor believes we have misread a claim, or publishes measurement conditions we could not find, we will correct this post and say what changed.

1. ASTM Committee F45, Robotics, Automation and Autonomous Systems, astm.org. Active standards cover navigation, docking, obstacle detection, communication and terminology. A gripper cycle-time test method exists only as a proposed work item under subcommittee F45.05, not as a published standard. Enumerated August 2026.
2. FEM 9.221, Performance Data of S/R-Machines, Reliability, Availability, First Edition 10.1981, fem-eur.com: the rule 'contains a standard method for determining the reliability and availability of S/R-machines', with 'throughput being defined as the number of storage and retrieval operations per unit of time'. Free download.
3. FEM 9.222, Rules for the acceptance and availability of installations with storage and retrieval machines and other equipment, 06.1989, fem-eur.com. Scope: 'gives recommendations for the determination of the availability and for the commissioning, hand-over and testing of installations with storage/retrieval machines, material-handling facilities and other machinery and their controls'. Clause 4.6.3: 'The availability of the installation should be specified, at the latest, before signing the contract with the customer.' Free download.