A free VDA 5050 conformance test for your AMR fleet.

Background

What VDA 5050 is, and why it exists

VDA 5050 is an open, vendor-neutral interface between a fleet manager (the software that decides what happens) and an AGV or AMR (the vehicle that does it). It was published by the German automotive industry association (VDA) together with the mechanical engineering association (VDMA), and it exists to solve one specific commercial problem: before it, buying a second brand of robot meant buying a second fleet manager, a second operator console, a second integration project and a second set of people who understand it.

That is not a technical limitation, it is a business model. Every AMR vendor had its own proprietary API, so the fleet-management layer — the part that actually knows your building, your traffic rules and your workflows — was locked to the vehicle vendor who happened to win the first order. Adding a second vendor to the same floor meant two systems that could not see each other's robots, which in practice means two systems that will eventually drive two robots into the same aisle.

VDA 5050 defines that boundary once, so any conforming vehicle can be driven by any conforming fleet manager. Concretely it specifies:

• MQTT as the transport, with a five-segment topic scheme: interfaceName/majorVersion/manufacturer/serialNumber/topic — for example uagv/v2/acme/AGV_01/state.
• A small set of topics with fixed meanings. The vehicle publishes connection (am I alive), state (where I am, what I am doing, what is wrong), factsheet (my footprint, speeds, and which actions I support) and optionally visualization (a cheap high-rate pose feed). The fleet manager publishes order (drive this graph of nodes and edges, do these actions) and instantActions (do this right now, regardless of the current order).
• A JSON payload shape for each, with a common header carrying headerId, timestamp, version, manufacturer and serialNumber.
• A state machine for orders: orders are identified by an orderId and revised by a monotonically increasing orderUpdateId, so a fleet manager can extend or amend a route that a vehicle is already driving without ambiguity about which version wins.
• A lifecycle for actions: every action moves through WAITING, INITIALIZING, RUNNING, PAUSED to a terminal FINISHED or FAILED, reported back in the vehicle's actionStates.

Yantrika is on the fleet-manager side of that boundary: it speaks VDA 5050 v2.1 so it can drive a mixed-vendor floor. This tester exists because that promise is only as good as the vehicles' implementations of the standard — and those vary a great deal.

The actual problem

What conformance actually means in practice

“Supports VDA 5050” is a claim on a datasheet. In practice it can mean anything from a complete, correct implementation to “we publish a JSON blob on an MQTT topic with roughly the right name.” The gap between those two things is where integration projects go to die: three weeks into commissioning, on a customer site, with a forklift-sized deadline.

The reason is structural. VDA 5050 is a protocol specification, not a certification programme with a mandatory test suite behind it. There is no conformance badge you must earn before shipping. So an implementation is typically validated against exactly one thing: the fleet manager it was first sold with. Everything both sides happen to agree on — including things neither side reads from the spec — looks like it works. Everything the pair never exercised is untested, and stays untested until somebody plugs in a different fleet manager.

Conformance testing, then, is not about a certificate. It is about answering three questions before the robots are on your floor:

• Does the vehicle publish what the specification requires, in the shape it requires? Not “does the fleet manager we tested with cope with it” — a consumer written to the spec will index a required field directly and raise on its absence. An empty array is a valid value; a missing key is not.
• Does the vehicle respond correctly to the commands the specification says it must answer? Publishing telemetry is the easy half. The interesting half is what happens when a fleet manager cancels an order mid-drive, asks for a factsheet, or replays a stale order update.
• Does it behave correctly in the failure cases? Duplicate deliveries, a fleet-manager restart, an unsupported action, a vehicle that loses power mid-order. These are precisely the paths a happy-path integration never touches, and precisely the ones that cause incidents.

A conformance test is not a substitute for a safety assessment, a site acceptance test, or reading the specification. It is a fast, repeatable, mechanical answer to “is this vehicle going to behave the way a standards-compliant fleet manager assumes it will?” — which is a question worth answering during vendor evaluation, not during commissioning.

Failure modes

The ways real implementations actually fail

Every item below is one of the tester's own checks, described by what goes wrong downstream when a vehicle gets it wrong. None of them are exotic. They are the things that pass a demo and fail a Tuesday.

• A stale order update rewinds the robot

  An orderUpdateId lower than or equal to the one already in progress must be rejected. MQTT redelivery and a fleet-manager restart both replay old orders, so this happens on ordinary days, not just bad ones.

  A vehicle that accepts them re-drives a route it already finished. Checked by order.reject_stale (critical).

• An action disappears instead of finishing

  An action that stops being reported must have reached FINISHED or FAILED first. Implementations commonly drop an action from actionStates the moment it completes.

  The consequence is that a fleet manager which lost a single QoS 0 state message waits forever for work that is already done — a hang with no error anywhere. Checked by actions.terminal_reported (major) and actions.no_regression (minor).

• cancelOrder is not really implemented

  This is the fleet manager's only clean abort path. Done correctly the vehicle stops, drops the remaining nodes and edges, fails every outstanding order action, and only then reports the cancelOrder action FINISHED.

  Partial implementations acknowledge the action and keep driving, or stop but never report the terminal status. Either way an operator pressing “abort” gets no confirmation that anything happened. Checked by instant.cancelOrder (critical).

• A required field is omitted rather than sent empty

  errors, nodeStates, edgeStates and actionStates are required on every state message. Omitting a key is not the same as sending an empty array, and a consumer written to the specification will fail on the missing key.

  This is the single most common way a vehicle that “works fine with our own software” breaks a different fleet manager on the first message. Checked by state.required_fields (critical).

• The topic says one robot, the payload says another

  manufacturer and serialNumber in the payload must equal the corresponding topic segments. When they are built from two different sources they drift — a serial number formatted differently, a manufacturer string with different capitalisation.

  A fleet manager subscribing per vehicle then attributes one robot's telemetry to another, which is the kind of bug that looks like a physics problem for a week. Checked by state.identity (critical) and connection.identity (major).

• Retention is backwards

  The connection topic must be retained; state must not be. It is easy to get exactly wrong, and both directions hurt.

  Without a retained connection message, a fleet manager that restarts sees no vehicles at all until each one happens to reconnect. With a retained state message, every reconnecting consumer is handed a stale pose — a robot that appears to have teleported, or frozen. Checked by connection.retained and state.not_retained.

• No last will, so a dead robot stays ONLINE

  The vehicle should register an MQTT last-will publishing CONNECTIONBROKEN on its own connection topic before connecting. Without it, a crashed or unplugged vehicle stays ONLINE on the wire until the fleet manager's own staleness timer fires.

  That delay is exactly when a traffic manager is still routing other robots around a vehicle it believes is alive and about to move. Checked by connection.lastwill (minor — see the note on skips below, because a last will is registered inside the broker and cannot be observed by another client).

• headerId is not per topic, or resets

  One counter per topic, incremented on every publish. Sharing a counter across topics, resetting it on reconnect, or reusing an id makes message-loss detection downstream impossible — which matters because state is normally published at QoS 0 and loss is expected.

  Checked by state.header_id_monotonic (major).

• Timestamps that are almost ISO-8601

  The specification wants UTC with a literal trailing Z. Local time, a naive timestamp with no zone, a +00:00 offset, or an epoch number all break consumers that parse strictly — and all four occur in the wild.

  Checked by state.timestamp_format (major), with state.timestamp_monotonic catching clocks that step backwards.

• An unsupported action is ignored, or reported FINISHED

  An instantAction the vehicle does not implement must be reported FAILED, with a resultDescription naming the type. Silently dropping it leaves master control waiting on an action that will never complete. Reporting FINISHED is worse: it claims work that never happened.

  Checked by instant.unknown_rejected (major).

• The version segment is not v2

  The second topic segment must be the literal v followed by the major version. Not 2, not 2.1, not v2.1.0. That segment is how a fleet manager subscribes to one protocol generation at a time.

  Get it wrong and a compliant fleet manager's subscriptions simply never match — the vehicle looks completely absent, with no error on either side. Checked by protocol.major_version_segment (major) and protocol.topic_scheme (critical).

• No factsheet

  The factsheet is how a fleet manager learns your vehicle's footprint, speed limits, supported actions and protocol limits without a PDF and a phone call. It should be published retained at startup, at QoS 1, and also on factsheetRequest.

  It is the most commonly skipped topic in the entire specification, because nothing visibly breaks without it — until commissioning, when every parameter has to be transcribed by hand. Checked by factsheet.published plus five structural checks on its contents.

The rule set

What the 52 checks cover

This table is the tool's own catalogue — the same data yantra-conform checks --json prints, which needs no broker and no network. Severity is a weight in the score: critical 10, major 5, minor 2, info 0. Section references are to VDA 5050 v2.1.

Two of these normally report not applicable rather than passing, and the report says so explicitly rather than quietly counting them in your favour: connection.lastwill, because a last will is registered inside the broker and is invisible to another MQTT client (verify it by pulling the vehicle's network cable and watching its connection topic), and visualization.wellformed, because the visualization topic is optional.

Usage

Running it

Install

Python 3.10 or newer, and nothing else. The [mqtt] extra pulls in paho-mqtt, which is needed only to talk to a broker — the rule set, replaying a capture and rendering a report are pure standard library.

$ git clone https://github.com/yantrika-ai/yantrika.git
$ cd yantrika
$ pip install -e "tools/conformance[mqtt]"

$ yantra-conform --version
$ yantra-conform checks          # all 52 rules, no broker needed

No entry point on your PATH? python3 -m yantraconform works identically everywhere below.

The one command

$ yantra-conform run --broker mqtt://YOUR-BROKER:1883 --html report.html

Open report.html in any browser. That is the whole tool. With credentials or TLS, and with a non-default topic prefix:

$ yantra-conform run \
    --broker mqtts://fleet.example.com:8883 \
    --username fleetops --password '...' \
    --interface myfleet --major v2 \
    --html report.html

Is it safe against a live fleet?

By default no, and the tool says so plainly, because it is an active tester. It publishes real order and instantActions messages: a cancelOrder, a stateRequest, a factsheetRequest, an initPosition, and four orders. It is deliberately conservative — it only ever dispatches a vehicle to a node that vehicle itself reported, and initPosition re-asserts the pose the vehicle already believes it has — but it will interrupt work in progress.

Test a vehicle on a test stand, scope the run to one serial, or go passive:

# publishes NOTHING; grades everything observable from the wire alone
$ yantra-conform run --broker mqtt://localhost:1883 --passive --html report.html

# or scope an active run to a single vehicle
$ yantra-conform run --broker mqtt://localhost:1883 \
    --manufacturer acme --serial AGV_01 --html report.html

--passive grades the state topic, headers, timestamps, identity, retention, topic hygiene and the factsheet, and reports the command-surface checks as not applicable rather than passing them on no evidence. It is the right mode for a first look at a fleet you do not own.

As a CI gate

--fail-under sets a score floor. Exit code 1 means the score was below it, 2 means the run could not be performed at all (no broker, no vehicles, bad arguments), 0 means it passed.

$ yantra-conform run --broker mqtt://localhost:1883 \
    --json report.json --html report.html \
    --fail-under 90 --quiet

# regression gate on a recorded capture, with no broker in CI at all
$ yantra-conform replay capture.json --fail-under 90 --quiet

Output

What the report actually gives you

One self-contained HTML file you own, plus optional JSON. Every finding carries four things, because a red label on its own is not actionable:

How the score works

Severity is a weight: critical 10, major 5, minor 2, info 0. Status is a multiplier: pass 1.0, warn 0.5, fail 0.0. Checks whose evidence was never observed are skip — excluded from both sides of the ratio rather than counted against you.

score = 100 × Σ(weight × credit) / Σ(weight)

Two caps then apply on top of the letter grade, because a weighted average alone lets a vehicle with one fatal defect still look like a B:

• any failed critical check caps the grade at F;
• any failed major check caps the grade at B.

The report always names which checks capped the grade, so nobody has to reverse-engineer why an 89 came out as a B.

“n/a” is not a pass

A skip means the tester never saw the evidence a check needs, and it always says why. A run full of skips is a run that did not exercise much — the report puts its warnings at the top for exactly that reason. Besides the two structural skips noted above, the order.* checks skip when another master control is dispatching the same vehicle: if a fleet manager hands the vehicle a new order while the tester's probe is in flight, the order being updated is no longer the order in progress and the result is genuinely unevaluable. The tester says that rather than guessing. To grade order handling properly, test a vehicle nothing else is driving.

One defect, one finding

A single defect should cost a vehicle once, not fifteen times. When the evidence a whole family of checks depends on is entirely absent, one check fails for the absence and the rest report skip: a vehicle that never publishes state fails state.published and skips the other state checks, rather than scoring a misleading zero on eighteen things that were never observed.

For everyone who cannot expose a broker

Hand over a capture instead of your broker

This is the path for the common case, not the exotic one. A vendor's broker sits inside a customer's plant network. An integrator's sits behind a VPN and a change-control ticket. “Give the tester access to production MQTT” is frequently not a request anyone can approve, and it should not have to be.

So the tool separates collecting evidence, which needs the broker and takes about a minute, from grading it, which needs nothing at all.

On your network, record the session:

$ yantra-conform run --broker mqtt://localhost:1883 \
    --capture capture.json --html report.html

Read capture.json before you send it. It is plain, indented JSON. It contains exactly the MQTT messages that crossed the wire and the probes the tester sent — nothing else. No broker hostname or credentials, no environment, no files from your machine. Every byte in it came off your own network, and you can verify that with a text editor:

{
  "kind": "yantra-conform-capture",
  "messages": [
    {"t": 0.41, "topic": "uagv/v2/acme/AGV_01/state",
     "payload": "{\"headerId\": 12, ...}", "qos": 0, "retain": false}
  ]
}

Anywhere else — a laptop with no network, your vendor's desk, a CI job — grade it offline, with no broker and without paho-mqtt installed at all:

$ yantra-conform replay capture.json --html report.html

The replayed report is identical to the live one, check for check, string for string, including the order and instantAction findings. It is a re-grade of the same evidence, not a summary of it. Which means you can also re-grade an old capture against a newer rule set, to see what a stricter version of the tool would have said about a vehicle you accepted last year.

Practically, this gives three workflows that do not require anyone to trust anyone:

• Buying robots. Ask the vendor for a capture from their test vehicle. Grade it yourself. No site visit, no broker access, no shared credentials.
• Selling robots. Ship a capture and its report with the vehicle, the way you would ship a calibration certificate. It is checkable rather than assertable.
• Arguing about a defect. A capture plus a check id plus a clause reference is a bug report that cannot be argued with in general terms.

Honest limitations

Why there is no “test my fleet” button on this page

Because a browser cannot open a raw MQTT/TCP connection. That is not a gap in this project; it is how browsers work. Nothing in a web page can talk to your broker unless you first stand up a websocket listener on it and hand a website your credentials — which is a strictly worse trade than running a command on your own machine.

So the split is drawn honestly:

• Testing your fleet is a local command. It stays that way. yantra-conform runs on your machine, on your network, against your broker. Nothing about your fleet leaves your network unless you choose to send someone the report.
• Everything that is not testing your fleet works fine on the web — browsing the rule set (the table above), reading a sample report, and, because a capture is just a JSON file, grading a capture that somebody hands you.

If you came here from the footer of a generated report: that report was produced by this tool, on somebody's own machine, from their own fleet's traffic. Nothing was uploaded anywhere to make it.

FAQ

Questions people actually ask

Is it really free?

Yes. MIT-licensed open-source software that ships inside the Yantrika repository. No account, no trial, no quota, no watermark on the report, no sales call. You can read every rule it applies in tools/conformance/yantraconform/checks.py.

Can I test my robots from a browser?

No, and no honest tool can — see above. A browser cannot open a raw MQTT connection.

Is it safe to run against a production fleet?

Not by default. It is an active tester and it will interrupt work in progress. Use a test stand, scope it to one vehicle, or use --passive, which publishes nothing at all.

What if I cannot give an outside party broker access?

Record a capture and hand that over instead. Full explanation above — the capture is human-readable JSON containing only the MQTT traffic, and grading it needs no broker.

Which version of VDA 5050 does it grade?

Version 2.1. Each check names the clause it comes from. protocol.supported_version reports (as info) whether the protocol version a vehicle announces is within this tester's scope, so you are never silently graded against the wrong specification.

Does it work with my broker / my topic prefix?

Any MQTT broker, with or without TLS and credentials (--broker mqtts://..., --username, --password). The default topic prefix is uagv/v2; override with --interface and --major.

Do I have to use Yantrika to use this?

No. The tester is a standalone command-line tool that talks to your broker. It does not need Yantrika running, does not need a database, and does not care what fleet manager you use — including none at all.

How is the tester itself tested?

It grades a purpose-built non-compliant robot and this repository's real simulator. Every check in the catalogue has a test that makes it fail on purpose, and a meta-test fails the build if a check is ever added without one — a conformance tester that cannot be shown failing is worthless. That is 188 tests for the tester alone, on top of the platform's own suites.

I think a check is wrong.

Then it might be. Where this tool and the specification disagree, the specification wins. Open an issue with the clause and the payload, and the rule gets fixed — here is how.