Layer 4 — Non-functional properties of the platform

Hai publishes four hard numbers about its own software. An architect is expected to know them, and — more importantly — to have a view on whether they are impressive for this workload. Being able to say "10,000 req/s is generous here, and here's why" is exactly the judgement the title implies.

High availability, failover and load balancing

What it is. The system's ability to keep running when a component fails.

Why it matters here. An automated warehouse has no manual fallback. If the software stops, the building stops — you cannot revert to paper picking when the inventory only exists in the system and the totes are 12 m up a rack.

Fundamentals.

• Availability targets translate to real downtime budgets: 99.9% is 8.8 h/year; 99.99% is 53 min/year. For a site running two shifts, the question is not the annual figure but how long a single incident lasts during a shift.
• RTO vs. RPO. Recovery Time Objective (how fast back up) and Recovery Point Objective (how much state can be lost). For a fleet controller, RPO must be effectively zero — losing task state means robots holding totes with no record of why.
• Active-passive vs. active-active. Active-passive with automatic failover is the common choice for control systems: simpler consistency, brief interruption. Active-active needs a consistency story you must be able to defend.
• Graceful degradation is the property that matters most in physical systems: when the scheduler dies, robots should stop safely and resume, not drop totes or forget where they are. Design the safe state first.
• Warm restart with physical reconciliation: after a failure the software must rediscover reality — which robot holds which tote, which locations are occupied.

How Hai applies it.

"It has high availability with automatic switching in case of failure, as well as load balancing and data disaster prevention features." — /solutions/haiq-software

That is active-passive with automatic failover, plus load balancing and backup. No availability figure, RTO, or RPO is published anywhere in the corpus.

Tradeoffs and pitfalls.

• Failover of the software does not recover the physical state. The hard part is reconciliation, and nothing in the corpus describes it — a strong question.
• Load balancing a stateful fleet controller is not the same problem as load balancing web servers; ask what is actually being balanced.

Key terms. RTO · RPO · active-passive · automatic failover · graceful degradation · safe state · warm restart · reconciliation.

External request throughput

What it is. The rate of inbound requests the platform boundary can absorb.

Why it matters here. It tells you the integration boundary is not the bottleneck — and it lets you sanity-check the whole design.

Fundamentals.

• Do the arithmetic before being impressed. A large site runs perhaps 2,000 cases/hour — under 1 request/second of genuine order traffic. Even with status polling, confirmations and telemetry multiplying that by a hundred, you are at ~100 req/s.
• So 10,000 req/s is roughly two orders of magnitude of headroom for a single large site. The number makes sense if the platform is multi-tenant, hosts many sites, or counts internal robot telemetry as "external requests".
• The honest architect's reading: this is a capacity claim, not a latency claim. For a control system, p99 latency and jitter matter far more than peak throughput — a scheduler that occasionally takes 2 seconds to answer stalls robots regardless of how many requests per second it could theoretically serve.

How Hai applies it.

"Capable of processing up to 10,000 external requests per second." — /solutions/haiq-software

Tradeoffs and pitfalls. No latency distribution, no test conditions and no definition of "external request" are published. Treat as marketing capacity until qualified — the question to put to the vendor is "is that sustained or peak, and what's the p99 latency at that rate?"

Key terms. throughput vs. latency · p99 · jitter · headroom · sustained vs. peak · control-loop latency.

Fleet scale ceiling

What it is. How many robots one scheduling instance can drive.

Why it matters here. It bounds the largest single site you can propose without splitting into multiple scheduling domains — a genuine architectural decision.

Fundamentals.

• Scaling limits in fleet control come from coordination, not compute. Pairwise conflict checking is O(n²) in the worst case; reservation-table contention grows faster than linearly with density.
• The real limit is usually floor density, not scheduler capacity — see congestion collapse in algorithms. You will hit traffic limits long before you hit 1,000 robots in one building.
• Scheduling domains: large sites are commonly split into zones with their own schedulers plus a coordinator, trading global optimality for scalability.

How Hai applies it.

"Capable of scheduling over 1000 robots." — /solutions/haiq-software

"HaiPick System 3 has been deployed in over 100 projects worldwide with 10,000+ robots." — /solutions/haipick-system-3

Keep the two straight: 1,000 is a per-scheduler claim; 10,000+ is the global installed base across 100+ projects — an average near 100 robots per project.

The largest published single site is Bosideng's Smart Distribution Center: 69 ACRs plus 219 AMRs, 288 robots on HaiPick System 3 (from Hai's video library, not the case pages). Corrected: an earlier version said 87, which was only the largest in the case-study pages. So the scheduler ceiling sits roughly 3.5× above the largest observed site — comfortable headroom, not an order of magnitude.

Tradeoffs and pitfalls. Quoting "1,000 robots" as if it were a site size is the kind of error that costs credibility. Real sites run from single digits to a few hundred robots, and the scheduler ceiling is not the binding constraint — floor congestion is.

Key terms. scheduling domain · coordination overhead · congestion collapse · installed base vs. per-site scale.

Robot communication security

What it is. Protecting the link between the fleet controller and the robots.

Why it matters here. These messages command physical motion in a space containing people. Integrity matters more than confidentiality.

Fundamentals.

• Warehouse robots run on Wi-Fi, which means the threat model includes interference, roaming dropouts and rogue APs — availability threats before malicious ones.
• Integrity and authenticity over confidentiality: an attacker who can forge a motion command is far worse than one who can read a tote ID.
• Fail-safe on link loss is the essential property: a robot that loses contact must come to a controlled stop, not continue on its last command.
• OT security norms — network segmentation, separate VLANs for robot traffic, no direct internet exposure — apply here and are worth naming.

How Hai applies it.

"The data communication between robots uses encrypted components." — /solutions/haiq-software

"The HaiPick system consists of Wi-Fi and other wireless networks." — /robots/haipick-a3

"The robots connect to the HaiQ intelligent management platform, which monitors their working status and health condition." — /robots/haipick-a3

The Robotics Engineering Manager JD lists network setup among deployment responsibilities — Wi-Fi coverage design is genuinely part of commissioning an ACR site, and poor coverage in aisles is a classic go-live problem.

Tradeoffs and pitfalls. "Encrypted components" is not a protocol, cipher suite or key-management story. Nothing else is published; do not extrapolate.

Key terms. OT security · integrity vs. confidentiality · fail-safe · link loss · network segmentation · Wi-Fi roaming · coverage survey.