The function call returns before the world has changed
A control unit changes its state in microseconds. The actuator it controls takes several hundred milliseconds. There are five orders of magnitude between these two figures, and this gap accounts for a significant portion of the errors that are only detected during integration and never during code review.
Nevertheless, it seems like every debate about faster product development revolves around software: software-defined vehicles, software platforms, feature teams, two-week sprints. Hardware does come up in these discussions, but often as a „solved problem“ or a legacy issue. This view is incorrect and explains why companies adopt agile methods and then still take just as long to develop a product as they did before.
The Economics of Change
Software development isn't fast because teams work in an agile way. They're fast because the economics of change allow for it, and agile work makes that possible. Software has three key characteristics:
- Marginal cost close to zero. A copy costs nothing. The build takes just a few minutes, and the infrastructure for over-the-air delivery is already in place.
- Reversibility. A rollback restores the system to its previous state. A failed attempt costs computing time, but no resources.
- Feedback in a matter of seconds to hours. Test suite, canary release, field telemetry.
That's why small batches are worthwhile for software, because merging them is inexpensive. Experiments are also extremely valuable. With Continuous Delivery the costs of change are close to zero.
In hardware, none of these three characteristics apply. An injection-molding mold has a lead time of eight to sixteen weeks and costs a five-figure sum. A geometry change after the mold has been approved requires a change request with an approval process, and the material that has already been used remains in place. Even when using rapid manufacturing, costs are incurred, and feedback comes from a test setup that someone must first build.
Anyone who introduces two-week sprints in this world isn't shortening the learning cycle—they're just breaking up the wait.
The copying error
This is exactly where Product Velocity which applies the principles behind software agility to hardware-intensive products. But that requires adaptation.
The difference is already evident in the tools of the trade. Software development has been using nonlinear version control for twenty years: branch, merge, pull request. Each developer tries out a different version without blocking the others. In hardware development, this experience is often lacking, as Martin Häuer points out in the Interview on Open Source Hardware describes. Without efficient branching and merging, parallel processing remains expensive, and without parallel processing, the critical path remains long.
A second difference carries more weight because it cannot be eliminated through organizational measures. Software is not subject to inertia. Hardware is.
Settling time
Control engineering has long had terms for this behavior. Between the command and the resulting state lies a Settling time, often preceded by a Dead time, in which absolutely nothing happens. Depending on the domain, the two differ by orders of magnitude.
From the perspective of the overall system, a state change in software occurs instantaneously. The write operation on a variable is complete before anything physical has had a chance to react. Electrical and electronic systems take longer—usually milliseconds—and during this time they oscillate or bounce. Mechanical, hydraulic, and pneumatic systems take tenths of a second to seconds, with a long decay period, until the final value is reached.
The conclusion that follows sounds trite, yet it is constantly overlooked:
The function call returns before the world has changed.
As long as a team is testing software against software, this never becomes apparent. The mock responds immediately. It is only when the system is actually set up that it becomes clear there is a time lag between setting a state and the state itself—a lag that no one has modeled.
The Parking Brake
Ilya Lebedev described an example on LinkedIn that simplified the case to two function calls. A vehicle has at least two braking systems: the hydraulic service brake and the mechanical Parking brake. An obvious design exposes two functions, ServBrk and ParkBrk, and handles them independently of each other. This results in three permissible states: both released and the vehicle is rolling; service brake engaged; parking brake engaged.
The transition from the parking brake to the service brake is a two-step process. Release the parking brake; apply the service brake. Logically correct, and seamless at the state level.
In reality, there is a window between the two commands during which the vehicle rolls freely. The parking brake actuator takes some time to engage, as does the build-up of pressure in the hydraulic system. On a slope, just a few hundred milliseconds are enough for the vehicle to start moving. The state machine does not recognize this state because it only recognizes target states.
This isn't a programming error. The code does exactly what the design specifies. The error lies in the model. It has three states, but the real system has more.
Abstraction That Knows Time
The usual reaction is: more testing. But that doesn't help much, because the problem lies in the interface, not in the coverage.
A Hardware Abstraction Layer is the right answer to the coupling problem. It hides physical details and makes clean black-box modeling possible in the first place. It is precisely this strength that becomes a trap as soon as the abstraction hides the dynamics as well. An interface that only a release() implies that solving a problem is an event. It is a process.
A functional interface must make three things visible:
- Target state and actual state are separate. The command is not the result. Anyone who combines the two in a single variable has already built in an error.
- Transitions as distinct states. „Solving“ and „attaching“ belong in the model, along with the question of which neighboring states are allowed and which are prohibited.
- Time limits as part of the contract. How long can a transition take, and what happens if it exceeds that time? If there is no timeout specified in the contract, the implementation determines this, and it varies from case to case.
This does not eliminate the free-wheel phase. It becomes visible and can therefore be safeguarded. Releasing the parking brake then depends on the measured brake pressure rather than on the command issued.
Move decisions to where it's inexpensive to make changes
This raises the real question. If the software’s practices aren’t transferable, then what?
In software, the answer is: make changes inexpensive. In hardware, that’s not possible. There, the answer is: shift the decision to the stage where making changes is still inexpensive. That’s the model, the simulation, the prototype, and the test bench. Every decision made in the model rather than in tooling saves exactly the amount by which tooling is more expensive.
However, this only works if the model includes the dynamics. A simulation in which actuators switch instantly will not detect the parking brake error. It simply validates the design. Front-loading without time behavior in the model merely shifts the problem downstream, where it becomes more expensive.
That is why a software-only platform falls short. McKinsey recommends To provide vehicle manufacturers with a unified software platform featuring standardized interfaces in order to reduce complexity and integration costs. A vehicle remains a combination of mechanical, electrical, and software components. This Faulty reasoning regarding software-defined vehicles I have described this in more detail elsewhere. What is needed is a system platform that covers the problem level as well as the logical and physical solution levels.
In his Keynote on Cybertronian Systems It adds a point that often gets overlooked in discussions about tools. No single system contains all product information. Version management, change management, and requirements tracking work differently in hardware and software, and no tool that claims to do it all can change that. What helps is a consistent data model that spans multiple systems.
Eliminate Waiting
The second answer concerns the organization, not the technology.
Joe Justice has demonstrated with WikiSpeed, ...that a team can design, build, and test a road-legal vehicle within a week. In the Interview he sums up the essence in one sentence:
„Rapid hardware development scales by eliminating wait times.“
Joe Justice, WikiSpeed In an interview with Michael Jastram
It’s not physics that slows things down, but the queue ahead of it. A team is waiting for approval, for a testing window, for a response from a supplier, or for a decision from another department. These delays add up to many times the actual processing time. They aren't listed on the schedule because no single trade caused them.
The solution lies in architecture. Stable interfaces decouple teams, and decoupled teams spend less time waiting. Joe Justice draws a parallel here with Conway’s Law. If you build modules that force constant coordination, you end up with an organization that is constantly coordinating. In the second part of the interview He describes a 30-day experiment designed to test whether a company is actually becoming more agile or has merely adopted agile terminology.
Added to this is the sequence within the design itself. „Hardware first“ creates complexity because the two domains operate in isolation, and the software ends up ironing out issues that could have been avoided in the design. Parallel Development of Hardware and Software It is not a question of tools, but a question of responsibility. Someone must take responsibility for the integration between the disciplines.
Conclusion
Attention is focused on the software because that's where visible progress is made. The product's rhythm, however, originates elsewhere. It arises from tool setups, inspection windows, approval loops, and actuators that take time to operate.
This doesn’t mean abandoning agile methods, but rather changing the order in which they’re applied. First, ask what a change at this stage will cost and how long it will take to get feedback. Then choose the practice that aligns with these economic considerations. A two-week sprint is no solution to a sixteen-week tool lead time. Shorter paths to the model and fewer bottlenecks, on the other hand, certainly are.
A test of your own architecture takes ten minutes. Take the specification for any actor interface and look for three pieces of information: measured current state, transition states, and time constraint. If any one of these is missing, the free-fall phase is already embedded in the design. It’s just waiting for the right slope.
