COLUMN – In this column, three lean and tech experts respond to the same  question shaping today’s tech/AI debate. This month, following OpenAI’s hacking incident, they discuss jidoka, human oversight, cybersecurity, and responsible technological experimentation.

Words: Eivind Reke, Sandrine Olivencia, and Fabrice Bernhard

THE QUESTION

Last month, OpenAI revealed that two of its secret new super models autonomously escaped a testing environment, exploited a security vulnerability, and hacked into AI startup Hugging Face without any human instruction. The incident is being called unprecedented. Toyota’s jidoka principle was built on the idea of giving machines the ability to detect problems and stop automatically, so that humans can intervene before defects spread. If jidoka is about building the capacity to halt the process when something goes wrong, what does this incident reveal about the absence of that principle in AI development and what would a jidoka-inspired approach to AI safety actually look like?

THE ANSWERS

The news that an OpenAI’s secret new super models “broke out of jail” is just another reminder that Silicon Valley’s “move fast and break things” capitalism does not care about externality costs. Companies once known for slogans, such as “don’t be evil” and “connect with friends and the world around you,” have shifted toward extravagant storytelling, aggressive algorithms, and mass lay-offs — all in the name of shareholder value. The business model has always been reliant on extreme levels of venture funding, but the latest AI craze seems to be completely removed from the fundamental cost structure of running a business. The lean one is Profit = Price – Cost, and not Price = Cost + Profit. Silicon Valley seems to operate on Price = whatever Capital we can raise + Costs. Of course, these kind of stories about AI going rogue are essential to keep the hype rolling.

Since ChatGPT surprised the world with a text-based interface that can talk back at us in a persuasive and confident manner, based on what might be one of the biggest intellectual property heists ever seen, the story has been that AI is transformative, that AI will become autonomous, that AI will replace humans, that AI will develop general knowledge and become smarter than us. I have no idea if any of this will actually happen; what I do know is that getting real measurable value from LLMs in manufacturing is incredibly hard work and that it will become a lot more expensive for customers if anyone besides NVIDIA and TSMC are to make anything resembling a profit on the technology. This doesn’t mean that Lean isn’t firmly rooted in capitalism, too. It is just that it is rooted in an altogether different financial ideology, one that takes into consideration more than just shareholders.

As lean CEOs — from Art Byrne to Akio Toyoda, Nicolas Chartier or Larry Culp — will testify, they are in the business of making money. However, they are also in the business of making money while contributing to the wider society. Wiremold famously had a very lucrative profit-sharing scheme that its employees benefited from. Toyota is deeply concerned about its future, but also of that of its supplier base, its employees, the environment and the Japanese automotive industry. AramisAuto’s Nicolas Chartier is building a no-frills online European car-sales system that promotes circular-economy thinking through reuse and refurbishment, based on Lean Thinking, while Larry Culp has restructured, stabilized and saved General Electric where others envisioned bankruptcy and closure.

For a company like Open AI, an incident like this is perfectly normal. It’s also encouraged and utilized for marketing purposes (“look how scary and independent our spooky, yet to be release, model is”). The lean path would look very different: a recent LEI article suggested that Toyota have been experimenting with AI for a long time. This is hardly surprising: Toyota has always been an early adopter of new technology, even if it doesn’t come across like this in public discourse (first mainframe computer installed to control the production line in the 1960s, digital engineering systems in 1980s, digital twins in the 1990s, and I could go on). Instead of pushing technology in a move-fast-and-break-things kind of way, lean practitioners should follow Toyota’s lead and experiment with new technologies (preferably in a small area, as suggested by Deming), learn how they work, what they can and can’t do, and in the process build the human capability that enables scaling them in a smart, sustainable way that supports value creation and makes life easier for customers and people at the gemba. This is how you create automation with a human touch.

Last month, OpenAI disclosed an extraordinary incident. During a cyber security evaluation, advanced AI models were given a goal: solve exploitation challenges. Production safeguards had been reduced, but the models were supposed to remain inside an isolated environment. Instead, they found and exploited a zero-day vulnerability in the infrastructure surrounding the sandbox, obtained internet access, escalated privileges, and eventually compromised Hugging Face’s production systems in pursuit of the benchmark objective.

From a lean perspective, what is most striking about this is not simply that the AI was powerful enough to escape. It is that the system allowed an abnormal condition to continue existing. This is precisely the problem jidoka was designed to address.

Jidoka is sometimes translated as “automation with a human touch,” but there is a deeper principle behind it: stopping at the first abnormality. Its origins lie in Sakichi Toyoda’s automatic loom. When a thread broke, the machine detected the abnormality and stopped itself rather than continuing to produce defective fabric. It didn’t simply raise an alert while continuing to run; instead, detection triggered the stop, making the problem visible so that a human could intervene.

That distinction matters enormously for AI. A jidoka-inspired AI system would first need standards that make “normal” explicit: what the AI is expected to do, which systems and tools it may access, what permissions it has, and what constitutes an abnormal condition. These standards could take different forms: hard technical boundaries, expected behaviors, or skills the model has learned for handling particular situations safely. Independent mechanisms would then monitor execution against those standards. If an agent unexpectedly attempted to leave its sandbox, access an unauthorized network, retrieve credentials or elevate its privileges, the system wouldn’t simply record the event for later investigation. It would stop. And, just as with Toyoda’s loom, the human would come in precisely because the machine has stopped.

But stopping is only half of what jidoka teaches us. The stop creates an opportunity for learning. Engineers investigate the gap between what was expected and what actually happened. Was the standard incomplete? Did the model lack a skill for achieving its objective safely? Did it encounter a situation nobody anticipated? Or did it expose a weakness in the surrounding system? That learning is then built back into the process: improve a guardrail, change an authorization rule, refine the standard, or teach the model a new skill. The next iteration should be more robust than the previous one.

This is why standards are fundamental: without an explicit understanding of what good looks like, there is no abnormality to detect, and without learning from each stop and improving the standard, an andon is merely an alarm.

The jidoka learning loop for AI, therefore, looks remarkably familiar: standard → detect the gap → stop → understand → learn → improve the standard.

The lesson is not that autonomous AI should never be given freedom. It is that the more autonomy we give a system, the stronger its built-in ability to detect abnormality and stop must become. The next frontier in AI is not simply building machines capable of doing more faster; it is building systems capable of knowing when not to continue.

The July 16th hacking incident OpenAI took responsibility for is a worrying but predictable development in AI's advance. How should we react to it, both in our organizations and as a society? Lean teaches us that this is a defect, and defects should be caught as early as possible. Certainly, before they intrude into someone else's infrastructure.

It's somewhat reassuring that the model involved had been specifically stripped of its security guardrails. Widely accessible models carry guardrails that would have flagged such an attempt and made it hard to carry through. But guardrails aren't foolproof, and bad actors will find ways to access unprotected models, so we need to think about the wider system.

This incident reveals two problems.

First, OpenAI's testing environment wasn't secure enough. In biology, scientists working on the most dangerous viruses enforce multiple layers of containment, so nothing escapes the lab. Here, OpenAI was testing its most dangerous models just one small door away from the Internet. Worse, the incident prompted Meta and Anthropic to review their own testing environments and discover that theirs had unrestricted Internet access. Like finding out they had left their lab door open all along.

Second, models with this kind of skill at finding vulnerabilities should be powerful allies for defenders. They can scale a security expert’s ability to catch issues across the critical systems running our world, fixing them before exploitation. AI vendors are working to empower trusted security partners, but the rollout is far too slow given the scale of the problem.

The deeper issue: AI vendors keep trying to solve the problems their AI creates with more AI. As with every technology before it, the new problems it creates will only be solved by humans empowered by the technology — what Toyota calls jidoka or "automation with a human touch".

It's humans who were responsible for the lousy testing environments and are best placed to understand why it happened and how to fix it. And it's through a large network of trusted humans that we can turn these powerful AI models into an amazing opportunity to address the root cause of all these cybersecurity incidents: finally fixing the millions of undetected vulnerabilities out there and raising the quality of our systems for good.

The future of cybersecurity is not about better AI models anymore, but about how we can use them to augment humans better, faster and at scale.

‍