By Ryo Kaneko, Director of Innovations, NEC X
Over the past six months, much of the AI conversation has focused on larger models, better benchmarks, and increasingly capable agents. But after watching how NEC X’s startups are actually deploying AI across healthcare, enterprise software, cybersecurity, and operations, I believe the next major challenge is no longer just generation. It is decision architecture.
Today’s large language models are fundamentally optimized to generate plausible responses from the context they are given. They are remarkably good at synthesizing information, drafting content, and producing conversationally natural outputs. However, most enterprise and safety-critical workflows require something very different.
They require systems that know:
- what information is missing,
- what questions must be asked,
- which uncertainties are dangerous,
- when confidence is insufficient,
- and when a human must take over.
This distinction is becoming increasingly important.
The Real Risk Is Not Hallucination
In many domains, the biggest risk is not that AI says something absurd. The bigger risk is omission.
A medical AI that fails to ask about chest pain duration or suicidal intent. A cybersecurity system that misses indicators of lateral movement. A financial workflow that never verifies the source of funds. In these cases, the system may sound reasonable while silently missing the very information required to make a safe decision.
Traditional LLMs are not naturally designed for this. They are optimized for conversational continuity, not structured uncertainty management.
That means the future of reliable AI will likely involve additional layers around the model itself:
- dynamic information gathering,
- structured state management,
- probabilistic risk estimation,
- escalation policies,
- evaluator agents,
- and deterministic workflows.
In other words, the future may belong less to standalone chatbots and more to AI-native decision systems.
From Answers to Questions
One of the most underrated capabilities in AI is not answering questions, but generating the right questions.
Experienced physicians, investigators, engineers, and operators are often valuable not because they already know the answer, but because they know what information is still missing. They understand which missing variable could radically change the outcome.
This suggests an important architectural shift.
AI systems should not simply retrieve knowledge from documents or databases. They should:
- ingest domain knowledge,
- automatically derive required information schemas,
- identify high-risk failure modes,
- generate adaptive questioning workflows,
- collect structured evidence,
- preserve “unknown” states explicitly,
- and escalate to humans when ambiguity or risk exceeds thresholds.
This is not just prompt engineering. It combines ideas from safety engineering, Bayesian reasoning, workflow orchestration, knowledge graphs, and human-computer interaction.
The Evolution of Human Roles
As AI systems mature, I suspect we will gradually move from HITL (Human-in-the-Loop) toward HOTL (Human-on-the-Loop).
That does not mean humans become less important. In many ways, the opposite is true.
AI may increasingly handle:
- large-scale information gathering,
- pattern recognition,
- workflow execution,
- and probabilistic inference.
But humans will remain responsible for:
- judgment under ambiguity,
- ethical responsibility,
- persuasion and trust,
- contextual tradeoffs,
- and difficult conversations.
Stanford physician Jonathan Chen recently described this well through three qualities that remain essential for humans:
- Competence — experience-based judgment,
- Communication — the ability to guide people effectively,
- Character — the integrity to communicate uncomfortable truths honestly.
Those qualities become even more important as AI becomes more capable.
Building the Infrastructure Layer
I believe one of the most important opportunities ahead is building platforms that can transform unstructured enterprise knowledge into operational decision systems.
Not just AI that can “answer questions,” but AI that can:
- determine what must be known,
- identify what is still unknown,
- continuously improve questioning strategies from failures,
- and integrate safely into real organizational workflows.
This may ultimately become one of the defining infrastructure layers of the AI-native startups.