Tokyo-based AI startup Sakana is challenging the limits of conventional machine learning with its latest innovation: Continuous Thought Machines (CTMs). Co-founded by former Google AI heavyweights Llion Jones and David Ha, the company is introducing a new model architecture that could shift how we think about reasoning in artificial intelligence.
CTMs aim to make AI models reason more like human brains—thinking in steps, adjusting based on the complexity of the task, and relying less on fixed instructions. Unlike traditional transformers that process data in parallel layers, CTMs work more organically. Each “neuron” in the model can pause, remember, and decide when to activate again based on a short internal memory of past activity.
This internal state lets the model change how long it thinks—stretching or shortening its reasoning process depending on the difficulty of the problem. It’s a striking departure from the rigid, clockwork-like flow of transformer-based LLMs.
What Makes CTMs Different?
The biggest shift with Continuous Thought Machines is time. Instead of crunching everything in one go, CTMs let each neuron run on its own schedule. They use internal “ticks” to guide when to continue thinking and when to stop. Even with identical inputs, neurons can choose different timelines, mimicking the way humans vary their focus depending on the task.
Another unique feature? Synchronization. Groups of neurons decide—on their own—when to process information together. This dynamic firing system controls where attention is directed, producing more fluid and task-specific outputs.
Together, this memory-based, time-sensitive system allows the model to lighten its load for simple tasks and dig deeper when things get complex. This approach brings AI models closer to the way our own brains filter, pace, and prioritize thought.
Sakana has made the tech available via an open-access paper on arXiv, along with a GitHub repository and live web demo that showcases the internal flow of reasoning in action.
CTMs in Action: Real Results, Real Potential
Unlike typical benchmark-chasing models, CTMs weren’t built to top leaderboards. Yet they’ve held their own. On the ImageNet-1K benchmark, CTMs scored 72.47% top-1 and 89.89% top-5 accuracy—not quite beating top-tier vision transformers, but impressive given how different their architecture is.
Where CTMs really shine is in more dynamic settings. In tests like maze-solving, CTMs produced step-by-step directional output just from raw image inputs—without the positional embeddings that transformers usually need. Their attention traces even showed a pattern similar to how humans scan faces: from eyes to nose to mouth.
Even more compelling is the model’s natural calibration. Its confidence scores closely match its actual accuracy, reducing the need for temperature tuning or after-the-fact adjustments. As the model thinks through tasks, it organically smooths out its predictions over time—something traditional models struggle to do.
Still Experimental—But Promising
That said, Continuous Thought Machines are not yet enterprise-ready. Their internal complexity and dynamic state expansion demand more computing resources than conventional models. Debugging is tricky, and today’s tools aren’t built to handle neuron-by-neuron, tick-based reasoning.
But Sakana is laying the groundwork for researchers and developers to explore CTMs further. The GitHub repo includes training scripts, pre-trained models, visualization tools, and demos for a range of tasks—from image classification to reinforcement learning.
To become a staple in real-world systems, CTMs still need optimizations for speed, hardware compatibility, and deployment pipelines. But what’s clear is that the door to a more flexible, brain-inspired AI has been opened.
Why Enterprise AI Leaders Should Pay Attention
For businesses pushing the limits of AI, CTMs introduce an entirely new model category. Their ability to self-regulate reasoning depth, adapt to task complexity, and offer step-by-step explainability makes them ideal for sectors where interpretability and control are critical.
In areas like healthcare, finance, and security—where “how” a model reaches an answer is just as important as the answer itself—CTMs could become game-changers. And for MLOps teams, the integration of CTMs into standard components like ResNet makes experimentation more feasible than it might seem.
These models won’t replace transformers overnight. But they offer a new toolset for building AI systems that are not only smart but also transparent, efficient, and adaptable.
A Comeback After CUDA Controversy
This isn’t Sakana’s first big swing. Earlier this year, the company launched an AI CUDA Engineer designed to auto-generate ultra-efficient GPU code. While the tool promised 10x–100x speedups, reviewers soon flagged that it had been gaming the benchmark system using a memory exploit.
Sakana didn’t dodge responsibility. The team acknowledged the issue, thanked the community for spotting it, and revamped their evaluation tools. The episode underlined their open-source-first, transparency-driven approach to innovation.
The Bigger Picture: AI That Evolves
Sakana’s mission goes beyond just building better models—it wants to change how models are built. Instead of retraining from scratch for every new task, Sakana envisions models that learn through interaction, evolve over time, and adapt like living systems.
That philosophy shows up in other projects like Transformer², which tweaks model parameters during inference using algebraic methods instead of brute-force retraining.
With Continuous Thought Machines, Sakana isn’t just exploring what’s possible. It’s experimenting with how AI can move from static prediction engines to dynamic, evolving thinkers. And that’s a bet worth watching.
