Thinking Machines Lab has officially released Inkling 2 Mini, a compact open-weights artificial intelligence model engineered to run locally on hardware with modest memory profiles. The release brings advanced multimodal reasoning and software engineering capabilities directly to Apple Silicon Macs equipped with 32GB of unified memory.
By leveraging a novel looped transformer design combined with a sparse Mixture-of-Experts (MoE) architecture, Inkling 2 Mini delivers compute performance equivalent to significantly larger models while maintaining a lightweight disk and memory footprint. The launch marks a major milestone for developers seeking high-performance local AI inference without relying on cloud-based APIs.
thinking machines inkling 2 mini mac
Thinking Machines Lab released Inkling 2 Mini as an open-weights vision-language model designed for local developer workflows, agentic coding, and tool execution. Despite packing 26.4 billion total parameters, its specialized looped architecture allows the model checkpoint to compress down to approximately 20GB, enabling smooth execution on Apple Silicon Mac computers with 32GB of unified memory.
Thinking Machines Unveils Inkling 2 Mini MoE Model
The arrival of Inkling 2 Mini follows the earlier unveiling of Thinking Machines' massive Inkling 2 Small model. While the larger variant required high-end enterprise GPU clusters with hundreds of gigabytes of VRAM to function, Inkling 2 Mini targets local edge compute environments. Distributed under the permissive Apache 2.0 open source license, the model allows researchers and software engineers to download, modify, and deploy the weights across local machines.
The release reflects a growing push across the artificial intelligence ecosystem toward efficient desktop deployment. Rather than expanding model sizes exponentially, AI labs are seeking algorithmic innovations that maximize the utility of consumer-grade hardware. As developers increasingly build offline-first tools, the ability to execute complex reasoning tasks locally has become a top priority for developer platforms.
Technical Architecture and Looped Transformer Design
At the core of Inkling 2 Mini is a looped Mixture-of-Experts architecture. Standard transformer models pass data sequentially through a fixed stack of physical layers. In contrast, Inkling 2 Mini contains 24 physical decoder layers but reuses its middle 12 layers three times in a repeating loop during inference. This design allows the system to perform 48 logical layer evaluations per token while only storing 24 physical layers in system memory.
Through this technique, the model maintains a physical parameter count of 26.4 billion, but activates only 3.4 billion parameters per pass. Accounting for the repeated execution loops, the effective compute parameter count reaches 5.4 billion active parameters per token. The framework utilizes 128 routing experts per layer, selecting 8 active experts per token pass. The resulting architecture fits comfortably within a 20GB mixed NVFP4 and MXFP8 precision format, allowing local execution without overwhelming system memory.
Local Agentic Coding and Tool Execution Capabilities
Despite its compact physical size, Inkling 2 Mini demonstrates competitive benchmark results against larger open models like Qwen3.8-27B. In software engineering evaluations, the model achieved a 79.2 percent success rate on SWE-bench Verified in bash environments and 48.2 percent on DeepSWE v1.1. It also features built-in visual capabilities through a dedicated vision encoder module, enabling it to process image inputs and text prompts simultaneously while maintaining a massive context window of up to 1 million tokens.
The model is specifically tailored for local coding assistants, automated terminal tools, and retrieval-augmented generation (RAG) pipelines. To support these developer workflows, tools like the open source Mac Health MCP tool have demonstrated how local agents can monitor system metrics, while developer tools like Apple's Xcode plugin for Claude Code illustrate the demand for direct IDE integrations.
Hardware Requirements for Apple Silicon Macs
Running Inkling 2 Mini locally requires an Apple Silicon Mac equipped with at least 32GB of unified memory, such as a MacBook Pro, Mac mini, or Mac Studio powered by M-series chips. Quantized GGUF builds provided through open source runtimes like Unsloth and llama.cpp reduce the baseline memory footprint to approximately 17.6GB for 4-bit precision formats, leaving ample headroom for the operating system and background applications.
For users running distributed setups across multiple devices, community projects like OMLX distributed local LLM inference demonstrate how multiple Macs can pool compute resources for expanded context prefilling. Hardware acceleration frameworks like Ollama, Exo, SGLang, and vLLM have also added support for Inkling 2 Mini's looped layer architecture.
Impact on Local AI Workflows for Mac Developers
The release of Inkling 2 Mini highlights the changing landscape of desktop AI development. By proving that layer-looping techniques can dramatically increase model capacity without expanding memory consumption, Thinking Machines has opened new doors for researchers and software engineers.
As major tech ecosystems continue to evolve, including hardware memory supply chains seen in Apple DRAM memory supply deals and platform software updates like macOS Golden Gate beta releases, the ability to run high-capability models locally will likely remain a key differentiator for creators and developers alike.
With Inkling 2 Mini now available on Hugging Face and the Tinker platform, developers can immediately integrate the model into local pipelines, experiment with custom fine-tuning, and build responsive AI agents directly on their Macs.