Research field report on fitting a 27B open model into a practical workstation build using llama.cpp, Vulkan, quantized KV cache, and Qwen’s hybrid Gated DeltaNet architecture.
Qwen3.8-27B is a 27-billion-parameter open model with a native context window of 262,144 tokens. On paper, those numbers sound more like datacenter territory than something intended for a single desktop GPU. Using an AMD Radeon RX 7900 XTX with 24 GB of VRAM, llama.cpp with the Vulkan backend, aggressive KV-cache quantization, and Qwen’s hybrid Gated DeltaNet architecture, we successfully ran Qwen3.8-27B locally with a 131,072-token context window.
It was usable for real coding work through OpenCode, while still leaving meaningful memory headroom on the GPU.
The Local Build
The model we used was:
Qwen3.8-27B-Q4_K_L.gguf
It was served locally with llama-server from llama.cpp on a Windows workstation using the Vulkan backend with full GPU offload.
Our important runtime settings were:
- Context window: 131,072 tokens
- GPU layers: all
- Flash Attention: enabled
- Main KV cache:
q4_1 / q4_1 - Speculative decoding: Qwen native MTP draft head
- Draft KV cache:
q5_1 / q5_1 - Batch size: 1024
- Micro-batch size: 512
- Parallel slots: 1
OpenCode connected to the server through an OpenAI-compatible provider named snakenet-llamacpp, with the model exposed under the alias:
qwen3.8-27b
This allowed the local model to function as a coding agent while retaining a context window twice the size of our previous 65K configuration.
Qwen3.8-27B uses 48 Gated DeltaNet layers and only 16 conventional Gated Attention layers. That hybrid design substantially reduces the long-context attention burden compared with a conventional Transformer stack.
The Architectural Breakthrough
The reason Qwen3.8-27B can support such large contexts on relatively modest hardware is not just quantization, its architecture changes where the expensive attention operations occur. A conventional Transformer typically applies full attention throughout the network. Full attention becomes increasingly expensive as context grows because its computational and memory requirements increase sharply with sequence length. Qwen3.8-27B instead uses a hybrid stack of 64 layers arranged as 16 repeated groups:
- Gated DeltaNet + FFN
- Gated DeltaNet + FFN
- Gated DeltaNet + FFN
- Gated Attention + FFN
In other words:
- 48 of 64 layers use Gated DeltaNet
- 16 of 64 layers use full Gated Attention
- 75% of the network uses the Gated DeltaNet path
- Only 25% of the layers carry the full quadratic-attention burden
When the context window gets large this becomes important because a conventional Transformer repeatedly pays the price of full attention as tokens accumulate. Qwen’s architecture performs that expensive operation in only one out of every four layers. This results in substantially less long-context pressure than an architecture built entirely from standard attention layers. The model spends less of its memory budget on the machinery required to remember very long sequences.
Quantizing the KV Cache
The architecture is only part of the story, quantized KV caching pushes the memory advantage further. Instead of storing the primary key/value cache in FP16, we used:
q4_1 for Kq4_1 for V
The model’s Multi-Token Prediction draft context used:
q5_1 for Kq5_1 for V
That dramatically reduces the memory required for long-context inference. The combination of hybrid attention plus quantized KV caching is what turns a 128K context from an impressive specification into something a 24 GB consumer GPU can actually run.
What the Benchmarks Show
The server successfully initialized the full 131,072-token context on the Radeon RX 7900 XTX. More importantly, we then used it for an actual OpenCode coding session rather than relying on a synthetic model-load test.
Observed Results
| Metric | Observed Result |
|---|---|
| Model | Qwen3.8-27B Q4_K_L |
| GPU | Radeon RX 7900 XTX 24 GB |
| Backend | llama.cpp + Vulkan |
| Configured context | 131,072 tokens |
| Live context observed | ~92,844 tokens |
| Dedicated GPU memory | ~19.5 GB |
| Shared GPU memory | ~1.7 GB |
| Generation throughput | ~12–14 tokens/sec |
| MTP draft acceptance | 99.4% |
| Accepted draft tokens | 167 of 168 |
The MTP result was particularly striking.
At one observed point, the speculative decoder accepted 167 of 168 generated draft tokens, a 99.4% acceptance rate. That means Qwen’s native Multi-Token Prediction head was doing useful work. Generation speed around 12 to 14 tokens per second is not instant, but for a 27B model operating locally with a very large coding context, it is practical for interactive development.
And the memory result was equally interesting. During the live session, Windows reported approximately 19.5 GB of dedicated GPU memory in use on a 24 GB card, even with the server configured for the full 131K context. That left enough headroom to make us curious about going farther.
The most important result is that a 27B-class model can run locally while also carrying a 128K working context. For agentic coding workflows, context capacity can be just as important as raw parameter count. Our coding agents need to retain:
- Architecture documents
- Repository structure
- Source files
- Previous tool results
- Test failures
- Implementation decisions
- Long-running conversational state
All of those compete for the same context window. A 65K context can become crowded surprisingly quickly during a substantial software-development session. At 128K, the model has substantially more room to keep an entire development problem in view before context management becomes necessary. Qwen3.8-27B achieves this through several techniques working together:
- A Q4 quantized model
- A hybrid Gated DeltaNet/full-attention architecture
- Quantized Q4 KV cache
- Flash Attention
- Native MTP speculative decoding
- Full GPU offload through llama.cpp
- A single large context slot rather than dividing memory among parallel users
None of these techniques alone explains the result, together, however, they produce a local inference configuration that would have looked implausible on a 24 GB workstation GPU not long ago.