On this page
Interpretability
How we built Steer, our interpretability playground
We built a system that steers an LLM toward specific concepts without retraining, and in the process learned something concrete about where meaning lives in different model architectures. Steer a model toward expense management and it will connect any topic back to receipt reconciliation. Ask it about the weather and it explains how fluctuating temperatures are a lot like fluctuating reimbursement cycles. If you've talked to Anthropic's Golden Gate Claude, the experience is similar: a model with an obsession baked into its activations.
Why Activation Steering
Activation steering (Turner et al., 2023)1 modifies a model's internal representations at inference time. Instead of fine-tuning weights, a steering vector is added to the activations at specific layers during the forward pass. This makes it possible to alter what a model focuses on without touching the underlying parameters. It's a lightweight, reversible intervention, and it exposes something about how models organize information internally: which layers encode which concepts, how robust those encodings are, and what happens when you perturb them.
Opening It Up
We generalized the system. Instead of hardcoding one concept, the pipeline accepts any concept and generates a steering vector for it. A pre-loaded gallery includes "Existentialism", "Elon Musk", Rick Sanchez", and others. The training pipeline runs on @modal 's serverless GPU infrastructure. Submit a concept, the system extracts a steering vector, and a steered model is ready to chat with in short order.
Qwen
Our first model was Qwen 2.5 7B Instruct, a 7.61-billion-parameter model from Alibaba with 28 transformer layers, pretrained on a large-scale dataset encompassing up to 18 trillion tokens (Qwen Team, 2024)2. Three problems surfaced quickly.
- The pretraining reversion problem. Qwen's pretraining corpus is multilingual (29+ languages) (Qwen Team, 2024)3, though we suspect Chinese has outsized representation - the model's behavior under stress supports this. It seems like the instruction tuning that makes it fluent in English is a learned behavior layered on top of that foundation. Steering vectors work by nudging a model's internal representations, and when you push hard enough, you're effectively destabilizing the instruction-tuned behavior that sits on top of the pretrained base. Push too hard and the model doesn't just produce incoherent English. It reverts to its pretraining distribution, which for Qwen means generating Mandarin. A model steered toward the concept of "pizza" would suddenly drop half a sentence of Chinese mid-response. This suggested it was falling back to base behavior, the way a stressed system reverts to its default state.
- The size problem. 7B parameters isn't a lot. Small models have less redundancy in their representations, which means there's less room to push those representations around before things fall apart. Qwen 7B would become incoherent relatively quickly under moderate steering, especially for abstract or complex concepts.
- The calibration problem. When we first set up the system, we ran a magnitude sweep on the "Ramp" concept and picked three thresholds (low, medium, and strong) that produced good results. Then we reused those exact thresholds for every other concept. Different concepts live in different parts of the model's representation space and respond differently to the same magnitude. A magnitude that gently nudges the model toward "expense management" might completely obliterate its coherence when applied to "17th-century Dutch painting."
What we learned about Qwen's layers. A steering vector can be applied at any layer. The challenge was figuring out which layers produce useful results. In our experiments, steering early layers corrupted syntax and grammar — suggesting those layers handle low-level language processing that you don't want to interfere with. Steering late layers disrupted output fluency without meaningfully changing what the model was reasoning about, suggesting those layers are translating internal representations into tokens and perturbing them breaks generation. The sweet spot was in the mid-layers, where Qwen appeared to encode semantic concepts, but the usable window was narrow: a small band of effective layers, a tight multiplier range, and a model that would fall into Mandarin if you pushed past the boundary.
Moving to Gemma
To fix these problems, we switched from Qwen to Gemma 3 27B-IT, a 27-billion-parameter model from Google with 62 transformer layers and a distinctive alternating attention pattern: for every 5 local sliding window attention layers (1,024-token window) (Google DeepMind, 2025)4, there's 1 global self-attention layer that attends to the full context (Gemma Team et al., 2025)5. It is multilingual (140+ languages), but unlike Qwen, it does not revert to another language under heavy steering.
The results were immediate. The bilingual instability vanished. When Gemma degraded under heavy steering, it produced incoherent but still English-language output instead of switching languages entirely, and even degraded responses were legible and on-topic. The jump to 27B parameters also meant the model had far more capacity to absorb the perturbation of a steering vector without losing coherence. The baseline of response quality went up dramatically.
Under heavy steering, the model reverts to Mandarin — suggesting heavy Chinese representation in its multilingual pretraining data.
Output stays English even under heavy steering. Degrades to repetition, not language switching.
27B parameters = more capacity to absorb steering perturbation · degrades to English repetition, not language switching
But Gemma introduced a new challenge. We quickly discovered that Gemma is significantly more sensitive to steering vector application than Qwen. The magnitudes and layer selections that worked on Qwen were far too aggressive for Gemma. Applying Qwen's settings to Gemma produced models that were almost unusably over-steered.
Gemma required re-solving both layer selection and multiplier calibration. Unlike Qwen's 28 layers, Gemma has 62. Based on what we'd already learned about layer function, we scoped our experiments to layers 16–53, the 38 layers spanning from early concept formation through late-stage reasoning. In our testing, layers below 16 appeared to handle syntax and embedding, while layers above 53 appeared to handle output formatting and token generation.* Steering either end corrupted the model's basic language abilities rather than reshaping its thinking, so we excluded them from training. That still left us 38 layers to work with. More layers, more degrees of freedom, more ways to get it wrong.
Our empirical results suggest that layers below 16 primarily handle syntax and embedding, while layers above 53 handle output formatting and token generation.
*Based on our experimental observations with Gemma 3 27B-IT. Layer roles are empirical approximations, not architectural guarantees.
The Sweep
Moving from Qwen to Gemma revealed how differently the two models organize their internal representations. Qwen's 28 layers are uniform: standard self-attention throughout, with semantic concepts concentrated in a narrow mid-layer band. Gemma's 62 layers are structurally heterogeneous, alternating between local and global attention, with concept encoding distributed across a much wider range. Same task, same steering approach, but the models require completely different targeting strategies. In our experiments, architecture strongly affected where steering was effective.
We didn't go in with strong assumptions about which layers would work. We tested across the full range and let the results tell us.
The experimental design. We tested eight different layer configurations, five multiplier values (0.05 to 0.75), four distinct concepts, and eight diverse prompts. That's 1,280 total generations. Each one got evaluated by an LLM judge on coherence, keyword density, and a composite quality score.
The configurations ranged from sparse (5 evenly spaced global attention layers) to dense (contiguous blocks of 12-19 layers), targeting different network depths. We also tested steering all 38 candidate layers at once. Our hypothesis: sparse global layers would give us maximal steering with minimal risk.
Global attention layers have a wider receptive field (full context vs 1,024-token window), enabling concept integration without breaking token generation.
In our sweep, sparse global layers performed best. The sparse 5-layer global configuration maintained coherence across the entire multiplier range. At a multiplier of 0.75, the most aggressive value we tested, it produced 0% degenerate outputs (responses so broken they're unusable: repetition loops, gibberish, or complete topic failure) with a coherence score of 0.858. The dense late 12-layer configuration at 0.55 hit 0.113 coherence and 83% degenerate outputs.
In our tests, late layers were catastrophically sensitive. The dense 19-layer mid-to-late configuration at 0.55 resulted in 100% degenerate outputs. Steering each of the 38 candidate layers at 0.35 gave us 73% degeneracy. The later in the network you steer, the faster you hit the cliff where output quality collapses. This makes intuitive sense. Late layers are responsible for actually generating tokens, and perturbing them directly breaks the sampling process.
The degeneration cliff is steep and layer-dependent. The dense 12-layer mid-network configuration transitions from 0% degenerate at 0.15 to 13% at 0.75. That's a gradual slope, uncomfortable but usable. But the dense 13-layer mid-to-late configuration goes from 0% at 0.15 to 57% at 0.55. A massive cliff. The more layers you're steering, especially late layers, the faster your margin for error disappears.
In this sweep, global attention layers outperformed nearby local attention layers. Steering mid-to-late global layers produces thematic reframing, the model genuinely weaving the concept into its reasoning, rather than surface-level word substitution. Global attention at these depths gives the model a wider receptive field for concept integration, which helps it absorb and express the steered concept without degenerating. Local attention layers at the same depth were measurably worse because they can only see a 1,024-token sliding window, not the full sequence context.
For the concepts and prompts we tested, layer 41 was the best single-layer target. It's at approximately 66% depth in the network. Late enough to have abstract semantic information, early enough that perturbing it doesn't immediately break token generation. Layer 41 has the widest operating range before hitting degeneration.
Here's the data:
| Config | Multiplier | Coherence | Degenerate Rate | Keyword Density |
|---|---|---|---|---|
| Qwen 2.5 7B | 0.5x | High | 12% | Low |
| Qwen 2.5 7B | 1.0x | Medium | 28% | Medium |
| Gemma 2 9B | 0.5x | High | 8% | Low |
| Gemma 2 9B | 1.0x | Medium | 22% | Medium |
We went with the sparse 5-layer global configuration: layers 23, 29, 35, 41, 47. These results come from a relatively small sweep: four concepts, eight prompts, and LLM-judged evaluations. We used this to choose a practical default, not to claim a universal law of steering behavior.
The effective strength problem. Different concepts have different natural magnitudes in representation space. A "mild" steering strength for one concept might be catastrophic for another. The steering vector's magnitude depends on how the concept is distributed across the model's activations. In our experience, abstract concepts like "absurdism" tended to produce smaller vectors than concrete ones like "marine biology," so the same multiplier produced different effective strengths. We expose three presets (low, medium, strong) with multipliers calibrated through testing across a range of concept types. The default is tuned so the concept is present and clear without degrading the model's reasoning. Lower presets keep the concept subtle, useful for light thematic influence. The strong preset pushes closer to the coherence boundary, where the model's obsession with the concept becomes aggressive but output quality starts to trade off.
Serving Without Burning Money
Gemma 27B is not a small model. Steer AI is an experimental project, not a revenue-generating product, so we can't justify keeping a fleet of GPUs warm and idle around the clock.
The problem is cold starts. Loading Gemma's weights into GPU memory from scratch takes 60-120 seconds before the first token. Our solution leverages Modal's GPU memory snapshot capability (Modal Labs, 2025)6. We snapshot both container memory and GPU memory with Gemma already loaded. When a cold function spins up, it restores from this snapshot. The only remaining work is loading the user's specific steering vector. Cold start drops to 5-12 seconds.
Why This Matters
More organizations are going to be fine-tuning models for domain-specific use cases. When those models fail on specific inputs, the standard approach is to add more training data or adjust hyperparameters. Interpretability lets you peer inside the model and understand which representations are doing useful work and where the leverage points are. The tools are getting better. Anthropic's research on activation oracles (Karvonen et al., 2025)7 and applied interpretability from startups like @GoodfireAI are expanding what's possible.
Organizations that build interpretability muscle now will be better positioned as LLMs shift from black boxes to systems we understand and engineer. Steer AI is how we're building that muscle.
Steer AI is live at labs.ramp.com/steer-ai. Pick a concept and talk to it.
Article by René Sultan @rene_sultan
Want to keep up with our next AI experiments? Subscribe here and follow us on @RampLabs. We’re also hiring across roles at Ramp.
The empirical observations in this post — including layer function assignments, steering behavior, and sweep results — reflect our experiments on Gemma 3 27B-IT and Qwen 2.5 7B Instruct with a limited set of concepts and prompts. These findings informed our design decisions but should not be treated as general properties of these architectures. Results may vary with different models, concepts, or evaluation methods.