---
title: "PorTAL: Portable Task Adaptation for LoRA"
description: "Learn a task adaptation once in a base-agnostic form, then port it to new frozen models by refitting only a thin per-base alignment — recovering ~98% of per-task LoRA's lift on an unseen model within the same family and ~94% across families."
date: "2026-07-01"
authors: "Ben Geist"
tag: "Research"
slug: "portal-portable-task-adaptation"
canonical: "https://labs.ramp.com/research/portal-portable-task-adaptation/"
---

> [Figure: portal-portable-task-adaptation/portal-hero — portal-hero: PorTAL learns the core task fine-tuning once.]

*PorTAL learns a shared task latent and decoder core on seen bases, then ports to new models by refitting only a thin per-base alignment.*

Parameter-efficient fine-tuning (e.g. LoRA) adapts a frozen LLM to a task, but the resulting adapter is welded to one base model: when a new model is released, the adaptation must be re-learned from scratch. We study **portable task adaptation** — learning a task adaptation *once*, in a base-agnostic form, and transferring it to new frozen models by refitting only a small per-model component.

> [Figure: portal-portable-task-adaptation/key-results — stat-grid: PorTAL at a glance]

*Headline portability and data-efficiency results across unseen Qwen3-8B and cross-family Gemma-3-4B.*

Concretely, we learn a base-agnostic **task latent** $z_t$ together with a **per-base hypernetwork decoder** $D_b$ that generates per-layer LoRA for a frozen base; to port to a new base we **freeze** $z_t$ **and the shared decoder core**, and refit only a thin per-base alignment on a small amount of data.

This works both **within a model family** and, more strikingly, **across model families**, recovering most of the accuracy gain that a from-scratch per-task LoRA provides over the unadapted base. Freezing a task latent and shared decoder core learned on Qwen3-1.7B and 4B, then refitting only a thin per-base alignment, recovers **~98% of LoRA's lift on an unseen Qwen3-8B**, and **~94%** on Gemma-3, a different model family. Additionally the refit is **data-efficient**: PorTAL reaches the from-scratch-LoRA accuracy plateau with roughly half the calibration data, and at equal accuracy is **consistently better-calibrated** (lower held-out log-loss) than a from-scratch LoRA at every data size. It far outperforms transferring an existing adapter: a faithful **Cross-LoRA** baseline that aligns a strong source LoRA into the new model's activation space recovers only ~14% of the gain on the unseen 8B, versus our ~98% — learning a shared latent beats translating an existing adapter.

## Introduction and motivation

New language models arrive at an accelerating pace: the number of notable foundation models released per year rose from 2 in 2020 to 9 (2021), 32 (2022), and 149 in 2023[Stanford HAI — AI Index Report 2024](https://www.deeplearning.ai/the-batch/stanford-ai-index-report-shows-the-state-of-ai-in-2024), and by 2024–2025 the frontier had become so crowded that the Elo gap between the top and 10th-ranked model collapsed from 11.9% to 5.4% in a single year[Stanford HAI — AI Index Report 2025](https://hai.stanford.edu/ai-index/2025-ai-index-report). In practice, a stronger base model — often several — ships every few months.

Adapting a model to a task, however, is a *per-model* cost that does not amortize across these releases. A fine-tune — full or LoRA — is welded to one base model's weight space; when the next model ships, the adaptation must be redone on the new base. Parameter-efficient methods lowered the *unit* cost (a LoRA on a 7B model runs ~$1–3k vs ~$12k for full fine-tuning[Alloc Labs — The Hidden Cost of LLM Fine-Tuning](https://www.alloclabs.com/blog/hidden-cost-llm-finetuning)) but not its structure: you still pay data curation + a training run + evaluation once per (task, model), and full fine-tuning cost still scales with model size, which keeps growing.

The result is an economic treadmill: maintaining a portfolio of fine-tuned capabilities on the current frontier model costs roughly (cost-per-adaptation) × (#tasks) × (#model releases tracked). Because per-adaptation cost is roughly fixed while release cadence rises, this total grows over time — re-fine-tuning per model becomes the dominant, ever-growing cost of keeping a system specialized while also gaining the raw intelligence of each newer, smarter base.

> **Adapt once, port cheaply**

Inspired by the [Platonic Representation Hypothesis (Huh et al., 2024)](https://arxiv.org/abs/2405.07987), PorTAL expresses task adaptations in a base-agnostic form and moves them to a new model by refitting only a light per-base map on a handful of examples — dropping the marginal cost of upgrading a fine-tuned system from a full fine-tune to a small per-base refit.

## Related work

Our contribution is a combination and framing rather than a wholly new primitive, so we position it against three lines of work and credit each.

**Hypernetworks that generate LoRA for a single base.** [Text-to-LoRA (Charakorn et al., ICML 2025)](https://openreview.net/forum?id=zWskCdu3QA), [SHINE (2026)](https://arxiv.org/abs/2602.06358), and [Profile-to-PEFT (2025)](https://arxiv.org/abs/2510.16282) amortize per-task or per-user adaptation into a single forward pass, but target a *fixed* base and generalize across tasks or users, not across models (Text-to-LoRA explicitly leaves cross-model transfer open).

**Generating LoRA across architectures.** [LoRAGen (2025)](https://openreview.net/pdf?id=mrafO7aTYj) uses a structural embedding (latent + module/layer embeddings) to emit LoRA for different bases, but is trained by *reconstructing existing LoRAs*; we share its decoder shape yet train end-to-end on task loss, and crucially, *freeze a shared task latent and a shared core, refitting only a thin per-base alignment* to reach an unseen base.

**Transferring an existing LoRA across models.** [Cross-LoRA (Xia et al., 2025)](https://arxiv.org/abs/2508.05232), [LoRA-X (2025)](https://arxiv.org/abs/2501.16559), and [CAST (2025)](https://arxiv.org/abs/2510.17902) target the same goal we do, but by *translating one already-trained adapter* via subspace or activation-manifold alignment; we instead learn a base-agnostic latent and *regenerate* the adapter per base. Head-to-head this matters: a faithful Cross-LoRA (strong source + activation alignment, subsuming CAST) recovers ~14% of LoRA's lift on the unseen 8B versus our ~98% (§6.2).

In short, the pieces — hypernetwork→LoRA, cross-architecture generation, and cross-model transfer — all have prior art. Our contribution is combining them into a *freeze the latent and a shared core, refit only a thin per-base alignment* porting recipe, framed as a maintenance-cost answer to an accelerating model-release cadence and shown to dominate the adapter-transfer line empirically.

## Background: LoRA and LoRA hypernetworks

**LoRA.**[LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)](https://arxiv.org/abs/2106.09685) For a frozen weight $W \in \mathbb{R}^{d_{out}\times d_{in}}$, LoRA learns a low-rank update $\Delta W = \tfrac{\alpha}{r} B A$ with $A \in \mathbb{R}^{r\times d_{in}}$, $B \in \mathbb{R}^{d_{out}\times r}$, $r \ll d$, applied as $y = Wx + \tfrac{\alpha}{r} B(Ax)$. Only $A,B$ train.

**LoRA hypernetworks.** Rather than training $(A,B)$ directly, a *hypernetwork* generates them from a conditioning input. Text-to-LoRA trains a hypernetwork to emit a full LoRA for a single base model from a task-description embedding, end-to-end through the frozen base. This amortizes per-task optimization into one forward pass — but is single-base and generalizes across *tasks*, not across *models*. Separately, Cross-LoRA, LoRA-X, and CAST transfer a single *existing* LoRA across bases via SVD-subspace or activation-manifold alignment (per-task, training-free, no shared latent). Our design borrows the hypernetwork-generates-LoRA idea but targets the orthogonal axis — **cross-base transfer of a shared, learned task representation**.

## Method

Let a frozen base $b$ have transformer layers $\ell = 1,\dots,L_b$ and adapted projections $W_{\ell,m} \in \mathbb{R}^{d^{out}_{\ell,m}\times d^{in}_{\ell}}$ for modules $m \in \{q\_proj, v\_proj\}$ (we extend $m$ to all attention and MLP projections in the full-module variant.) Let $\theta_b$ denote the frozen base parameters.

**Task latent.** Each task $t$ is represented by a base-agnostic latent $z_t \in \mathbb{R}^{d_z}$ ($d_z = 256$).

**Per-base decoder (hypernetwork).** A small network $D_b$ maps the latent + a layer embedding $e_\ell$ to the LoRA factors of each (layer, module):

$$
(A_{\ell,m},\, B_{\ell,m}) = D_b(z_t, e_\ell, m), \qquad A_{\ell,m}\in\mathbb{R}^{r\times d^{in}_\ell},\; B_{\ell,m}\in\mathbb{R}^{d^{out}_{\ell,m}\times r}
$$

Internally $D_b$ computes a per-layer trunk with FiLM conditioning on $z_t$,

$$
h_\ell = \phi\big(W_2\,\big[(1+\gamma(z_t))\odot \psi(W_1[z_t; e_\ell]) + \beta(z_t)\big]\big),
$$

then per-module heads emit low-rank factors that a per-base linear alignment maps to $(A_{\ell,m}, B_{\ell,m})$ (the **Canonical decoder** below), with the $B$-path zero-initialized so $\Delta W_{\ell,m}=0$ at start (training begins from the clean base). The generated adapter is injected as a standard LoRA delta:

$$
y_{\ell,m} = W_{\ell,m}\,x + \tfrac{\alpha}{r}\, B_{\ell,m}\,(A_{\ell,m}\,x).
$$

**Canonical decoder — a portable factorization.** To make porting cheap, $D_b$ factors into a large **base-agnostic core**, shared across all bases, plus a **thin per-base alignment**. The core maps the latent and layer embedding to low-rank factors in a fixed *canonical* width $d_c$ that is independent of any base's dimensions; the per-base alignment is a learned per-layer embedding plus a **linear** map ($A = \hat A\,P^{in}$, $B = P^{out}\hat B$) projecting those canonical factors into the target base's actual per-module dimensions. The shared core is trained once on the seen bases; **porting to a new base reuses the core verbatim and fits only the thin linear alignment** — a small, near-linear cross-model map.

**Objective (source training).** Only $\{z_t\}$ and $D_b$ are trainable; $\theta_b$ is frozen. We minimize the gold-continuation NLL (loss only on answer tokens):

$$
\min_{\{z_t\},\, D_b}\; \sum_{t}\, \mathbb{E}_{(x,y)\sim \mathcal{D}_t}\big[-\log p_{\,\theta_b\,\oplus\, D_b(z_t)}(y \mid x)\big].
$$

Multi-task training uses balanced per-task steps with EMA loss-normalization (without it, hard tasks collapse to chance).

**Shared latent across bases.** For a set of seen bases $\mathcal{B}$, we share both $\{z_t\}$ and the base-agnostic core across all bases, keeping only a per-base alignment, and optimize $\sum_{b\in\mathcal{B}}\sum_t \mathcal{L}_{b,t}$. Naively, a small base monopolizes the shared latent's gradient; we apply **gradient-norm balancing on** $z_t$, rescaling each base's accumulated gradient on $z_t$ to equal norm before the optimizer step.

**Portability (the transfer step).** Given an unseen base $b'$, we **freeze** $\{z_t\}$ **and the shared core**, and fit only the thin per-base alignment of a fresh decoder $D_{b'}$ on a small calibration set:

$$
\min_{D_{b'}}\; \sum_t \mathbb{E}_{(x,y)\sim \mathcal{D}^{cal}_t}\big[-\log p_{\,\theta_{b'}\,\oplus\, D_{b'}(z_t)}(y\mid x)\big].
$$

> [Figure: portal-portable-task-adaptation/porting-algorithm — algorithm: PorTAL portability refit]

*Porting freezes the task latent and shared decoder core; only the thin per-base alignment is refit on calibration data.*

## Experimental setup

**Tasks (14, standard multiple-choice).** TruthfulQA, RTE, CB, COPA, WiC, WSC (SuperGLUE + TruthfulQA; higher-headroom), and BoolQ, ARC-Easy, ARC-Challenge, HellaSwag, OpenBookQA, WinoGrande, CommonsenseQA, SciQ (broader/bigger-eval).

**Metric.** Length-normalized log-likelihood over choices (`acc_norm`); we also report held-out **log-loss** (token-mean NLL of the gold continuation). §6.1–6.3 use **best-epoch** held-out selection (per-epoch eval) while §6.4 uses final-epoch eval. All are 3-seed means ± std.

**Data.** Up to 2,000 examples/task — a hard cap applied to **both** source training **and** the per-base alignment refit (some smaller tasks have fewer). §6.1–6.3 fit on the full 2,000/task; the data-efficiency study (§6.4) shows far fewer suffices. Evaluation sets range from 56 (CB) to 1,000 (BoolQ/WinoGrande/CSQA/SciQ); ~7,200 eval examples total on the 14-task suite.

**Models.** Seen bases: Qwen3-1.7B, Qwen3-4B. Unseen bases: Qwen3-8B (same family), Gemma-3-4B (different family). Per-task LoRA baselines: rank 16 on q/k/v/o + MLP. LoRA Hypernet/PorTAL (§6.1–6.3): rank 8 on q/v, $d_z=256$.

**Experiments reported.** (i) LoRA Hypernet vs per-task LoRA; (ii) portability to unseen bases within and across families; (iii) data-efficiency of the per-base alignment refit.

## Results

### Source base: LoRA Hypernet matches per-task LoRA

> [Figure: portal-portable-task-adaptation/source-base-results — table: Source base: LoRA Hypernet vs per-task LoRA (Qwen3-4B, 14 tasks)]

*On Qwen3-4B across 14 tasks, a jointly trained hypernetwork recovers ~94% of per-task LoRA's average lift and matches or beats it on 6/14 tasks (RTE, CB, COPA, WiC, ARC-Easy, CommonsenseQA).*

We first confirm that a hypernetwork — a learned task latent $z$ and the decoder, trained jointly on the source base — can match per-task LoRAs trained independently on the same base. It can: the generated **LoRA Hypernet** recovers **~94%** of per-task LoRA's lift on average.

### Within-family portability: unseen Qwen3-8B

> [Figure: portal-portable-task-adaptation/within-family-portability — table: Within-family portability on unseen Qwen3-8B (14 tasks)]

*Refitting only the per-base alignment on an unseen Qwen3-8B recovers ~98% of from-scratch LoRA's lift — far above a faithful Cross-LoRA transfer baseline at ~14%.*

We then test the central portability claim: a latent and core learned on the seen bases should port to an unseen base by refitting only the thin alignment. It does — on an unseen Qwen3-8B, refitting only the per-base alignment recovers **~98%** of the from-scratch LoRA's lift, reaching **0.792** against the 0.795 LoRA ceiling and far above the best cross-model **transfer** baseline. Freezing the latent is **free**, not a tax: training it *natively* on 8B from scratch reaches 0.785 ± 0.002 (~92%), statistically on par with the ported latent, so reusing a latent learned on *other* models costs essentially no quality while sparing the cost of re-deriving it per base.

### Cross-family portability

> [Figure: portal-portable-task-adaptation/cross-family-portability — table: Cross-family portability (14 tasks)]

*The same Qwen-trained frozen latent ports to Gemma-3 — a different family, tokenizer, and architecture — recovering ~94% of its from-scratch LoRA lift via only a thin per-base map.*

Finally, the cross-family test: does the *same* Qwen-trained frozen latent port to a different model family, not just a larger Qwen? It does — on Gemma-3, refitting only the lightweight per-base alignment recovers **~94%** of its from-scratch LoRA's lift, so cross-family transfer is nearly lossless.

### Data efficiency: PorTAL vs from-scratch LoRA

We hypothesize that PorTAL **amortizes** task adaptation: a latent and core learned once on the seen bases should make every *subsequent* model cheap to adapt, so porting to a new base needs far less data than training a LoRA from scratch there. We test this on the unseen Qwen3-8B, sweeping the refit-set size for three methods: PorTAL q/v r8, PorTAL full r8, and a separate per-task r16-full LoRA. All use the same per-size schedule and eval over 3 seeds, reporting final-epoch acc and log-loss. Raw 14-task averages use a base-8B floor acc of 0.667 and log-loss of 3.819.

<Tabs defaultValue="accuracy">
  <TabsList>
    <TabsTrigger value="accuracy">Accuracy</TabsTrigger>
    <TabsTrigger value="logloss">Held-out log-loss</TabsTrigger>
  </TabsList>
  <TabsContent value="accuracy">
    

> [Figure: portal-portable-task-adaptation/data-efficiency-accuracy — chart: Data efficiency: accuracy vs calibration size (unseen Qwen3-8B)]

*PorTAL q/v r8 reaches the from-scratch LoRA accuracy plateau (~0.77) at ~500 examples/task; from-scratch r16-full LoRA needs ~1,000. 3-seed means on unseen Qwen3-8B.*

  </TabsContent>
  <TabsContent value="logloss">
    

> [Figure: portal-portable-task-adaptation/data-efficiency-logloss — chart: Data efficiency: held-out log-loss vs calibration size (unseen Qwen3-8B)]

*PorTAL achieves lower held-out log-loss than from-scratch LoRA at every calibration size — a calibration effect from gentler logit edits by smaller adapters.*

  </TabsContent>
</Tabs>

**We find PorTAL is substantially more data-efficient.** It reaches the strongest from-scratch LoRA's ~0.77 accuracy max with roughly **2× less calibration data**: PorTAL q/v r8 hits that max at ~500 examples/task and full r8 at ~750, while from-scratch r16-full LoRA only settles there around ~1,000. PorTAL even ends slightly *above* it at the high end, 0.769 vs 0.762 at 2,000. Because the frozen base dominates per-step cost, reaching the target with half the data is also roughly **half the adaptation FLOPs** — the amortization we set out to test, since the latent and core are paid for only once.

PorTAL is also **better-calibrated**, with lower held-out log-loss than from-scratch LoRA at every data size. We read this as a calibration effect rather than a separate efficiency gain: held-out NLL penalizes overconfidence and a smaller adapter makes gentler logit edits, so the gap tracks adapter capacity.

## Future work

**Capacity competition on a cluster of hard tasks.** With best-epoch selection most tasks recover (10/14 reach ~80–110% of LoRA's lift), but a small cluster of harder commonsense/knowledge tasks underfits — OpenBookQA (~42% of lift), WinoGrande (~57%), HellaSwag (~61%) — the most-distinct tasks losing the shared rank-8 decoder's gradient competition. This is an **optimization** limit, not a capacity one: a rank-16 generated adapter did **not** help (it regressed and merely shifted which task starved). In the future we hope to pursue better multi-task optimization (per-task capacity, curriculum), or a small per-task residual on top of the shared decoder.

**Amortized text-description variant.** A natural extension replaces the free per-task latent with an encoder over a task *description* ($z_t = E(\mathrm{emb}(\text{desc}_t))$), so a brand-new task could be adapted **zero-shot from its description alone** (à la Text-to-LoRA), with no per-task training. We leave a full study to future work.

**Other directions.** Larger and instruction/generation tasks beyond multiple-choice; and theory on when a frozen latent suffices vs. when base-specific adaptation is required.

## Appendix A — Architecture and training details

### Architecture

Concrete configuration and dimensions for the §4 decoder:

- **Task latent** $z_t$: 256-d (a free per-task vector; or the encoder output in the amortized variant).
- **Layer embedding** $e_\ell$: learned `nn.Embedding(L, 32)` — breaks symmetry so one decoder yields a different adapter per layer.
- **Trunk**: Linear(256+32 → 512), GELU; FiLM modulate ($(1+\gamma)\odot h + \beta$, with $\gamma,\beta$ linear in $z_t$); Linear(512 → 512), GELU.
- **Heads** (one A- and one B-head per module $m\in\{q,v\}$): A-head Linear(512 → $r\cdot d^{in}$) reshaped to $[r,d^{in}]$; B-head Linear(512 → $d^{out}\cdot r$) reshaped to $[d^{out},r]$.
- **Adapter**: rank $r=8$ on `q_proj`/`v_proj` at every layer; scaling $\alpha/r=2$ ($\alpha=16$); injected by *wrapping* the module's real forward and adding $\text{scaling}\cdot(xA^\top)B^\top$.
- **Init**: B-heads and FiLM $\gamma,\beta$ zero-initialized, so the generated adapter is the identity ($\Delta W=0$) at start.
- **Per-base alignment**: a learned per-layer embedding plus per-module linear maps ($P^{in}$ projecting the canonical $d_c$ factors to the base's input width, $P^{out}$ to its output width).
- **Parameters**: the core holds the bulk; the latent table is 256 × #tasks.

### Training and hyperparameters

> [Figure: portal-portable-task-adaptation/hyperparameters — table: Training and hyperparameters]

*Full training configuration for source training, multi-base sharing, and portability refit.*

## Appendix B — Metrics: recovered lift vs retention

We report **recovered lift**; prior cross-model-transfer papers (Cross-LoRA, CAST) instead report **retention** (a ratio of accuracies). For a method $m$, unadapted base $b$, and from-scratch per-task LoRA $L$:

$$
\text{recovered lift} = \frac{\mathrm{acc}_m - \mathrm{acc}_b}{\mathrm{acc}_L - \mathrm{acc}_b}, \qquad \text{retention} = \frac{\mathrm{acc}_m}{\mathrm{acc}_L}.
$$

Retention is near 100% whenever there is little headroom ($\mathrm{acc}_b \approx \mathrm{acc}_L$) — the regime those papers operate in (their *trained* LoRA adds only ~1% over base), so it is not discriminative. We evaluate in a higher-headroom setting and therefore use recovered lift. For comparability, in retention terms our faithful Cross-LoRA reimplementation scores ~86% (within CAST's reported 85–95% band) while recovering only ~14% of the lift, whereas our porting scores ~99% retention / ~98% recovered lift — i.e. we win on both metrics, and the gap is only visible under the stricter one.

Research author — Ben Geist [@b_geist](https://x.com/b_geist)

Want to keep up with our next AI experiments? Subscribe [here](https://ramplabs.substack.com/) and follow us on [@RampLabs](https://x.com/RampLabs). We're also [hiring across roles](https://jobs.ashbyhq.com/ramp?utm_source=RampLabs) at Ramp.

## Citation

## Citation

See article metadata for citation details.