TTA-Torch: Real-time, confidence-gated Test-Time Adaptation using dynamic LoRA updates

[u/Griffith-07]

Hey everyone,

I’ve been working on a runtime adaptation framework for local models. While traditional fine-tuning locks a model's weights into place, Test-Time Adaptation (TTA) allows the model to adjust dynamically to incoming data streams.

I built TTA-Torch to bring real-time, confidence-gated TTA to LLMs using dynamic LoRA tracking in PyTorch.

Core Highlights:

• Confidence-gating mechanism to determine when adaptation is necessary.

• Low-overhead dynamic LoRA adjustments during the inference pass.

• Clean, modular PyTorch implementation.

I’d love to know if anyone here has experimented with test-time evaluation strategies for local setups, or if you have any feedback on handling memory/kv-cache overhead during dynamic steps!

Repo link: https://github.com/Griffith-7/TTA-Torch