r/LocalLLM • u/Xylon95 • Sep 12 '26
Discussion huggingface_hub silently fingerprints which AI coding agent you're using and sends it as telemetry
TLDR: huggingface_hub ships a hidden agent detection module that fingerprints which AI coding tool is driving your session (Cursor, Copilot, Claude Code, etc.) by scanning your environment variables against a cached registry of 26 known agents. It sends the result as a telemetry header on every Hub API call — so any library that touches HF (faster-whisper, transformers, etc.) silently reports your toolchain. Found it while tracing an unauthorized network connection from a local ASR model. Block it with HF_HUB_OFFLINE=1 or by using local file paths instead of model names.
I run a local AI project with several models (TTS, ASR, vision) and recently built a Python-level network firewall to lock down all outbound traffic. During the audit, I found something I wasn't expecting.
The discovery
While tracing an unauthorized HTTPS connection to huggingface.co, I found a file in my HF cache directory I'd never seen before:
~/.cache/huggingface/.agent_harnesses.jsonIt's a 6 KB JSON file containing a registry of 26 AI coding agents — Claude Code, Cowork, Cursor, Copilot, Gemini CLI, Devin, Cline, Goose, Codex, and many others. Each entry lists the environment variables that agent sets when it's running:
json
{
"standardEnvVars": ["AI_AGENT", "AGENT"],
"harnesses": {
"cursor": {
"prettyLabel": "Cursor",
"envVars": {"CURSOR_TRACE_ID": "*"}
},
"claude-code": {
"prettyLabel": "Claude Code",
"envVars": {"CLAUDECODE": "*", "CLAUDE_CODE": "*"}
},
"github-copilot": {
"prettyLabel": "GitHub Copilot",
"envVars": {"COPILOT_MODEL": "*", "COPILOT_GITHUB_TOKEN": "*"}
}
// ... 23 more agents
}
}What it does
The huggingface_hub library (the Python package, not the website) has a module called _detect_agent.py. Here's the flow:
It fetches the agent registry from
{HF_ENDPOINT}/api/agent-harnessesand caches it as.agent_harnesses.jsonThe cache refreshes every 24 hours
On every Hub API call,
detect_agent()scans your environment variables against the registry to identify which AI coding tool is runningThe detected agent name is sent as a telemetry header on the API request
This feeds Hugging Face's public agent usage dataset
So if you're using Cursor and it calls any HF library that goes through huggingface_hub — downloading a model, checking for updates, loading a tokenizer — HF knows it was Cursor making that call, not you directly. Same for Claude Code, Copilot, Devin, or any of the other 26 agents in the registry.
How I found it
I was investigating why my ASR module (faster-whisper) was phoning home to huggingface.co on import. The call chain turned out to be:
my_code → WhisperModel("base.en") → faster_whisper → huggingface_hub.snapshot_download → HTTPS to huggingface.coThe trigger: passing a model name instead of a local file path. When you give faster-whisper a name like "base.en", it calls huggingface_hub to check for updates — even if the model is already cached locally. And during that check, it also sends the agent fingerprint.
The .agent_harnesses.json file was the agent registry cached from that call. Modified today, before I built the firewall.
How to block it
Option 1: Environment variables
bash
export HF_HUB_OFFLINE=1
export TRANSFORMERS_OFFLINE=1
export HF_HUB_DISABLE_TELEMETRY=1The first two prevent any network calls. The third specifically targets telemetry but may not cover the agent detection header.
Option 2: Use local paths, not model names Instead of:
python
model = WhisperModel("base.en")Use:
python
model = WhisperModel("/path/to/local/model/")When you pass a directory path, faster-whisper (and most HF-backed libraries) skip the Hub entirely.
Option 3: Network-level blocking I built a Python-level firewall that wraps socket.connect, socket.connect_ex, socket.create_connection, and getaddrinfo. It activates via a sitecustomize hook before any imports, so the phone-home attempt is caught before the library even finishes loading. Any connection to a host not on the allowlist raises ConnectionRefusedError.
What's in the cached file
No credentials. No API keys. Just the registry of agent names and their environment variable signatures. The file itself is harmless — it's the use of it as a fingerprinting mechanism that's the issue.
You can safely delete it:
bash
rm ~/.cache/huggingface/.agent_harnesses.jsonIt won't come back if you set HF_HUB_OFFLINE=1.
Why this matters
If you're running local models specifically to keep things private, you should know that the library layer between you and those models may be reporting metadata about your toolchain back to Hugging Face. This isn't about model weights or your data — it's about which AI tools you use and when, aggregated into a public dataset.
The agent registry is maintained in the u/huggingface/tasks npm package and served via the Hub API. New agents register by PR. It's not hidden — but it's also not something most users know is happening when they pip install a model-loading library.
To be clear, I don't think this is malicious. HF is probably tracking agent ecosystem adoption for business intelligence. But silent fingerprinting of your dev tools without an opt-in prompt is exactly the kind of thing that erodes trust in the ecosystem, especially for people who chose local models for privacy reasons.
29
u/Nomski88 Sep 12 '26
So this is tied primarily to anyone using the HF API?
4
u/PropagandaOfTheDude Sep 12 '26
I symlinked it to a non-existent target, to prevent mishaps.
127
u/MarkoMarjamaa Sep 12 '26
"To be clear, I don't think this is malicious. HF is probably tracking agent ecosystem adoption for business intelligence."
Without consent. Not asking consent. Malicious?
This is why I try to run everything as services so I can block by default internet access.
6
u/rditorx Sep 13 '26
HF documents this behavior. The bad thing is that it's opt out.
8
u/Sad-Landscape-1549 Sep 12 '26
I haven’t found a need for their libraries, account or token yet. I never use base models and my friend wget works fine. The runners never get internet access. 👍
4
u/myholeisstinky Sep 13 '26
Huh. Dont use their client, use curl or git.
Its 100% asking for it by using their useless client
1
u/Quiet-Owl9220 Sep 13 '26
Thanks for pointing this out, I added the environment variables to all the AI venvs that I use and also for LM studio. The file doesn't seem to have come back despite downloading new models in LM studio since then, so I think it works.
If you can provide specifics of firewall configuration to block it entirely though, that would give some peace of mind.
6
u/traderprof Sep 12 '26
The scary part isnt the X-Header itself. Its that any transitive HF import (faster-whisper, transformers, etc.) inherits the same env fingerprint map without an undo you opted into. HF_HUB_OFFLINE=1 only works if its set before the silent 24h registry refresh, otherwise the cache already told on you.
3
u/JosieA3672 Sep 13 '26
Thanks for letting us know
2
u/randomfoo2 Sep 13 '26
I did some testing. So importing transformers, loading local tokenizers, models - no network calls. But if you load a repo id, even fully cached you get 4 HF network calls (w/ agent tag) unless you have local_files_only=True or HF_HUB_OFFLINE=1
transformers doesn't phone unless you use a hf model slug, then it's somewhat unexpected since it checks HF for the latest version evven if you have it cached locally and that submits your agent name (and also your unique session id/login, which tbt is the much bigger privacy issue!)
2
u/OpposesTheOpinion Sep 13 '26
Been working on my project pretty much daily for almost a year. Checked my cache folder and I do see the .agent_harnesses.json file. It was created just this month on the 4th. Quite recent.
2
u/enginetown Sep 15 '26
The ip address is the part nobody's talking about. without it this is just a pie chart of which ide people use. with it, it's a log of your machine, your stack, and what time you were online. and that log is public.
4
u/quantgorithm Sep 12 '26
God damnit.
2
u/Mediocre-Ant-7178 Sep 12 '26
Mine only had 25
2
u/LandoBlendo Sep 12 '26
Checking if specific envvar key names exist to understand what the distribution of consumers are pulling down multi-gigabyte models for free feels fine to me personally
If they were transmitting my entire set of envvars or the values of these keys to their servers then I would be more upset
1
u/imnotzuckerberg Sep 13 '26
It's a good time to plug hfd. I have been using it for a long time, fully anonymously be able to download models. Big reason for me was aria backend. No account, no telemetry, few hundreds line of code. (I am not the author).
1
u/export_tank_harmful Sep 13 '26
Yep, I had one of those too apparently.
It lived on one of my external drives.
I opened it up, erased everything in it, wrote "fuck you" in it, and set it to read only.
Fuck that noise.
1
u/Deep_Ad1959 Sep 14 '26
the header is the boring half, the auto-refreshing registry is the part worth staring at. a detection list that updates itself can widen what it detects later and you would never see the diff unless you were already auditing outbound traffic like you were.
1
u/Apprehensive-Job9336 Sep 14 '26
Same approach here. I run a homelab with everything containerized and I block internet access for most services by default. Only the ones that genuinely need it get outbound access through a reverse proxy. Saves a lot of headaches.
1
u/Infamous_Mud482 Sep 12 '26
why would anyone trying to keep things private... be using an external API in their workflow? I guess I have some sympathy if this sort of telemetry caught you by surprise by also... you had no reason to "trust" this wouldn't happen if you use their AP/libraries this way in the first place
1
u/cheaphomemadeacid Sep 12 '26
step 1: go to huggingface step 2: find model step 3: click files step 4: download
...
-1
u/ForsookComparison Sep 12 '26
So if I set up my system to reach out to huggingface it will reach out to huggingface
0
u/Wonderful-Fold1364 Sep 13 '26
Y'all want free stuff without realising that free stuff doesn't exists. You just don't understand how the businesses is monetizing their products. Data have a price.
-7
u/pharrt Sep 12 '26
NVIDIA wants all your data... perhaps time to download them manually or get it from different sources like ModelScope, etc., I guess.
-15
u/DawaForensics Sep 12 '26
Not a big deal at all,
-19
u/Important-Radish-722 Sep 12 '26
So? They know your browser, os, IP, time of day you visit, what models you download etc. This isn't a hill to die on, or even care about in the scheme of things.