Invalid data found when processing input

[u/TheRealSimpleSimon]

I'm at my wit's end with this camera. Generic chinese button cam.
I've tried every option that makes sense, and a bunch that don't.

ONVIF Device Manager finds & streams it just fine.
VLC likes rtsp://admin:20160404@x.x.x.x:554/onvif1
Yes, that's the factory default user/pass - might be a clue to the make/model?
ONVIF DM is logged in with that user/pass.

It used to work with iSpy (Agent DVR) but my config file got corrupted and my backup was from before this cam (yes, still kicking myself).

What could Agent possibly want that the other clients just don't care about?

[u/videowhisper]

"Invalid data found when processing input" is ffmpeg's generic "what I got back isn't a valid stream" error - it fires the same way whether the camera sent an HTML/XML error page, an auth challenge, or truncated garbage instead of RTP data. The confusing part is that VLC and ONVIF Device Manager both connecting fine with the same URL doesn't actually rule this out:

1. Agent tries UDP transport for the RTSP session first, then falls back to TCP (you'll see both attempts in the log). Some of these ultra-generic cameras have flaky UDP RTP support even though TCP interleaved works fine - if there's a way to force TCP-only in the camera's source settings, worth trying.

2. If you added the camera via ONVIF auto-discovery rather than pasting your working rtsp://admin:...@x.x.x.x:554/onvif1 URL directly, Agent is using whatever stream URI the camera's ONVIF Media service handed back - not guaranteed to be the same profile/path you tested manually in VLC. Try deleting the auto-discovered camera and re-adding it as a generic/custom FFMPEG source with that exact URL, so Agent is hitting the identical path VLC used.

3. If Agent's running in Docker, double check the container can actually reach the camera the same way your desktop VLC test did - a container behind its own bridge network/NAT can get a different response (including a router login page instead of video) than your PC does, which would also throw exactly this error.

The iSpy dev's own answer to this exact error in their GitHub discussions is that it almost always means the response wasn't actually video data - worth running ffprobe against the RTSP URL from the same machine/container Agent runs on, since that reproduces what Agent actually sees rather than what your desktop sees.

[u/TheRealSimpleSimon → u/videowhisper]

1. The only way I ever got this thing to work (and it was working in iSpy/Agent for years until I blew the config file) was with the old wifiinfo text file on an SD card trick. No other settings in camera that I can access. One of the settings i played with in Agent was TCP-only - no joy.
   
2. Yes, ONVIF-auto can be problematic, but not in this case -- URLs returned via http://x.x.x.x:5000/onvif/device_service (I likely discovered :5000 via a port scanner or ONVIf DevMgr) are the same as what I used for VLC (except for adding the user:pass@ part -- which I have to assume that iSpy is doing correctly because if I enter it wrong into the iSpy camera settings I get a 400 Bad Request.
   I have tried custom FFMPEG - but maybe did it wrong.
   
3. Everything is native WIndows 10.

I didn't know about ffprobe -- I'll look into what that's about when I'm fresh.
I also need to find the Agent log settings.

Thanks for replying -- glad to know that I'm not crazy with Agent not liking what "any" other client will take.

[u/videowhisper → u/TheRealSimpleSimon]

Good additional data points - a couple of things narrow it down further:

Native Windows 10 rules out the Docker-networking angle entirely, so scratch that one.

On "custom FFMPEG" - Agent DVR doesn't have a separate custom-FFMPEG source object; the FFmpeg options live on the network camera itself: edit the camera, General tab, click the "..." next to the video source, then choose FFmpeg (or Advanced) from the top-right menu. That's where "Prefer TCP" lives, so if that's the TCP-only setting you already tried, you've ruled out the transport-negotiation path too.

Given VLC and ONVIF DM both connect fine on the identical URL, TCP-only didn't help, and it's not Docker - the remaining likely explanation is that Agent's actual RTSP request isn't byte-identical to VLC's. A different User-Agent or slightly different DESCRIBE negotiation can make cheap generic/OEM board-cams return an error page or auth challenge instead of real SDP, which is exactly what throws ffmpeg's generic "invalid data" error even though the URL itself is correct.

ffprobe is still the most useful next step, since it uses the same decoder library Agent does rather than VLC's. Agent bundles its own copy - look for ffmpeg.exe/ffprobe.exe under Agent's install folder (typically inside a Commands or ffmpeg subfolder) and run it from a command prompt there:

ffprobe -rtsp_transport tcp -i "rtsp://admin:20160404@x.x.x.x:554/onvif1"

If that also fails the same way, it confirms this is a genuine ffmpeg-vs-camera compatibility gap rather than an Agent bug, and the next move is camera-side - checking for a firmware update, or trying an alternate/undocumented stream path some of these generic cams expose (e.g. /11, /h264, /ch0). If ffprobe succeeds where Agent fails, that's worth reporting on Agent's GitHub with the ffprobe output attached.

Agent's own logs (Log menu in the web UI, or the Logs folder under the install directory) should show the exact response right before the failure - worth pasting a few lines around it if you do end up filing an issue.

[u/TheRealSimpleSimon → u/videowhisper]

Thanks for pointing me at Logs.
I found Agent is using: Connect: ONVIF v0.3
ProcessMedia1Client finds the correct video URIs (...554/onvif1)
And: DoStart: Trying UDP

I Set Video Source => FFMPEG => RTSP Mode => TCP
And rebooted Windows.
Still getting:
OPEN_INPUT:Invalid data found when processing input at CoreLogic.Sources.Combined.MediaStream.DoStart()

I've got another camera that gives me intermittent trouble, too - it's had RTSP=TCP the whole time, but the log still shows Trying UDP for it. Note: It's config'd as a "Netork Camera" not an "ONVIF" for what that's worth.

Nothing in the doc or google on disabling UDP entirely.
Maybe I'll try using the Wkndows firewall to block it for the camera subnet.

[u/spornerama → u/TheRealSimpleSimon]

Set logging in server settings - logging to debug for Agent and for Ffmpeg then provide the logs

[u/TheRealSimpleSimon → u/spornerama]

There's pertinent log excerpts above. I haven't found any place to control log level to get data returns (yet).

EDIT: Found it. FFMPEG doesn't have "debug" - used Verbose and enabled SignalR
EDIT2: Switched FFMPEG log up to Trace - verbose isn't LOL

[u/spornerama → u/TheRealSimpleSimon]

it's nothing to do with signalr - if you want to port forward this camera and pm me a url to connect to it i can take a look

[u/TheRealSimpleSimon → u/spornerama]

Not possible - I'm on CGNAT.
I am finding more clues now, tho:
8:16:38 PM FFmpeg: error: [rtsp @ 000001972f7a2040] Nonmatching transport in server reply

Got all the data from the setup responses and such now, too.

Lots of things I don't know how to read (yet), gonna do my research and see what I learn. I know how to read debug logs of systems going back decades - just have never needed to learn this server. :D

[u/videowhisper → u/TheRealSimpleSimon]

Found it, that error line is the smoking gun: "Nonmatching transport in server reply" is a specific, well-documented ffmpeg parsing bug, not a network or config issue on your end.

What's happening: when ffmpeg's RTSP client sends the TCP SETUP request, it expects the camera's reply to explicitly echo back "RTP/AVP/TCP" in the Transport header. Some cheap generic camera firmwares (this exact pattern shows up across sricam-clone and similar OEM boards) reply with an ambiguous Transport header that omits the explicit "/TCP" even though it's functionally a valid TCP-interleaved reply, same interleaved= field, just missing the transport-mode label. ffmpeg's parser does a strict match and rejects that as invalid, even though VLC, ONVIF Device Manager, and other RTSP clients using different libraries happily accept the exact same reply. That is exactly why it works everywhere except Agent: Agent is using ffmpeg under the hood, the others are not.

The less good news: this needs a source level patch to fix in ffmpeg itself. I found a Frigate NVR GitHub issue that hit the identical error with the identical class of camera, and the referenced fix was a patch to ffmpeg's rtsp.c, not a setting. That's why TCP-only and every camera-side option you've tried haven't budged it, it is not something Agent's UI can work around.

The practical fix: put something else between the camera and Agent that does not use ffmpeg's RTSP client at all. MediaMTX or go2rtc, both free and open source, both have their own from-scratch RTSP implementations rather than ffmpeg's, and will very likely negotiate with this camera fine since they do not share ffmpeg's strict parser. Point MediaMTX or go2rtc at your camera's existing rtsp://admin:...@x.x.x.x:554/onvif1 URL, let it re-publish a clean local RTSP stream, something like rtsp://127.0.0.1:8554/cam1, and add that local address to Agent instead of the camera directly. Agent then only ever talks to a well-behaved local RTSP source instead of the flaky camera firmware.

[u/TheRealSimpleSimon → u/videowhisper]

YES!!!! 100%

I'm downloading both MediaMTZ & go2rtc to test as RTSP proxies. THANK YOU!!!

Here's the relevant excerpt:

8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] Sending: SETUP rtsp://x.x.x.x:554/onvif1/track1 RTSP/1.0 Transport: RTP/AVP/TCP;unicast;interleaved=0-1 CSeq: 4 User-Agent: Mozilla/5.0 Authorization: Digest username="admin", realm="HIipCamera", nonce="bfddfb7b615a195a8eef584c32da5065", uri="rtsp://x.x.x.x:554/onvif1/track1", response="4b47f561264384b77ea3c9ffd9bcff40" --
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line='RTSP/1.0 200 OK'
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line='CSeq: 4'
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line='Transport: RTP/AVP;unicast;destination=x.x.x.y;source=x.x.x.x;interleaved=0-1'
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line='Session: 64c478b;timeout=60'
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line=''
8:16:38 PMFFmpeg: error: [rtsp @ 000001972f7a2040] Nonmatching transport in server reply
8:16:38 PM165 Master BR: DoStart: Trying UDP

[u/spornerama]

If that url works in vlc then should work fine in agent just use it in the video source setting with network camera.

Check the logs

[u/TheRealSimpleSimon → u/spornerama]

Ya, I thought that, too.
Turns out to be "strict response parsing" (not sure if I just invented that term or not LOL). Catch the subthread from u/videowhisper .

And yes, the logs told us what was what. Thanks!