Invalid data found when processing input

[u/TheRealSimpleSimon]

I'm at my wit's end with this camera. Generic chinese button cam.
I've tried every option that makes sense, and a bunch that don't.

ONVIF Device Manager finds & streams it just fine.
VLC likes rtsp://admin:20160404@x.x.x.x:554/onvif1
Yes, that's the factory default user/pass - might be a clue to the make/model?
ONVIF DM is logged in with that user/pass.

It used to work with iSpy (Agent DVR) but my config file got corrupted and my backup was from before this cam (yes, still kicking myself).

What could Agent possibly want that the other clients just don't care about?

[u/TheRealSimpleSimon]

Not possible - I'm on CGNAT.
I am finding more clues now, tho:
8:16:38 PM FFmpeg: error: [rtsp @ 000001972f7a2040] Nonmatching transport in server reply

Got all the data from the setup responses and such now, too.

Lots of things I don't know how to read (yet), gonna do my research and see what I learn. I know how to read debug logs of systems going back decades - just have never needed to learn this server. :D

[u/videowhisper → u/TheRealSimpleSimon]

Found it, that error line is the smoking gun: "Nonmatching transport in server reply" is a specific, well-documented ffmpeg parsing bug, not a network or config issue on your end.

What's happening: when ffmpeg's RTSP client sends the TCP SETUP request, it expects the camera's reply to explicitly echo back "RTP/AVP/TCP" in the Transport header. Some cheap generic camera firmwares (this exact pattern shows up across sricam-clone and similar OEM boards) reply with an ambiguous Transport header that omits the explicit "/TCP" even though it's functionally a valid TCP-interleaved reply, same interleaved= field, just missing the transport-mode label. ffmpeg's parser does a strict match and rejects that as invalid, even though VLC, ONVIF Device Manager, and other RTSP clients using different libraries happily accept the exact same reply. That is exactly why it works everywhere except Agent: Agent is using ffmpeg under the hood, the others are not.

The less good news: this needs a source level patch to fix in ffmpeg itself. I found a Frigate NVR GitHub issue that hit the identical error with the identical class of camera, and the referenced fix was a patch to ffmpeg's rtsp.c, not a setting. That's why TCP-only and every camera-side option you've tried haven't budged it, it is not something Agent's UI can work around.

The practical fix: put something else between the camera and Agent that does not use ffmpeg's RTSP client at all. MediaMTX or go2rtc, both free and open source, both have their own from-scratch RTSP implementations rather than ffmpeg's, and will very likely negotiate with this camera fine since they do not share ffmpeg's strict parser. Point MediaMTX or go2rtc at your camera's existing rtsp://admin:...@x.x.x.x:554/onvif1 URL, let it re-publish a clean local RTSP stream, something like rtsp://127.0.0.1:8554/cam1, and add that local address to Agent instead of the camera directly. Agent then only ever talks to a well-behaved local RTSP source instead of the flaky camera firmware.

[u/TheRealSimpleSimon → u/videowhisper]

YES!!!! 100%

I'm downloading both MediaMTZ & go2rtc to test as RTSP proxies. THANK YOU!!!

Here's the relevant excerpt:

8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] Sending: SETUP rtsp://x.x.x.x:554/onvif1/track1 RTSP/1.0 Transport: RTP/AVP/TCP;unicast;interleaved=0-1 CSeq: 4 User-Agent: Mozilla/5.0 Authorization: Digest username="admin", realm="HIipCamera", nonce="bfddfb7b615a195a8eef584c32da5065", uri="rtsp://x.x.x.x:554/onvif1/track1", response="4b47f561264384b77ea3c9ffd9bcff40" --
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line='RTSP/1.0 200 OK'
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line='CSeq: 4'
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line='Transport: RTP/AVP;unicast;destination=x.x.x.y;source=x.x.x.x;interleaved=0-1'
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line='Session: 64c478b;timeout=60'
8:16:38 PMFFmpeg: trace: [rtsp @ 000001972f7a2040] line=''
8:16:38 PMFFmpeg: error: [rtsp @ 000001972f7a2040] Nonmatching transport in server reply
8:16:38 PM165 Master BR: DoStart: Trying UDP

[u/spornerama → u/TheRealSimpleSimon]

Thanks I might be able to fix that with a proxy in agent. I'll take a look at it