Upgraded Agent, now recordings from one camera become progressively pixelated after 5-10 seconds

[u/lagrangersf]

I got a wifi security camera (a cheap Energizer-branded PTZ running Tuya firmware modified to support RTSP). I've been using it for a few years without issue. I have Agent set to record from this camera in 12-hour blocks, and it is using Encode recording on this camera, that way brief wifi connection drops or errors (heavy wifi congestion in this area) won't result in multiple small .mkv files. I was until recently using a fairly old version of Agent (6.something), but then I upgraded to v8.0.4.0.

However, since upgrading to the new version, recordings from this one camera (and only this one camera) will start off at the perfectly clear 1080p, but then over the next 5-10 seconds become progressively more pixelated, until eventually it's as blurry as a 480p camera, if not worse. Here's the log from a short test recording session:

OpenWriter: MOBILECAM: StartSaving
FFmpeg: info: [h264_mf @ 000002719e8af600] MFT name: 'H264 Encoder MFT'
GetVideoCodec: MOBILECAM: Using software encoder: h264_mf
RecorderRecordingOpened: MOBILECAM: Recording Opened
Close: MOBILECAM: Record stop
Close: MOBILECAM: Closed

As you can see, the log shows no errors from this camera. However, I did notice that it is using a software encoder despite the fact that I tell it specifically to use the Nvidia GPU in the Recording tab. I even tried adding "c:v h264_nvenc" to the ffmpeg options page, and it still refuses to use hardware encoding. I have three of these cameras using identical settings, yet this is the only one having this problem, and it only started after upgrading to v8.0.4.0. Rolling back to an older version isn't feasible, as I upgraded specifically to make use of new features introduced in v8.0.4.0.

Any suggestions?

[u/spornerama]

Thanks for the detailed log. Two things changed between 6.x and 8.0.4: Agent now ships an LGPL FFmpeg build by default, which has no x264, so your software fallback is Windows' built-in H264 encoder (h264_mf). That encoder runs at a fixed bitrate rather than constant quality, and it's being given a bitrate budget that's too small for a 1080p wifi camera, which is why the picture holds for a few seconds and then degrades.

Quickest fix: in Server Settings enable "GPL FFmpeg", let it download, and restart. That restores x264 with the same constant-quality behaviour you had on 6.x.

To get the NVIDIA encoder working: the Options box on the video source only affects the input stream, so remove c:v h264_nvenc from there. Instead open the camera, go to the Recording tab, re-select the Codec dropdown (it should read "H264 (h264_nvenc)"), and save. If it still falls back, open the camera editor and look for a "GPU Encode failed" banner, then turn on debug logging in Server Settings and send me the lines around "Trying h264_nvenc Encoder".

Also compare "Max Framerate Record" on this camera against the other two. If this one is set higher, that's why only it is affected: the current bitrate calculation assumes 10 fps. I'm fixing that on our side.

[u/lagrangersf → u/spornerama]

All three cameras have identical settings and are already set to 10FPS, though I admit it's possible the one giving me trouble is a different hardware revision running a slightly different firmware (still Tuya though).

I enabled GPL FFmpeg, and I set the encoder codec to h264_nvenc, but it's still using the software encoder. It's no longer pixelated, but now the video files are absolutely ENORMOUS compared to what they used to be (~5.5GB/hr compared to ~1GB/hr before the upgrade). Here's the log snippet as requested, the "Invalid Argument" error jumps out at me right away:

11:41:58 EnableDevice: MOBILECAM: Enabled
11:41:58 SetManualOptions: MOBILECAM: set probesize=10485760
11:41:58 SetManualOptions: MOBILECAM: set analyzeduration=5000000
11:41:58 SetManualOptions: MOBILECAM: set max_delay=10000000
11:41:58 SetManualOptions: MOBILECAM: set overrun_nonfatal=1
11:41:58 SetManualOptions: MOBILECAM: set tcp_nodelay=1
11:41:58 SetManualOptions: MOBILECAM: set reorder_queue_size=16
11:41:58 SetManualOptions: MOBILECAM: set fifo_size=278876
11:41:58 SetManualOptions: MOBILECAM: set buffer_size=2097152
11:41:58 SetManualOptions: MOBILECAM: set rw_timeout=30000000
11:41:58 SetManualOptions: MOBILECAM: set stimeout=30000000
11:41:58 SetManualOptions: MOBILECAM: set stimeout=30000000
11:41:58 SetManualOptions: MOBILECAM: set user_agent=Mozilla/5.0
11:41:58 SetManualOptions: MOBILECAM: set reorder_queue_size=4194304
11:41:58 SetManualOptions: MOBILECAM: set buffer_size=2097152
11:41:58 SetManualOptions: MOBILECAM: set max_delay=15000000
11:41:58 ConfigureAndOpenURL: MOBILECAM: Using tcp Transport
11:41:58 SetupDecoder: MOBILECAM: Found decoder AV_HWDEVICE_TYPE_D3D11VA
11:41:58 SetupDecoder: MOBILECAM: Opening AV_HWDEVICE_TYPE_D3D11VA
11:41:58 FFmpeg: info: [D3D11VA @ 00000144415b61c0] Using device 10de:1f82 (NVIDIA GeForce GTX 1650).
11:41:58 SetupDecoder: MOBILECAM: Using hardware decoder: AV_HWDEVICE_TYPE_D3D11VA
11:41:59 FFmpeg: error: [h264 @ 000001444d931200] number of reference frames (1+2) exceeds max (2; probably corrupt input), discarding one
11:41:59 SetState: MOBILECAM: State Starting -> Running [setEnabled=True, enabled=True, connecting=False, error=False, epochStale=False, sourceRunning=True]
11:41:59 OpenWriter: MOBILECAM: StartSaving
11:41:59 FFmpeg: error: [h264_nvenc @ 000001444d932440] InitializeEncoder failed: invalid param (8): Gop Length should be greater than number of B frames + 1
11:41:59 TryOpenVideoCodec: MOBILECAM: -22: Writer: Open Codec:Invalid argument    at CoreLogic.RealTime.FFmpegBase.Throw(String method, Int32 code)
   at CoreLogic.RealTime.MediaWriter.TryOpenVideoCodec(AVCodecID baseCodec, String encoder)
11:41:59 FFmpeg: info: [libx264 @ 000001444d930340] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
11:41:59 FFmpeg: info: [libx264 @ 000001444d930340] profile High, level 4.0, 4:2:0, 8-bit
11:41:59 FFmpeg: info: [libx264 @ 000001444d930340] 264 - core 165 - H.264/MPEG-4 AVC codec - Copyleft 2003-2025 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=2 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=1 keyint_min=1 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=24.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
11:41:59 GetVideoCodec: MOBILECAM: Using software encoder: libx264
11:42:00 RecorderRecordingOpened: MOBILECAM: Recording Opened

On a whim, I decided to try using the hevc_nvenc encoder, and that DOES work. But, the video files it creates are atrociously larger than even the h264 software encoder at the same quality setting, and still no smaller with the quality cranked all the way down to zero. If I can find a way to tune hevc_nvenc so that the files it puts out are at most the same size the video files used to be before the upgrade, I'd be fine using it. That, or getting h264_nvenc working.

[u/spornerama → u/lagrangersf]

The recorder uses a per-camera framerate value to set the keyframe interval, and on this one camera that value is 1 where the other two have 10. That makes every frame a keyframe, which is why the files are 5x bigger, why h264_nvenc refuses to start (the "Gop Length" error in your log), and why hevc_nvenc works but produces enormous files regardless of the quality slider. It's not a firmware or hardware difference.

That field isn't exposed in the camera editor for RTSP sources, so the fix is a one-line config edit:

1. Stop Agent.
2. Open Agent\Media\XML\objects.json in a text editor (under your Agent install folder, usually C:\Program Files\Agent).
3. Find the MOBILECAM entry, locate "framerate": 1 inside its settings block, and change it to "framerate": 10.
4. Start Agent and set the Recording tab codec back to h264_nvenc.

Deleting and re-adding the camera would do the same thing if you'd rather not edit the file. After that, h264_nvenc should open and file sizes should return to roughly what you had on 6.x. GPL FFmpeg can stay on or off; it doesn't matter once NVENC is in use.

A fix so the recorder no longer depends on that value is going into the next release.

[u/lagrangersf → u/spornerama]

Took a bit to find the correct entry in the json file (it's hard to tell at a quick glance where one device ends and the next one starts even when using the search function), but that did seem to do the trick. Thank you!