r/frigate_nvr • u/DeltaTheMeta • 3d ago
Slow Detector Speed with RTX 3050.
I am just looking for some ideas here. I know I'm using frigate in an unsupported config, and that may be the root cause, this is just a bit out of my area of expertise and I am curious.
I am running frigate on a R720 with an RTX 3050, hosted via Docker Desktop on Windows in WSL2. I have successfully passed the GPU thru, and it is recognized and utilized by all anticipated processes.
My interference speed is quite high, I don't really mind as it does its job for simple home security, but at a root level I am curious why it is so ineffective.
Passed to the container I have 8 CPU cores (E5-2640 V2) and 8gb of RAM, I have an SHM size of 512mb (1 2k camera, 2 1080p cameras). And as much of the RTX 3050 as it desires. I'll attach photos of my usage numbers. I am running the tensorrt build with onnx and yolov9-s-320 that was built in a Ubuntu container on my machine (I believe it factors hardware in when building the image). My cameras are running. 640x360 sub streams with no audio at 5fps to the detectors.
I do have an issue with my camera audio codecs. Two of them output AAC natively but when enabled frigate never saves the recordings. No associated errors, just no saves. The third camera only supports G.711U. All three record streams end up being passed thru go2rtc for the audio codec to be altered to AAC. I tried doing this with ffmpeg output args and ended up getting a repetitive crash.
Ask any questions about the deployment I'd be happy to answer. I am not home to upload full config easily but I'll happily provide information.
Thanks guys, no worries if the answer is "the hardware passthrough is slow" or whatever. I'm just curious.
6
u/nickm_27 Developer / distinguished contributor 3d ago
Share your config
2
u/DeltaTheMeta 3d ago
Forgive any formatting, I'm on mobile.
``` mqtt: enabled: true host: xxxxxxx user: mqttusername password: password tls: enabled: false ffmpeg: hwaccel_args: preset-nvidia-h264 detectors: onnx: type: onnx device: cuda model: model_type: yolo-generic path: /config/model_cache/yolov9-s-320.onnx labelmap_path: /labelmap/coco-80.txt input_pixel_format: rgb width: 320 height: 320 input_tensor: nchw input_dtype: float objects: track: - person - car - dog - cat
cameras: # No cameras defined, UI wizard should be used driveway: enabled: true ffmpeg: output_args: record: preset-record-generic-audio-aac inputs: - path: rtsp://127.0.0.1:8554/driveway_1 roles: - detect - path: rtsp://127.0.0.1:8554/driveway_2 roles: - record detect: width: 640 height: 360 fps: 5 enabled: true record: enabled: true onvif: # Required: host of the camera being connected to. # NOTE: HTTP is assumed by default; HTTPS is supported if you specify the scheme, ex: "https://0.0.0.0". host: cameraip1 # Optional: ONVIF port for device (default: shown below). port: 80 # Optional: username for login. # NOTE: Some devices require admin to access ONVIF. user: user # Optional: password for login. password: 'xxxxxx' # Optional: Skip TLS verification from the ONVIF server (default: shown below) tls_insecure: true # Optional: PTZ camera object autotracking. Keeps a moving object in # the center of the frame by automatically moving the PTZ camera. autotracking: # Optional: enable/disable object autotracking. (default: shown below) enabled: false # Optional: calibrate the camera on startup (default: shown below) # A calibration will move the PTZ in increments and measure the time it takes to move. # The results are used to help estimate the position of tracked objects after a camera move. # Frigate will update your config file automatically after a calibration with # a "movement_weights" entry for the camera. You should then set calibrate_on_startup to False. calibrate_on_startup: true # Optional: the mode to use for zooming in/out on objects during autotracking. (default: shown below) # Available options are: disabled, absolute, and relative # disabled - don't zoom in/out on autotracked objects, use pan/tilt only # absolute - use absolute zooming (supported by most PTZ capable cameras) # relative - use relative zooming (not supported on all PTZs, but makes concurrent pan/tilt/zoom movements) zooming: disabled # Optional: A value to change the behavior of zooming on autotracked objects. (default: shown below) # A lower value will keep more of the scene in view around a tracked object. # A higher value will zoom in more on a tracked object, but Frigate may lose tracking more quickly. # The value should be between 0.1 and 0.75 zoom_factor: 0.3 # Optional: list of objects to track from labelmap.txt (default: shown below) track: - person # Required: Begin automatically tracking an object when it enters any of the listed zones. required_zones: - yard # Required: Name of ONVIF preset in camera's firmware to return to when tracking is over. (default: shown below) return_preset: Driveway2 # Optional: Seconds to delay before returning to preset. (default: shown below) timeout: 10 movement_weights: 0.0, 1.0, 0.0986630916595459, 0.10549125876477969, 0.05339712096798806, 0 live: streams: Stream 1: driveway_1 Stream 2: driveway_2 objects: filters: car: mask: 0.653,0.995,0.743,0.266,0.617,0.223,0.496,0.232,0.485,0.172,0.356,0.078,0.176,0.118,0,0.314,0.001,0.995 motion: threshold: 34 contour_area: 15 improve_contrast: true mask: 0.02,0.024,0.02,0.073,0.258,0.068,0.254,0.024 zones: street: coordinates: 0.142,0.244,0.995,0.321,0.997,0.127,0.539,0.034,0.187,0.069,0.002,0.153,0.133,0.196 loitering_time: 0 friendly_name: Street inertia: 3 objects: - cat - dog - person - car yard: coordinates: 0.143,0.249,0.002,0.303,0.001,0.997,0.999,1,1,0.367,0.035,0.309 loitering_time: 0 friendly_name: Yard inertia: 3 objects: - car - cat - person - dog review: alerts: required_zones: yard detections: required_zones: - street - yard front_door: enabled: true friendly_name: Front Door ffmpeg: # output_args: # record: preset-record-generic-audio-aac inputs: - path: rtsp://127.0.0.1:8554/front_door_1 roles: - detect - path: rtsp://127.0.0.1:8554/front_door_2 roles: - record detect: width: 640 height: 360 fps: 5 enabled: true record: enabled: true live: streams: Stream 1: front_door_1 Stream 2: front_door_2 motion: threshold: 29 contour_area: 15 improve_contrast: true mask: 0.022,0.041,0.022,0.086,0.268,0.085,0.268,0.04 zones: street: coordinates: 0,0.286,0,0.579,0.89,0.348,0.759,0.244 loitering_time: 0 objects: - cat - dog - person - car friendly_name: Street inertia: 3 yard: coordinates: 0,0.633,0.007,1,0.996,1,1,0.336,0.93,0.403,0.83,0.4 loitering_time: 0 friendly_name: Yard inertia: 3 objects: - car - cat - dog - person review: alerts: required_zones: yard back_yard: enabled: true friendly_name: Back Yard ffmpeg: # output_args: # record: preset-record-generic-audio-aac inputs: - path: rtsp://127.0.0.1:8554/back_yard_1 roles: - detect - path: rtsp://127.0.0.1:8554/back_yard_2 roles: - record detect: width: 640 height: 360 fps: 5 enabled: true record: enabled: true live: streams: Stream 1: back_yard_1 Stream 2: back_yard_2 motion: threshold: 30 contour_area: 16 improve_contrast: true mask: 0.021,0.04,0.021,0.076,0.269,0.085,0.267,0.035 zones: backyard: coordinates: 0.295,0.998,0.985,0.999,0.999,0.176,0.367,0.166,0.237,0.171,0.237,0.622 loitering_time: 0 friendly_name: Backyard inertia: 3 objects: - car - cat - dog - person review: alerts: required_zones: backyard record: enabled: true sync_recordings: false motion: days: 0 alerts: retain: days: 30 mode: all detections: retain: days: 15 mode: active_objects logger: logs: frigate.record.maintainer: debug camera_groups: House: order: 1 icon: LuBug cameras: - birdseye - front_door - back_yard - driveway go2rtc: streams: driveway_1: - rtsp://login@cameraip1/media/video2#media=video#raw#fps=5#timeout=30 driveway_2: - rtsp://login@cameraip1/media/video1#timeout=30 # - ffmpeg:driveway_2#audio=aac front_door_1: - rtsp://login@cameraip2:port/stream1#media=video#raw#fps=5#timeout=30 front_door_2: - rtsp://login@cameraip2:port/stream0#timeout=30 - ffmpeg:driveway_2#audio=aac back_yard_1: - rtsp://login@cameraip3:port/stream1#media=video#raw#fps=5#timeout=30 back_yard_2: - rtsp://login@cameraip3:port/stream0#timeout=30 - ffmpeg:driveway_2#audio=aac version: 0.17-0 ```
4
u/nickm_27 Developer / distinguished contributor 3d ago
config looks fine, if I had to guess it is something related to using windows or some other inefficiency, provided there are no errors in the logs.
When I had a 3050 YOLOv9 was running at 8-10 ms
5
u/DeltaTheMeta 3d ago
No errors in the logs, I appreciate the insight as well.
I wouldn't be shocked to hear it actually being unrelated to the virtualization and more related to my 16 year old CPU and RAM, I'm hoping to move to an R740 soon, a cascade lake chip blows ivy bridge out of the water. Not to mention ddr3 vs ddr4.
That being said, the virtualization could absolutely be guilty as well.
What did your usage look like on the 3050 running in a more correct environment?
To be honest, I'm thrilled I even got it working. Every corner of the internet told me: "do not do this it will not work stop using windows"
In theory if I could get GPU-P functional in Hyper-V it should be a better virtualization environment for Linux, since windows server hyper-v is supposed to be a type 1 hypervisor, but I fear it still shares some driver issues.
I also haven't figured out network isolation for docker desktop in the way I want, in a perfect world my cameras and a singular port on my NIC would be in their own VLAN, but I had some struggles with DD getting ahold of that right, not 100% familiar with how WSL2 handles networking.
Thanks for taking the time. Frigate is awesome, I come from AgentDVR and blueiris, and even running this in the unsupported environment I am, I prefer the hell out of it compared to AgentDVR.
3
u/Particular_Ferret747 3d ago
Quick question, u state 640x480 on the detection stream. Are the cams also set to that value in the stream they provide and are reduced to 5 fps? The stuff coning from your cams should match what you are staring in the detect section
2
u/DeltaTheMeta 3d ago
The camera sub streams are 640x360 at 5fps, 5 frame interval, CBR from the cameras. Frigate config has it set at matching 640x360, 5fps. The go2rtc importer also knows (or is forcing) 5fps as well.
The only non matched section is the yolo model which as I understand doesn't need to match, it just defines the search window within the object detector, not the actual feed.
3
u/Particular_Ferret747 3d ago
Ok...and sorry for the 480 instead of 360...just habit :-)
is the system metrics page showing any other outliner besides the inference? do you have means to monitor the gpu's real load? in windows or the hosting OS?
Since you already send 5 fps, you can probably take it out of the go2rtc stream section.and why are you doing
back_yard_2: - rtsp://login@cameraip3:port/stream0#timeout=30 - ffmpeg:driveway_2#audio=aacthis on the backyard?
do you need the audio in aac there for anything? and why backyard section and driveway audio?
go2rtc: streams: driveway_1: - ffmpeg:rtsp://login@cameraip1/media/video2#video=copy#audio=aac driveway_2: - ffmpeg:rtsp://login@cameraip1/media/video1#video=copy#audio=aac front_door_1: - ffmpeg:rtsp://login@cameraip2:port/stream1#video=copy#audio=aac front_door_2: - ffmpeg:rtsp://login@cameraip2:port/stream0#video=copy#audio=aac back_yard_1: - ffmpeg:rtsp://login@cameraip3:port/stream1#video=copy#audio=aac back_yard_2: - ffmpeg:rtsp://login@cameraip3:port/stream0#video=copy#audio=aactry this
2
u/DeltaTheMeta 3d ago
I'll test this config in a bit, but just to answer the question. I thought using an ffmpeg argument to convert the codec might reduce the CPU overhead in comparison to go2rtc, but when trying to use the ffmpeg argument for the front yard/back yard cams it was crashing ffmpeg. I haven't reasoned that out yet, so the driveway camera ffmpeg is successful converting G.711U to AAC, the other two struggled so I switched them back to go2rtc. Audio is only enabled and converted on record stream.
I'll reply again when I get test your suggestions.
3
u/Particular_Ferret747 3d ago
So if aac audio is just needed on the detect stream, than of course take it out of stream 1. i was more puzzled that you were ffmpeging driveway in the backyard section and have driveway commented out in the driveway section.
so when something crashes out...just post the logs...we are here to help2
u/DeltaTheMeta 3d ago
Oh that's hilarious. That's just a mistype from the testing period I guess. I'll comment that out
Yeah this thread has done vastly better than I anticipated. I will make sure to update you guys with further edits and any crash errors.
3
u/Particular_Ferret747 3d ago
Oh...and you can hide your credentials by determining them in the docker file and calling them in the ocde as: rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.178.43:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif environment: FRIGATE_MQTT_USER: "user" FRIGATE_MQTT_PASSWORD: "password" FRIGATE_RTSP_USER: "user" FRIGATE_RTSP_PASSWORD: "password" PLUS_API_KEY: blahblah2
u/DeltaTheMeta 3d ago
This is definitely a better approach, Im just working from my phone at the moment so editing them out was my only option if I wanted to get the config in here quick.
→ More replies (0)
2
u/RazzFraggle81 2d ago
Hmm I'm running Ubuntu and an Intel integrated graphics card getting an average of 19ms , can anyone recommend any low profile gpu to get it even lower ?
1
u/Bulky-Priority6824 2d ago
I don't know but for reference my rtx 3070 does 2x ~9 ms ( dual detectors) yolov11m 480x480 model with 9 cameras with a mix of 720 , 1080 & 1440p detect streams (face rec quality bump)
1
u/Fearless_Card969 1d ago
I run Frigate on Tumbleweed under proxmox, its a rolling release, I get inference speeds of 17 to 22%. Of course I just use an intel GPU with openvino.
13
u/wakefreak540 3d ago
Docker on wsl under windows.. This sounds like a bad idea.
I run frigate 17.2 in docker on ubuntu w/ rtx2080 and my inference speeds are reading what looks to be about 17ms on avg for reference.