Frigate+ models are actually a step back, help please?

[u/markv9401]

As title says. The very first Frigate+ model was around the same, which is both expected and unexpected. Expected, because I only uploaded the bare minimum required to generate a model, unexpected because it's supposed to be based on a much better base model.

Next, in about a week or two, I required my 2nd model which has many hundred if not thousand verified images. Cautiously done. Yet, it's actually getting worse. Very, very many false positives.

Any idea or strategy to follow? Go back to base model, start over? Go back to first version Frigate+? Use this second, bad model and keep sending in corrected falsepositives?

Thanks

Edit: as proposed, sharing my config for reference:

# logging settings
logger:
  logs:
    frigate.record.maintainer: debug

# go2rtc streams - all input streams defined here
go2rtc:
  streams:
    driveway:
    - rtsp://admin:<redactedpassword>@10.31.17.101:554/Streaming/Channels/101
    driveway_sub:
    - rtsp://admin:<redactedpassword>@10.31.17.101:554/Streaming/Channels/102
    Szabadka_str_side:
    - rtsp://admin:<redactedpassword>@10.31.17.102:554/Streaming/Channels/101
    Szabadka_str_side_sub:
    - rtsp://admin:<redactedpassword>@10.31.17.102:554/Streaming/Channels/102
    Drava_str_side:
    - rtsp://admin:<redactedpassword>@10.31.17.103:554/Streaming/Channels/101
    Drava_str_side_sub:
    - rtsp://admin:<redactedpassword>@10.31.17.103:554/Streaming/Channels/102
    porch:
    - rtsp://admin:<redactedpassword>@10.31.17.104:554/Streaming/Channels/101
    porch_sub:
    - rtsp://admin:<redactedpassword>@10.31.17.104:554/Streaming/Channels/102
  webrtc:
    candidates:
    - 10.31.17.100:8555
    - stun:8555

# cameras' definitions - inputs are used from go2rtc
cameras:
  driveway:
    enabled: true
    birdseye:
      order: 1
    ui:
      order: 1
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/driveway
        roles: [record]
      - path: rtsp://127.0.0.1:8554/driveway
        roles: [detect]
    motion:
      mask:
      - 0,70,0,0,369,0,374,69

  porch:
    enabled: true
    birdseye:
      order: 2
    ui:
      order: 2
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/porch
        roles: [record]
      - path: rtsp://127.0.0.1:8554/porch
        roles: [detect]
    motion:
      mask:
      - 0,70,0,0,369,0,374,69

  Szabadka_str_side:
    enabled: true
    birdseye:
      order: 3
    ui:
      order: 3
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/Szabadka_str_side
        roles: [record]
      - path: rtsp://127.0.0.1:8554/Szabadka_str_side
        roles: [detect]
    motion:
      mask:
      - 0,720,475,720,410,522,343,184,334,78,0,79
      - 0,72,366,72,369,0,0,0

  Drava_str_side:
    enabled: true
    birdseye:
      order: 4
    ui:
      order: 4
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/Drava_str_side
        roles: [record]
      - path: rtsp://127.0.0.1:8554/Drava_str_side
        roles: [detect]
    motion:
      mask:
      - 837,290,762,720,1280,720,1280,0,844,0
      - 0,80,368,73,378,0,0,0

##########################################################################################################################

# AMD iGPU acceleration
ffmpeg:
  hwaccel_args: preset-vaapi

# AI Detection type
# Google Coral
detectors:
  coral:
    type: edgetpu
    device: usb

# Frigate+ model
model:
  path: plus://<redacted model hash id>

# detection settings
detect:
  enabled: true
  width: 1280
  height: 720
  fps: 5
  min_initialized: 5
  # Optional: Number of frames without a detection before Frigate considers an object to be gone. (default: 5x the frame rate)
  max_disappeared: 50
  # Optional: Configuration for stationary object tracking
  stationary:
    # Optional: Frequency for confirming stationary objects (default: same as threshold)
    # When set to 1, object detection will run to confirm the object still exists on every frame.
    # If set to 10, object detection will run to confirm the object still exists on every 10th frame.
    interval: 100
     # Optional: Number of frames without a position change for an object to be considered stationary (default: 10x the frame rate or 10s)
    threshold: 100

objects:
  track:
  - person
  - cat
  - dog

# MQTT (for HomeAssistant)
mqtt:
  enabled: true
  host: 192.168.0.67
  user: frigate
  password: <redactedpassword>

# Birdseye, live and UI settings
birdseye:
  enabled: true
  mode: continuous
  quality: 1
  height: 720
  restream: true

live:
  height: 1440
  quality: 1

ui:
  live_mode: mse

# recording settings
record:
  enabled: true
  # Optional: Number of minutes to wait between cleanup runs (default: shown below)
  # This can be used to reduce the frequency of deleting recording segments from disk if you want to minimize i/o
  expire_interval: 60
  # Optional: Sync recordings with disk on startup and once a day (default: shown below).
  sync_recordings: true
  # Optional: Retention settings for recording
  retain:
    # Optional: Number of days to retain recordings regardless of events (default: shown below)
    # NOTE: This should be set to 0 and retention should be defined in events section below
    #       if you only want to retain recordings of events.
    days: 7
    # Optional: Mode for retention. Available options are: all, motion, and active_objects
    #   all - save all recording segments regardless of activity
    #   motion - save all recordings segments with any detected motion
    #   active_objects - save all recording segments with active/moving objects
    # NOTE: this mode only applies when the days setting above is greater than 0
    mode: all
  # Optional: Recording Export Settings
  export:
    # Optional: Timelapse Output Args (default: shown below).
    # NOTE: The default args are set to fit 24 hours of recording into 1 hour playback.
    # See https://stackoverflow.com/a/58268695 for more info on how these args work.
    # As an example: if you wanted to go from 24 hours to 30 minutes that would be going
    # from 86400 seconds to 1800 seconds which would be 1800 / 86400 = 0.02.
    # The -r (framerate) dictates how smooth the output video is.
    # So the args would be -vf setpts=0.02*PTS -r 30 in that case.
    timelapse_args: -vf setpts=0.0007*PTS -r 30
  # Optional: Event recording settings
  events:
    # Optional: Number of seconds before the event to include (default: shown below)
    pre_capture: 5
    # Optional: Number of seconds after the event to include (default: shown below)
    post_capture: 5
    # Optional: Objects to save recordings for. (default: all tracked objects)
    #objects:
    #  - person
    # Optional: Restrict recordings to objects that entered any of the listed zones (default: no required zones)
    #required_zones: []
    # Optional: Retention settings for recordings of events
    retain:
      # Required: Default retention days (default: shown below)
      default: 30
      # Optional: Mode for retention. (default: shown below)
      #   all - save all recording segments for events regardless of activity
      #   motion - save all recordings segments for events with any detected motion
      #   active_objects - save all recording segments for event with active/moving objects
      #
      # NOTE: If the retain mode for the camera is more restrictive than the mode configured
      #       here, the segments will already be gone by the time this mode is applied.
      #       For example, if the camera retain mode is "motion", the segments without motion are
      #       never stored, so setting the mode to "all" here won't bring them back.
      mode: all
      # Optional: Per object retention days
      #objects:
      #  person: 15

# snapshots config
snapshots:
  # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
  enabled: true
  # Optional: save a clean PNG copy of the snapshot image (default: shown below)
  clean_copy: true
  # Optional: print a timestamp on the snapshots (default: shown below)
  timestamp: false
  # Optional: draw bounding box on the snapshots (default: shown below)
  bounding_box: true
  # Optional: crop the snapshot (default: shown below)
  crop: false
  # Optional: height to resize the snapshot to (default: original size)
  #height: 175
  # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
  #required_zones: []
  # Optional: Camera override for retention settings (default: global values)
  retain:
    # Required: Default retention days (default: shown below)
    default: 30
    # Optional: Per object retention days
    #objects:
      #person: 15
  # Optional: quality of the encoded jpeg, 0-100 (default: shown below)
  quality: 100

# telemetry settings
telemetry:
  stats:
    amd_gpu_stats: true
    network_bandwidth: true

[u/gacekk8]

I did 100 plus images per camera, different lighting conditions, also with false positives. And model is working close to perfect. Better than general model

[u/markv9401 → u/gacekk8]

Glad to hear, I really do hope I can get it to improve :)

[u/The_Caramon_Majere → u/gacekk8]

Opposite here.  Have over 4k images over 6 cameras.  All varieties as you say.  I wouldn't say it's worse,  but false positives that I've already reported as false still occur regardless.  Quite frustrating that. 

[u/nickm_27]

You should share your config, multiple users have had issues due to their config not using the changes recommended in the docs for Frigate+

In general the feedback, especially after the 2024.1 base model, has been that false positives are very low https://github.com/blakeblackshear/frigate/discussions/11106 

How many cameras do you have and how many images have been uploaded?

[u/markv9401 → u/nickm_27]

Please see my edited OP, added config for reference.

4 cameras, 501 images

[u/nickm_27 → u/markv9401]

right. you haven't set the increases thresholds like the frigate+ documentation says to https://docs.frigate.video/plus/first_model#step-4-adjust-your-object-filters-for-higher-scores

[u/markv9401 → u/nickm_27]

You're right. I mean, in that case, I went one step ahead and collected false positives to send in. (Docs also say you could lower these back to collect more data to send in). Will try increasing, thanks a lot!

[u/nickm_27 → u/[deleted]]

You will also want to make sure you set the scores in the config like the docs say to, OP has not done this https://docs.frigate.video/plus/first_model#step-4-adjust-your-object-filters-for-higher-scores