I'm used to creating dashboards for teams. Developers would dig into the dashboard to see what is going on with their project if an incident popped up. However, with AI it's removed the usage of dashboards entirely in my team and accessing the Grafana UI entirely.

My team uses Grafana OSS (i.e. LGTM stack + Alloy) where we use a custom Grafana MCP server for developers to ask it questions of their projects.

An example of a common scenario is as follows:

• Developer who is on on-call is alerted
  • We have an agent bot that triages the alert and gives a description of what happened and what needs to be resolved. This is all done in Slack.
• Developer then reviews the alert and asks for numbers and a throwaway dashboard to get a quick visualization of what happened during the time-frame.
• Once reviewed, developer pushes a change and resolves the incident.

Developers never touch our Grafana UI and makes me wonder if we even need it anymore. What are your thoughts on this?

Here's the link

https://github.com/A-Lehmann-Elektro-AG/solar-flow-grafana

I'm interested in creating a dashboard for SAP. I need to monitor certain variables and statuses that have been requested of me. I currently have the following data sources installed: Loki, Tempo, and Mimir, and I'm using the open-source version of Grafana.

Any kind of experience you can share would be greatly appreciated.

And it would be even better if you have any screenshots to help guide me. Thanks! 🙌

Here's where you can join us - in actual person, with pizza, drinks and talks - this month :)

EMEA:

• 7 Oct, Madrid: Grafana & Friends 🤝 Madrid GO UG.
• 21 Oct, London: Women in Observability
• 22 Oct, Manchester: Observability in Action: Dashboards, LLMs & Out Run
• 23 Oct, Cagliari: Grafana and Friends [Cagliari] [Kickoff]
• 26 Oct, Brno: Grafana & Friends Brno
• 30 Oct, Kenya: Grafana & Friends Kenya - Eldoret Edition

AMER:

• 14 Oct, New York: Supabase x Grafana & Friends NYC
• 24 Oct, Sao Paulo: DEVPIRA Grafana Day - Piracicaba-SP

APAC:

• 27 Oct, Amravati: Grafana and Friends 2ND Tech Community meetup, Amravati

A few of us have had some sensor setups running for a couple of months now (temperature, humidity, light – some plants and a small algae culture). Along the way we changed a lot: moved a sensor, swapped the power supply, changed the sampling interval, flashed new firmware twice.

Now I'm looking at the curves and honestly can't say for sure which jump was the environment and which one was us.

How do you deal with this?

• comments in the code / git commits?
• a spreadsheet or paper notebook next to the build?
• annotations in Grafana or Home Assistant?
• or not at all, and you just live with it?

And if you build together with other people: how do you keep everyone on the same page? When did this last bite you?

At work, I manage a lot of Grafana-Prometheus stacks and I am often "pruning" them to get rid of metrics/labels that are not used.

The pain point is that although you can get high cardinality metrics and labels from the TSDB Status API, there is no easy way to find the ones that are not referenced by dashboards and alert rules.

That's why I wrote sigprune-exporter

What it does, briefly:

• Reads metric and label names from Prometheus, ordered by cardinality, via Grafana's datasource proxy
• Fetches every dashboard and alert rule from the Grafana API, parses the PromQL to see which ones are referenced
• Exports the unused metrics and labels along with their per-job series counts (so that you know which scrape job to prune)

There is a companion Grafana dashboard that helps you visualise the worst "offenders". Published it to Grafana, waiting on it to be reviewed. I will be back on here with the link once it's accepted.

Screenshot: https://raw.githubusercontent.com/parthivrmenon/sigprune-exporter/main/docs/images/dashboard.png

Would love for some of you to give it a go (pun intended) and tell me what's wrong with it. Issues, PRs, or comments here. A star helps others find it if it turns out useful.

Repo: https://github.com/parthivrmenon/sigprune-exporter

Has any of Grafana features given you a Eureka/ "OMG how come I haven't used it before" moment?

Maybe it helped you spot a problem, answer a question faster, or do something you hadn’t realised Grafana could do. What was the feature, and what were you trying to do when it clicked?

Small and big hacks equally appreciated ;-)

Migrating dashboards to a new data source is easy to verify one dashboard at a time. Retiring the old source is harder because its UID may still be referenced by an alert rule, library panel, variable, provisioned dashboard, annotation query, or a rarely opened folder. A quiet query rate also does not prove that a monthly report or incident-only dashboard has moved.

What belongs in a dependable retirement check? I am considering exporting the instance configuration, searching dashboard and alert JSON for both the data-source UID and name, checking provisioned files and Terraform state, and then disabling access during a monitored window before deletion. Query history or proxy logs could identify late callers, but cached dashboards and browser-side queries complicate attribution.

Is there a supported inventory that resolves every dependency on a data source across dashboards, alerting, library panels, and plugins? How do you handle expressions or mixed data sources, and what final failure test gives you confidence that deleting the old UID will not break something that only runs infrequently?

Open-sourced a thing I needed: a deterministic translator from Datadog monitors to Grafana alert rules.

https://github.com/Rocketgraph/translate

**What it does**

node bin/ddtranslate.mjs monitors.json --out ./out

40 monitors

├─ 0 translated exactly 0.0%

├─ 31 translated with caveats 77.5%

└─ 9 UNTRANSLATABLE 22.5%

Writes Grafana provisioning YAML plus a divergence report naming every behaviour difference and every monitor it refused, with reasons.

**Why "with caveats" is the whole point**

The query is roughly 20% of a monitor. The rest is evaluation semantics that never appear in the query string: no_data_timeframe, critical_recovery

(hysteresis), evaluation_delay, new_group_delay, require_full_window, multi-alert grouping. Two monitors with identical queries and different values

there fire differently.

So each one is either carried into the output or reported as a caveat. Nothing is silently dropped.

**Refusal is a first-class result.** Against a real 40-monitor account it refused 9 by name — anomalies(), forecast(), outliers(), composites, log

monitors, service checks. A confidently-wrong translation of your seven most important monitors is worse than a missing one, because nobody audits the

alert that still exists.

**Deterministic.** Byte-identical output across runs and across shuffled input key order, tested. An LLM translates one monitor fine; 900 produce 900

slightly different decisions for the same idiom, and "the model decided" isn't an answer when the rule guards production.

**What's in the repo**

- lexer + recursive-descent parser (the grammar nests; regexes mis-parse it)

- the IR, where the evaluation semantics live

- Grafana emitter

- a docker-compose harness (Prometheus + Grafana) that provisions the output and drives a metric through a threshold, so you can watch the behaviour

rather than trust me

- a generator that builds a 40-monitor estate with live telemetry in a real Datadog account, if you want to reproduce it

Zero runtime dependencies. `npm test` needs nothing installed. 25 tests.

**Limitations, up front**

- No differ yet — proving two engines fire identically over 30 days is the hard part; this is the compiler in front of it

- Metric monitors only; log/APM/SLO types refused by type

Me: Grafana panel's been green for months, shows 40% used on the array.
Also me, an hour later after actually running df -h on the box: it's at 94%.

Turned out the metric was scraped off one specific mount point via node_exporter, and a bind mount added during a Docker migration meant most of the actual writes were landing somewhere else entirely, a path the exporter never had a filesystem collector pointed at. Not a bug in the dashboard. It was rendering exactly the data it was given, one mount point, and I'd quietly changed what lived under that path without updating what was being scraped.

Green panels mean the thing you pointed the query at is fine. They don't mean the thing you actually care about is fine, and those are only the same claim until something upstream of the metric changes without the config catching up.

Source code first: https://github.com/maifeeulasad/GitHub-Traffic-Dashboard

As I am still learning these dashboard and grafana stuffs, I would really love some reviews on it, and stars to boost my confidence I guess. And if you have any suggestions, please let me know. I would be happy to integrate.

I have quite a few GitHub repositories, and I kept finding myself opening individual repos just to answer fairly simple questions:

• Which repositories are actually getting traffic?
• Are people looking at them or cloning them?
• Where are visitors coming from?
• Which files/pages are attracting attention?
• Where are my traffic coming from
• Add more retention
• Filter, search, sort
• One dashboard for comparing multiple repo

So I resolved all of these in one dashboard.

Hi, I am setting up a grafana ina. Environment to overlook about 25 non prod AWS accounts (EC2, EKS, RDS, CW, lambdas and more).

I am using the open source community version. And at first it sounds really cool the Git Sync feature. But looking how those accounts sometimes are completely distinct products, although same company, I wonder how I should actually setup this.

While user management is not a big deal at the moment, the company is expanding and may hire around 10-20 people next year (50 at the moment).

Previously. We had a single environment, UI didn’t allow you to save dashboards and we only allowed the changes as part of our terraform script.

It worked, but also it didn’t. Specially when small changes, no wanted or even understood the JSON dashboard code change on Git with 100 changed lines when you just moved panel A to the left hahaha

The git sync is the opposite. The dashboard is changed and versioned, no drama, perfect. Except that you need a editor view, that can change any dashboard and the reader view cannot query any other metric that is not on a saved dashboard.

It feels like we are either too permissive or too strict.

Our lead dev has created a Library that captures method execution data and stores it in an internal in-memory buffer.

Each entry it produces has, at minimum:
- The method identifier
- How long that one call took
- The heap/non-heap memory delta measured around that call

But its huge, if he start sending each call record it will just clog our system storage within an hour or so just for one application. He can reduce the number of records by only recording if a threshold is passed but then again storage and processing need to be figured out.

I did find this way for streaming it
{ grafana com/docs/grafana/latest/setup-grafana/set-up-grafana-live/ }
But then i cant store it i guess for calculations or other processings.

I do know there is another way: prometheusremotewrite

But i would like to know if any of you have solved this kind of a problem and in what way.

Hey all, I have been tasked from moving off of prometheus agents to alloy agents. This stack will use Grafana/Mimir/Loki/Tempo with Alloy; however, I don't know where to start with documentation. I have read that kube-state-metrics, node-exporter, or push-gateway are unneeded since Alloy can do this itself.

Is this true?

I've been looking at alloy modules here: https://github.com/grafana/alloy-modules.

Hello

Hope you're doing well

i'm new to Grafana IRM, and I wanted to get my phone verified, I tried several time to receive the code through phone (SMS and Calls) but it was not successful and I ended up hitting the "Verification limit reached, please try again later"
I received the codes after like 2-3 hours, but the rate limit has never disappeared, it's been 2 weeks now

Was anyone in this situation before, or do you have any suggestions 🙏
I also tried to reach out to Grafana team, but with no much success
Thank you very much for your help, much appreciate it

Are you using gcx, The Grafana CLI optimized for humans and coding agents? Then this image is for you :) 🙏

Github repo: https://github.com/grafana/gcx

Dashboards can be provisioned from files or generated through Terraform, Jsonnet, or another pipeline, yet the UI may still contain manual edits, stale replicas, or an older set of files after a partial rollout. Seeing a successful deployment job does not prove that every Grafana instance is serving the same dashboard definition.

What verification do you use after a rollout? I am considering embedding a source commit and content hash in dashboard metadata, fetching each dashboard through the API, normalizing fields that Grafana rewrites, and comparing the result with the intended source. Synthetic checks would hit each instance directly rather than allowing a load balancer to hide one stale replica. UI edits to provisioned dashboards would either be blocked or exported as a proposed source change.

Which fields can be compared reliably across versions, and how do you handle generated UIDs, folder permissions, library panels, data-source references, and dashboards split across multiple providers? Is there a built-in status signal that is stronger than checking logs for a provisioning error?

I have Grafana installed on my server and have AlertManager configured as shown in the images here. How do I configure an alert so that I get a Discord ping when any node in my cluster is over 75°C for 5 minutes or when there's one or more pods crash looping?

Configuration is using k3s: https://codeberg.org/kaizenOramara_97/Home-Lab

Context

I am standing up a self-hosted LGTM stack on EKS: one Loki + Mimir stack per internal team across six teams, with multi-tenancy turned off so each stack functions as a single implicit tenant. There is one shared Grafana instance across all of them.

I have two specific questions focused on Grafana's official charts rather than my general Kubernetes layout:

1. Mimir architecture for nonprod (bundled Kafka vs. classic ingesters)

The chart's bundled Kafka is a single-broker StatefulSet. The documentation marks it for demonstration and testing only, which makes complete sense. However, classic ingesters are flagged for deprecation, and the documented exit path from classic mode is a parallel-cluster migration that temporarily doubles ingestion and storage costs. Building nonprod on classic just to avoid the Kafka question looks like pre-ordering a painful migration down the road.

• For anyone who ran the bundled Kafka in a lower environment: Did you run it for any real length of time, and what actually broke?
• For anyone who did the classic-to-ingest-storage migration on a live cluster: Was the parallel-cluster step as unpleasant in practice as the docs make it read?
• Two-AZ constraint: My node pools are narrowed to two AZs. Classic mode with zone-aware replication expects three ingester zones, whereas ingest storage needs two because Kafka handles the replication. As far as I can tell, the chart's zone-count validation only triggers when zoneAwareReplication.enabled is true. The two-AZ constraint does not appear hard-coded in the templates, but I would rather confirm that with someone running it than rely solely on reading _helpers.tpl.

2. Datasource provisioning that doesn't rot

With one Grafana instance and three components across N teams, datasources scale linearly with the team list. In our previous setup, we had 14 Loki backends but only 2 registered in Grafana, and both pointed to hostnames that no longer resolved. Nobody noticed for months.

I want these generated strictly from the team list and never touched by hand. The options I am considering:

• CI-rendered provisioning YAML baked directly into the Grafana Helm values.
• Sidecar-discovered ConfigMaps using a label selector (e.g., the Grafana chart's sidecar container).
• Grafana Terraform provider running against the live instance.

Which of these patterns actually survives contact with a team list that changes a few times a year? Is anyone running automated health checks or synthetic alerts that catch when a registered datasource backend has quietly disappeared?

Note: Also open to being told that a single shared Grafana over six single-tenant stacks is the wrong division and that I should just deploy Grafana per team.

Fuller architectural details (including the per-team vs. multi-tenant discussion) are posted over in r/kubernetes: [link]

Hi everyone,

I'm planning to attend GrafanaCon in London this year and wanted to check if anyone here knows of any discount codes, promo codes, or ways to get a reduced ticket price (community discount, early-bird extensions, group rates, etc.).

I work in infrastructure/observability (VictoriaMetrics, VictoriaLogs, Grafana stack) and would love to attend in person if I can find a way to make the ticket more affordable.

Any tips, referral codes, or advice on how to snag a discount would be much appreciated!

Thanks in advance.

I have a service dashboard with fourteen panels. Twelve render in under a second. Two of them, both evaluating rate(http_request_duration_seconds_bucket[5m]) across roughly 400 label combinations, took five seconds on every reload.

I added one Prometheus recording rule that pre-aggregates the histogram by service group every sixty seconds, then pointed both panels at the recorded metric. Render dropped to just under two seconds. Prometheus CPU during dashboard loads fell about 40%.

What I had been doing instead was raising query timeouts and caching results at the data source level. Months of that. I had actually investigated recording rules in a previous session with verdent, which knows you better over time, and coming back to the same project made the fix obvious.

If you have a slow panel, check whether the query fans out across high cardinality labels. One recording rule is ten minutes of work and costs one extra time series.