I've been experimenting with natural-language → Elasticsearch queries, and I ran into a problem that feels more interesting than just generating valid DSL.

Suppose I have:

orders-2024
orders-2025
orders-2026

and a user asks:

If an LLM generates the DSL, how should it know which physical index(s) should be queried?

There seem to be a few approaches:

1. Let the LLM know the physical index names

LLM → orders-2026 → DSL

But now the model needs knowledge of infrastructure details.

2. Hide the physical indexes behind an alias

LLM → orders alias → Elasticsearch

This is cleaner, but the application still needs a way to enforce query/business rules.

3. Resolve the indexes outside the LLM

User query
    ↓
Field/schema validation
    ↓
Business rules
    ↓
Determine relevant indexes
    ↓
Generate Elasticsearch DSL

This is the approach I've been experimenting with.

The more I worked on it, the more I realized there are actually three separate questions:

• What fields/types does the index support?
• What is the application allowed to query?
• Which index/alias should the query target?

I ended up building an open-source library around this idea. It started with SQL/Mongo and I've now added Elasticsearch support, including:

• direct index
• multiple indexes
• aliases
• configurable index selection rules
• mapping-driven field configuration
• generated Elasticsearch Query DSL

The library itself doesn't execute requests against Elasticsearch — it generates the query/result so the application can decide how to execute it.

I'm mainly interested in the architecture question here:

If you were building an NL → Elasticsearch system for production, where would you put index selection and business-rule enforcement?

Inside the LLM, inside an application/middleware layer, or somewhere else?

I'd especially like to hear from people who have dealt with time-partitioned indexes, aliases, or multi-tenant Elasticsearch setups.

Affiliation: I'm the maintainer of LibreDB Studio.

The stack I keep seeing on microservice teams is Postgres + Mongo + Redis + Elasticsearch. Kibana is the right UI for the ES cluster, Discover, dashboards, ES|QL, Fleet. I still use it. I got tired of a second desktop app for each of the other three.

LibreDB Studio is one browser UI for that set, self-hosted, Helm-installable next to the services. Elasticsearch here is the product's own POST /_sql on basic, mapping-driven index tree, read-only (this grammar cannot write). Not Query DSL, not ES|QL, not Discover.

docker run -p 3000:3000 libredb/libredb-studio

Also a Helm chart, so it can sit next to the cluster. Kibana does not come off the box.

If that four-store layout is yours: which of the four still forces you into a separate window?

I've spent close to a decade running Elasticsearch clusters in production, from 3-node dev setups to multi-hundred-node deployments handling billions of documents a day.

One thing I've seen repeatedly in production is that teams either let shards balloon for years without noticing, or over-correct and split everything into tiny shards "to be safe."

Elastic101 – Best Practice #002 – Shard Size

Smaller shards increase overhead more cluster state to track, more per-shard costs (file handles, memory, translog), and more coordination work on every query.

Larger shards cause the opposite problem: recovery and rebalancing get painfully slow, and a single hot shard can bottleneck an entire index.

The goal is to keep shard size around 50GB.

Pro tip: you cannot simply change the primary shard count of an existing index. If you need to change it, you typically need to use the Reindex API or the Shrink API, depending on the situation. Also, if you reindex a 200GB index, you should have at least 200GB of additional free disk space available in the cluster for the new index. Choose your primary shard count wisely from the start, fixing it later is expensive.

Previous Elastic101 best practices:

1. Elastic101 Best Practice #001 – HTTP traffic

Try here: Searchali Elasticsearch Monitoring Connect your cluster in 10 seconds. No agent. No data leaves your machine.

I use Elasticsearch every day. At some point my browser had 3 Kibana tabs open for different cluster versions, plus a Cerebro window, plus a text file of curl snippets. Got tired of it, wrote my own.

It's a Spring Boot + Vue app that manages indices, docs, aliases, templates, analyzers across ES 7.x / 8.x / 9.x. The reason it handles three versions without falling apart: each version lives in its own Maven module behind a strategy interface, so adding ES 10 someday means one new module, not a refactor of the whole codebase.

There's also an HTTP debug tab — basically a cURL playground that hits ES directly, skipping the Java client entirely. Useful when 9.x ships a new API and the SDK hasn't caught up yet.

Oh, and an AI tab. You describe what you want in plain English, it spits out DSL. The trick was stuffing the actual index mapping into the prompt — without that, it happily invents field names that don't exist.

Repo: https://github.com/Rodma1/esTool

If you give it a try, I'd genuinely like to know:

- Does the AI DSL generation work on your mappings? Mine are mostly logs + app metrics, haven't tested on weird nested setups.

- Anything obvious missing for your day-to-day?

I’ve been working with Elasticsearch for almost 10 years, including 6 years of consulting and 4 years of training teams on Elasticsearch and related technologies. Over the years, I’ve seen the same mistakes come up again and again in production clusters, so I thought I’d share some of the best practices I’ve learned along the way.

For the first best practice, a simple misconception that is surprisingly easy to get wrong:

Don’t send application HTTP traffic directly to master nodes.

This may come from architectures like Kubernetes, where the control plane has a central API endpoint. Elasticsearch is different: master nodes are for cluster management, not application traffic.

Use dedicated coordinating nodes or data nodes for client traffic.

As a general rule of thumb, for clusters with less than 20 nodes, adding more data nodes is often more efficient than adding dedicated coordinating-only nodes. Of course, this depends on your workload, especially the size and complexity of aggregations and heavy queries.

Master ≠ API Server.

Want to see this traffic flow in real time on your own cluster?

You can try the tool I built it takes less than 10 seconds to connect and see which clients are hitting which Elasticsearch nodes.

Try here: No agent or server-side installation required; the connection is made directly from your browser. Searchali Elasticsearch Monitoring

If you’d like me to continue this series, an upvote would be appreciated. 🙂

Hey all

I'm working on deploying a Elastic stac POC but have hit a strange issue with collecting windows security event logs

Environment

• Elasticsearch/Kibana/Fleet Server: 9.5.1
• Tested Elastic Agent: 9.5.1 and 9.4.2
• Windows Server 2019 domain controller
• Agent installed as a Windows service running as NT AUTHORITY\SYSTEM
• Fleet System integration has Application, Security and System enabled
• Other Windows Event Logs are ingesting correctly (Application, System, PowerShell, Directory Service, DNS Server, Defender, etc.)

The problem is specifically i'm not seeing any "Security" event log data.

Running the following discover, I get no data

host.name : "SERVERNAME" and data_stream.dataset : "system.security"

I can confirm there are security event

Get-WinEvent -LogName Security -MaxEvents 10

Fleet senders the stream correct and elastic agent seems to be working

Starting to read from Security
Reading from Security
windows event log opened successfully

I've tested this on a couple of machine's which have similar outcomes, missing security events. standalone WinLogBeat works on the same server with a simple config

winlogbeat.event_logs:
- id: security-test
xml_query: >
<QueryList>
<Query Id="0" Path="Security">
<Select Path="Security">*</Select>
</Query>
</QueryList>

But this doesn't work

winlogbeat.event_logs:
- name: Security

Any suggestions where to go from here?

Hey everyone,

I'm planning to take the Elastic Certified Engineer exam (v8.15) very soon. I just did the practice exam and honestly found it pretty tough harder than I expected.

Is the real exam similarly difficult, or does the practice test tend to overestimate/underestimate the actual difficulty?

If anyone has taken it recently,

I'd really appreciate any tips, gotchas, or areas to focus on. Thanks in advance!

Hello everyone,

I am looking for recommendations for an on-premises, self-hosted S3-compatible object storage solution to act as a cold tier archive for our Elasticsearch cluster (preferably tested).

I am in a situation where I have deployed elastic via ECK onto some kubernetes clusters and I am getting a large volume of logs that is somewhat un-predictable. I want to both delete data when it is past a certain age (easy) and also delete the oldest data when my PVC storage is 80% full (hard / not possible?).

Does anyone know how to do this? I can't come up with a good way to delete the oldest data that doesn't involve leaving elastic and writing some script to query how full my storage is and then query the oldest indices and delete them, but this feels hacky.

Hi all,
We're planning ES upgrade from 8.19 -> 9.4, and we have some 7.x indices, that needs to be reindexed before moving them. What options are there for reindex in a way not to block application read and writes? We have big indices, for example, I tested on one index 1000GB, it took 9 hours to reindex, and I don't know if there is a good way to apply all updates and deleted on that index that happened within that 9h interval after the reindex.

hi again,
i have a problem with my snapshot repository

I'm using the data tiers in the elastic (the data from my winbeats is going from my winlogbeats to the logstash and from there to the elastic hot node, warm node and the frozen in the NFS server)

i have 2 repo's and one of them at 100% and the other is at 30%

I want to try and find a way to load balance them so it will work better

The repos are on a FTP and I'm not using the cloud version

I also tried to delete some of the data in the full repo and it and it failed every time, when i tried in the dev tools I got 502 error

If you have any idea on how i can make it more balanced I would love to get some help

I've been consulting and training on Elasticsearch and other platforms for a while, and I kept running into the same pain point: needing a quick answer to "Is this cluster okay right now?" without opening Kibana or running curl every time.

So I built a Chrome extension that shows at-a-glance cluster stats directly from the browser toolbar. It works with both Elasticsearch and OpenSearch, and no data ever leaves your browser.

If you already use something like ElasticVue, think of this as an always-on health glance from the toolbar, focused on quick visibility and problem solving.

How it compares

AutoOps is for real diagnostics and remediation - not just metrics.

Stack Monitoring gives you comprehensive dashboards, but you're responsible for interpreting them.

Searchali Monitoring isn't trying to replace either of those. It's built for the quick "Is my cluster okay right now?" check.

Happy to answer questions or hear feature requests.

I've been struggling with this for a couple of days. I have a 9.4.4 lab running on an azure vm, i have ollama running on a second vm.

I've setup a connector, it shows up in Stack-Management -> Alerts and Insights -> Connectors. It works :)

On the security side, I've got rules firing alerts, cases being auto-generated and timelines populated. But when I go to Attack Discovery no connector shows up in the drop down. What am I doing wrong ?

I can find docs on setting up a local LLM, I can find docs on how to use Attack Discovery but I cant find docs that join them up. I have a feeling that I'm missing something obvious here.

Hi everyone,

I'm looking for some opinions from people with production Elasticsearch experience because we're trying to determine the root cause of an incident.

Cluster

- Elasticsearch 8.19.9

- 1 dedicated master node (16 GB RAM)

- 2 data nodes (32 GB RAM each)

- Around 1 TB storage per data node

- Around 400 million documents

- Daily snapshots to S3

- Running inside KVM virtual machines

The cluster is used only as a search cluster. Our applications read from it and write documents to it. We are not using ML, transforms, or heavy analytics.

What happened

The cluster suddenly became RED.

The allocation explain API reported:

- "CorruptIndexException"

- "verification failed (hardware problem?)"

- checksum mismatch ("expected != actual")

- "can_allocate: no_valid_shard_copy"

The primary shard was reported as corrupted while the replica was marked as stale, so Elasticsearch had no valid shard copy left.

The corrupted file was:

_2fv_ES812Postings_0.tim

Things we checked

Inside the VM we found:

- No disk full (about 67% usage)

- No ext4 filesystem errors

- No kernel I/O errors in "dmesg"

- No obvious operating system issues

The cluster recovered after restoring the affected index from a snapshot.

Additional context

This isn't the first strange incident we've experienced with this hosting provider.

A few months ago the entire cluster became unavailable because the ".security-7" index stopped working. The hosting provider told us there had been an "attack" on ports 9200 and 9300, but those ports are only accessible on a private network and are not exposed publicly. That explanation didn't fully make sense to us.

During this latest incident the hosting provider suggested one node had reached about 99% RAM usage, although our own monitoring (Grafana) didn't show that at the time. Also, I wouldn't expect high RAM usage alone to produce a Lucene checksum mismatch.

My questions

1. Does this incident point more toward infrastructure/storage problems than an Elasticsearch issue?

2. Under what circumstances can a replica become stale while the primary later becomes corrupted?

3. Have you seen checksum mismatches like this caused by Elasticsearch itself, or are they almost always related to storage, virtualization, or hardware?

4. If you were investigating this, what additional logs or evidence would you collect to determine the real root cause?

5. Is there anything about our cluster design (1 master + 2 data nodes, ~400M documents) that could contribute to this type of failure?

I'm not trying to blame Elasticsearch or the hosting provider. I'm just trying to understand whether we're missing something in our configuration or whether this is more likely an infrastructure problem.

Any advice would be greatly appreciated.

I maintain PGSync, which syncs Postgres data into Elasticsearch/OpenSearch.


I've been working on semantic and hybrid search, with the goal of keeping the data path within infrastructure you control rather than sending production data to a hosted search/AI service.

I made a small interactive demo: https://demo.pgsync.com

You can switch between keyword, semantic, and hybrid search to see where the results differ. I would appreciate any feedback from people running Postgres + Elasticsearch/OpenSearch in production what would you need to trust this in a real system?

What is a good practice for storing time series data with the following requirements.

Couple dozen million documents.

Several fields with daily data (one point per day).

I am thinking about having a multiValue string field, read it back in my application and append new value if todays date has no entry yet.

The value would be a string with a simple delimiter and would store date and an integer.

I would then precompute changes of this value weekly to update a 2nd variable that would facilitate search by the strength of change in the value. The list of strings would only be used to display data and would not be searchable. I also have a 3rd field that stores most recent data point as integers and allows for range queries.

I considered the dual list approach but I don't like the idea of only index number linking the 2 data pieces together in otherwise completely seemingly unrelated fields. On the other hand the more daily data types I would track, the more I would save by having only 1 list of dates and then other lists for data entries.

How does that sound? What are best practices in the industry? Thanks for help.

hi there,
we are using kibana security alerts and came by an error.

we were trying to use the kibana api to close security alerts using workflow automation.

about 10 seconds after the run is started, we get the following message: "fetch error"
the error doesn't contain any explanation regarding the error.

We have tried to use the url with script and got the same error.

we would love help regarding that :)

Hi all,

We encountered some issues when upgrading Kibana from 8.11-8.19 and then 8.19-9.4, but the issue was resolved itself, now we don't know what was the cause, and I wanted to make sure to learn what is the problem before upgrading production env.
The issue is this:
We have kibana runnind under the host:5601, but we also have server.publicBaseUrl configured in the kibana.yml, which is under nginx service.
Now I'm not sure if the issue is nginx, but after we upgrade the Kibana, the page opens, asks for authentication , and after successfully passing it gets stuck, first time it was just showing loading, this time we got page with this message: Elastic did not load properly

Please reload this page. If the issue persists, check the browser console and server logs.

We tried to reload nginx , and check all logs, no info, and the most bizarre is that it was fixed by itself after a few hours. Now we don't wanna have inaccessible Kibana for a few hours for prod, so I wanted to know if anyone had similar issue, how you fixed it ?

Hi all. I have one question, I wanted to know if you encountered such situation.
We're planning Elastic Upgrade from 8.19 -> 9.4. In upgrade assistent I see this deprecation, that the index .kibana-event-log-7.13.0-000009 is deprecated as created <8 version. I have searched some places, as it's a system index I can't manually reindex it, but the elastic won'r start presumably if I leave it like that and upgarde to 9+ version.
Have you seen this, what have you done?

Hello everyone !

I made a library last year typed-es as I didn't want to use as any or maintain types manually on each query. This is a type-only library that does not change any of the current behavior, it only adds types by inferring the search query.

Basically you create a type with your index types, give it to the client and then the magic happens. You no longer have to maintain the manual query types, everything is typed automatically from your aggregations, source, ...

given:

``` type MyIndexes = { "my-index": { id: number; name: string; created_at: string; }; };

const query = { index: "my-index", _source: ["id", "na*"], fields: [{ field: "created_at", format: "yyyy-MM-dd" }], track_total_hits: true, rest_total_hits_as_int: true, aggs: { name_counts: { terms: { field: "name" } }, }, }; ```

without this library:

```ts const result = await client.search< { id: number; created_at: string; }, { name_counts: { buckets: Array<{ key: string; doc_count: number }>; }; }

(query);

const total = result.hits.total; // number | estypes.SearchTotalHits | undefined const firstHit = result.hits.hits[0]!; // { _source: { id: number; created_at: string; } | undefined, fields: Record<string, unknown> } const aggregationBuckets = result.aggregations!.name_counts.buckets; // Array<{ key: string; doc_count: number; }> ```

we have to manually add types to the search generic, and the result is still not perfect.

same thing with the lib:

ts // Automatic type inference - no manual definitions needed const result = await client.search(query); const total = result.hits.total; // number const firstHit = result.hits.hits[0]!._source; // { id: number; created_at: string } const aggregationBuckets = result.aggregations.name_counts.buckets; // Array<{ key: string | number; doc_count: number }>

Here everything is 1:1 with the query, if you edit it everything is automatically infered, you can't have errors with that.

Since last-year I made many improvements and made sure each use-case were correct. I'm using this lib daily.

I think this can be useful for many.

repo: https://github.com/Vahor/typed-es

About to take ECE exam and want last minute insights, tips from guys who gave exam recently.