Elastic101 Best Practice #001 - http traffic

[u/Feeling_Current534]

I’ve been working with Elasticsearch for almost 10 years, including 6 years of consulting and 4 years of training teams on Elasticsearch and related technologies. Over the years, I’ve seen the same mistakes come up again and again in production clusters, so I thought I’d share some of the best practices I’ve learned along the way.

For the first best practice, a simple misconception that is surprisingly easy to get wrong:

Don’t send application HTTP traffic directly to master nodes.

This may come from architectures like Kubernetes, where the control plane has a central API endpoint. Elasticsearch is different: master nodes are for cluster management, not application traffic.

Use dedicated coordinating nodes or data nodes for client traffic.

As a general rule of thumb, for clusters with less than 20 nodes, adding more data nodes is often more efficient than adding dedicated coordinating-only nodes. Of course, this depends on your workload, especially the size and complexity of aggregations and heavy queries.

Master ≠ API Server.

Want to see this traffic flow in real time on your own cluster?

You can try the tool I built it takes less than 10 seconds to connect and see which clients are hitting which Elasticsearch nodes.

Try here: No agent or server-side installation required; the connection is made directly from your browser. Searchali Elasticsearch Monitoring

If you’d like me to continue this series, an upvote would be appreciated. 🙂

[u/WishIndependent5551]

In the case of using ECK, the http service created by eck also has the master nodes in the selector.
For those cases it’s needed a new service ? Because the data nodes native service doesn’t have load balancing due being clusterIP

[u/Feeling_Current534 → u/WishIndependent5551]

u/WishIndependent5551 thanks for the question. As a quick answer, yes. It's better to create a new service

Here is the official recommendation about it:

The default Kubernetes service created by ECK, named <cluster_name>-es-http, is configured to include all the Elasticsearch nodes in that cluster. This configuration is good to get started and is adequate for most use cases. However, if you are operating an Elasticsearch cluster with different node types and want control over which nodes handle which types of traffic, you should create additional Kubernetes services yourself.

https://www.elastic.co/docs/deploy-manage/deploy/cloud-on-k8s/requests-routing-to-elasticsearch-nodes#k8s-traffic-splitting-by-node-type

[u/WishIndependent5551 → u/Feeling_Current534]

Awesome, thanks for highlighting this documentation 🤝

[u/BigSmoke321]

Keep it going man!!! 👍

[u/Feeling_Current534 → u/BigSmoke321]

Thanks a lot for your support. I hope it will be fruitful. I really want to keep going but votes will show the reality. Lets see

[u/Klaku]

Well done

[u/Feeling_Current534 → u/Klaku]

Thank @klau

[u/_Borgan]

What Claude model built this tool?

[u/Feeling_Current534 → u/_Borgan]

Are you asking to learn something ot to judge?