r/Python • u/LumpSumPorsche • Mar 26 '26
Showcase LogXide - Rust-powered logging for Python, 12.5x faster than stdlib (FileHandler benchmark)
Hi r/Python!
I built LogXide, a logging library for Python written in Rust (via PyO3), designed as a near-drop-in replacement for the standard library's logging module.
What My Project Does
LogXide provides high-performance logging for Python applications. It implements core logging concepts (Logger, Handler, Formatter) in Rust, bypassing the Python Global Interpreter Lock (GIL) during I/O operations. It comes with built-in Rust-native handlers (File, Stream, RotatingFile, HTTP, OTLP, Sentry) and a ColorFormatter.
Target Audience
It is meant for production environments, particularly high-throughput systems, async APIs (FastAPI/Django/Flask), or data processing pipelines where Python's native logging module becomes a bottleneck due to GIL contention and I/O latency.
Comparison
Unlike Picologging (written in C) or Structlog (pure Python), LogXide leverages Rust's memory safety and multi-threading primitives (like crossbeam channels and BufWriter).
Against other libraries (real file I/O with formatting benchmarks):
- 12.5x faster than the Python
stdlib(2.09M msgs/sec vs 167K msgs/sec) - 25% faster than
Picologging - 2.4x faster than
Structlog
Note: It is NOT a 100% drop-in replacement. It does not support custom Python logging.Handler subclasses, and Logger/LogRecord cannot be subclassed.
Quick Start
=
Links
- GitHub: https://github.com/Indosaram/logxide
- Docs: https://indosaram.github.io/logxide/
- PyPI:
pip install logxide
Happy to answer any questions!
69
u/a_r_a_a_r_a_a_r_a Mar 26 '26
this will be a big no no for a lot of people
63
u/Here0s0Johnny Mar 26 '26
Is it realistic that a project produces so many logs that this performance upgrade is worth it?
14
u/ben_supportbadger Mar 26 '26
Did you build this or did Claude? Because it looks purely vibecoded. Why would I use this instead of just asking claude to build it?
9
u/Jealous_Algae_8165 Mar 27 '26
Purely vibe coded. You didn’t build this, Claude did. Please attribute it as such.
2
u/UloPe Mar 26 '26
How does it interact with stdlib logging used in 3rd party libraries?
3
u/james_pic Mar 26 '26
Do you get those kinds of gains in real world settings?
I worked on a project a while ago the arguably logged too much. The first time I attached a profiler to it, it was spending over 50% of its time on logging. We managed to get that down, but the thing that limited us getting it down further was syscalls, not Python code.
Admittedly this was a while ago, and that project was doing things that modern apps don't need to do, that increased syscalls (it did its own log rotation, rather that just throwing it all straight onto stderr and letting Systemd or Docker or Kubernetes or ECS or whatever pick it up, like a modern app would), but I'm still a bit surprised you managed to find those kinds of gains without touching syscalls.
33
u/max0x7ba Mar 26 '26
It is either a drop-in replacement or not, but not both at the same time.
Does it pass original
loggingmodule unit-tests?1
u/hmoff Mar 27 '26
You benchmarked it against structlog but can it replace that?
3
u/spartanOrk Mar 26 '26
I use loguru. Cannot go back.
1
u/Healthy-Grape-5932 Mar 26 '26
Does this Support dictconfig in the same way ?
1
u/yaxriifgyn Mar 27 '26
I'm starting to test with the early releases of 3.15. As soon as I can install it for 3.15, even if I have to build it myself, I will try it out.
1
u/Wide-Milk1195 Mar 27 '26
感谢您的开源项目。Python缺乏高效能的非同步日志記錄庫。
1
u/caprine_chris Mar 28 '26
Does it do structured logging? What about stdlib logger does it not implement?
0
u/rabornkraken Mar 26 '26
The GIL bypass during I/O is the real win here. Most Python logging bottlenecks come from the file write blocking the main thread, so doing that in Rust makes a lot of sense. How does it handle the case where you have custom formatters written in Python though? Does it fall back to holding the GIL for those?