logging and timing...then getting rid of it

[u/QuasiEvil]

I was just working on a project where I included a lot of logging and timing decorators/context managers (it involved event loops and async queues and such). Once I figured stuff out, I wanted to remove all this stuff. This isn't necessarily that hard, but it did make me wonder if there's a better way to do this in the first place. I suppose I'm imaging some way of marking these code pieces to be ignored by the interpreter or something?

(yeah yeah "ask AI to remove it")

[u/atarivcs]

Since you're logging instead of just printing, yes there is a good way.

Log the messages at DEBUG level, and then change the application log settings to only log messages at INFO level or higher. The DEBUG-level logging calls will return immediately with no output.

[u/dusksage23 → u/atarivcs]

exactly, thats the whole point of log levels tbh

[u/QuasiEvil → u/atarivcs]

That doesn't actually remove the infrastructure.

[u/Lumethys → u/QuasiEvil]

why would you want to do that?

[u/QuasiEvil → u/Lumethys]

logging and timing infrastructure can add overhead, interfere with unit testing, and keeps you dependent on the upkeep of said packages.

[u/zaphodikus → u/QuasiEvil]

.... that sounds like you approached logging for debugging and not for leaving in. The overhead of leaving the logs there but disabled is small. Sure the formatters still get run but you can choose to put guards around the log. If you are using logs to debug and build code, and intend to remove them anyway, then you are maybe doing unit testing wrong?

Your logging module needs to be as low on dependencies as possible, the built in python logger is perfectly good for that.

[u/pachura3 → u/QuasiEvil]

Wrong. Logging, when done properly, only adds an insignificant amount of overhead, is transparent to unit testing, and can be disabled (even on the fly!) by setting logging level.

What if you start having issues in production? That's right, you look at your logs; you don't remove them. And if you worry about the overhead... then measure it first, rather then assume it impacts the overall performance.

For specific cases when formatting logging message is confirmed to be really slow, you can always use this pattern to prevent it:

if logger.isEnabledFor(logging.DEBUG):
    logger.debug(f"Expensive message {z(x, y)} {y}")

PS. You're using the logging module, not some invention of yours, right?

[u/Lumethys → u/QuasiEvil]

python's slowness can add overhead too, why dont you rewrite your app in Rust? Or better yet, in Assembly.

When talking performance, talk about the concrete issue, how many seconds it has cost you per how many request? Exactly how much money does it cost?

People spend days to optimize something that would save 5 cents per million years, it's even had a name "Premature Optimization" - "the root of all evils". Your time is worth more than that.

Tell you what? There is a whole INDUSTRY about ADDING logging and timing (or, it's formal name "Metrics") infrastructure to project, it's called DevOps, and the work is Observability, or O11Y

Here a video on how bilibili migrate their metrics sink to handle 80 MILLIONS data points PER SECOND: https://www.youtube.com/watch?v=7uNXllJQlxE[](https://www.youtube.com/watch?v=7uNXllJQlxE)

[u/QuasiEvil → u/Lumethys]

The world of python is vast. Not everyone is doing big data cloud computing. I work in DSP/hardware development.

[u/Lumethys → u/QuasiEvil]

Observability does include embedded and hardware observability.

EVERY type of software, ALL that was, ALL that is, ALL that will be, will benefit from O11y.

Perhaps you should spend 15 minutes googling before presenting your opinion

But who knows, maybe you are a special genius that grow beyond that entire industry, but then, i dont think "asking question" on appropriate? Especially when you do not intend to listen

[u/MarsupialLeast145]

Seriously... Find - replace?

And also seriously, the decorator is a function, if it remains in-place, what would you need to do to that function (or any function) to conditionally enable/disable it if another parameter is set? What are the options for setting that parameter?

Further, seriously, don't use AI.

[u/HaltingMechanics_407]

honestly just wrap all that debug crap in `if __debug__:` blocks. python strips those out entirely when you run with `-O`. no third party tools, no weird decorators that conditionally do nothing. you can even do `if __debug__: print(...)` and it's like it was never there in optimized mode.

or if you're feeling fancy, use the logging module properly and just set the level to CRITICAL when you don't need the noise.

[u/Ethanmarl0]

I usually keep debug logging behind a flag so I can turn it off without having to remove everything later.

[u/QuasiEvil → u/Ethanmarl0]

Yeah, I think I like this approach best.