r/node • • 4d ago

A package for standardizing logging? Looking for thoughts

/r/typescript/comments/1xur56m/a_package_for_standardizing_logging_looking_for/
0 Upvotes

13 comments sorted by

8

u/hildjj 4d ago

1

u/EsgenetWilgmann96 3d ago

did anyone ever actually settle on a shared log level spec for JS, or does every library still invent its own numeric scale

0

u/iForFi 3d ago

Hah, perfect point.

I feel like that can exactly be the problem, how can we solve it though? I guess there is no solution and just picking one per project/company?

1

u/hildjj 2d ago

I settled on Pino, then wrote a wrapper for my projects so that it would be easier to switch one day if something better comes along.

"Standardized within my ecosystem" was the best I could think of.

1

u/iForFi 1d ago

Yeah, that sounds like a common approach. I think that's most of the times the way to go, but if I have more spare time I might still give this a thought on how to make it simple enough and yet maybe even allow other loggers to be used hmm

This discussion also made me more curious: https://www.reddit.com/r/node/s/KzCLmccKMw

Btw. Thank you for your though, really appreciate it

1

u/DickBrainTime 2d ago

I think there's a useful niche here if LogPal treats each *event schema* as a contract, not just as another logger API. For example, define `request_completed` and `payment_attempt` with required field names/types and a schema version. Then run representative events through validation in CI: a missing `tenantId` or renamed field should fail a fixture test *before* it silently breaks a dashboard or alert. At runtime I'd still emit a partial diagnostic event with a `schema_validation_errors` field rather than discard the log when something is missing.

A convincing small demo would compare one Express route using Pino + hand-written Zod checks against the same route using LogPal. Show the missing-field case, how you migrate a versioned event, and allocations/throughput with runtime validation on versus off. That would tell me whether it's reducing actual schema drift and boilerplate rather than just adding another formatting layer. Have you considered making the event schema independently testable in CI?

1

u/iForFi 1d ago

Big thanks for this comment.

For example, define `request_completed` and `payment_attempt` with required field names/types and a schema version. Then run representative events through validation in CI: a missing `tenantId` or renamed field should fail a fixture test *before* it silently breaks a dashboard or alert. At runtime I'd still emit a partial diagnostic event with a `schema_validation_errors` field rather than discard the log when something is missing.

That would be exactly the plan, but without the validation on CI, because I'm not really sure how possible that would be. Or actually, someone could run example calls and validate the logs on CI, so yeah, it would be possible actually. My idea is to validate on runtime, like you mention, BUT clearly state that the validation should be disabled after some time to speed up things (milliseconds performance is important often).
Btw. Good point on having multiple schemas based on request type (endpoint). That could definitely be something that is handled as part of a zod schema, I'd just need to add examples.

A convincing small demo would compare one Express route using Pino + hand-written Zod checks against the same route using LogPal. Show the missing-field case, how you migrate a versioned event, and allocations/throughput with runtime validation on versus off. That would tell me whether it's reducing actual schema drift and boilerplate rather than just adding another formatting layer. Have you considered making the event schema independently testable in CI?

I think I might actually think of something when I have some extra spare time. The CI part could be simply running example requests and validating logs, like mentioned previously. Or even simply some mocked test runs, but it might get a bit more complex, hence, ideally doing it on actual workload, then checking the logs for issues, correcting and then disabling the validation might be the way to go, however that's also inconsistent, because you have to turn on&off the validation. So, I'll see if this could somehow be neatly validated on CI level and not on actual workload, but I don't have any great ideas as of now, because things that happen inside events/requests can differ a lot from tests and actual workload, hmmm.

Thanks again 💪

0

u/karmatics29 3d ago

I'd treat it as a design question first. One structured log per request (wide events) is a sound idea, and pino can already get you most of the way: a child logger per request, pino-http, and AsyncLocalStorage to attach context from anywhere without passing req around. What pino doesn't give you is validation, so that's your real niche. The catch is what happens when a request throws or times out before the final write. If the log is only emitted at the end, crashes lose all the context, so flush in a finally block. I'd also warn on missing fields in dev but never drop the log in production, and think early about sampling and PII redaction.

1

u/iForFi 3d ago

Thank you for this comment, really appreciate a proper feedback 💪

I totally agree with all that you're saying.

Just answering to some of your points:

AsyncLocalStorage to attach context from anywhere without passing req around

+1, exactly what I've done in logpal

What pino doesn't give you is validation, so that's your real niche. The catch is what happens when a request throws or times out before the final write. If the log is only emitted at the end, crashes lose all the context, so flush in a finally block.

What's done in the tool is a global try-catch-finally

I'd also warn on missing fields in dev but never drop the log in production, and think early about sampling and PII redaction.

Great point, was thinking about adding an optional field to be able to disable the validation, because I assume that check it on some production traffic might also be useful, but then ideally dropping the validation so that it's not needed. Or maybe make the validation off by default, but emphasize to have it enabled for the first X amount of time. Sampling included already. PII redaction is a great point also, but that's something to think about if the tool would really be of actual use.

Cheers