Repository navigation
Structured log package in node core #49296
Description
Activity
Hi,
Can I work on thisYou can work on this, but I must give a disclaimer: this kind of feature will probably involve a lot of "soft" work in terms of achieving consensus both on its necessity and then on the implementation.
if you are looking for a smooth entry to this project I would choose an issue with less complexity that also has a consensus about how it should be handledYou can work on this, but I must give a disclaimer: this kind of feature will probably involve a lot of "soft" work in terms of achieving consensus both on its necessity and then on the implementation.
if you are looking for a smooth entry to this project I would choose an issue with less complexity that also has a consensus about how it should be handledyeah sure,
please give me an issue with less complexityThere has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.
For more information on how the project manages feature requests, please consult the feature request management document.
There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.
For more information on how the project manages feature requests, please consult the feature request management document.
Reopening ref to https://x.com/vinii_joga10/status/1773674350102659178?s=20
As soon I get some time, I will research how the other langs are handling logs, and I will try to bring some API suggestions on how we can introduce logging at Node.
I think if we create a good abstraction for logging, we can then bring the pino implementation to have some default features, like
structured logging,redact, and other features./cc @mcollina to bring more ideas on this topic since you know a lot about logging & pino.
Reacted by Toni Villena, Igor Savin, Debadree Chatterjee and Maksym ShenderukReacted by Yves M.I will research how the other langs are handling logs
Here's a 2023 article of the official Go blog about this : https://go.dev/blog/slog
I think we could bring 80% of the pino infrastructure down to core. pino is composed of:
- public api
- off thread logging processing (especially useful for network destinstions and heavy processing).
- main thread log redaction
- opinionated log format
- re-do of createWritableStream to maximize speed and handling of overflow situations (disk is busy, what do you want to do?)
It'd need to loose some of the opinions and add some overhead to be more generic, but that can be balanced.
Reacted by Igor Savin, Debadree Chatterjee, Vinicius Lourenço, Toni Villena, Aviv Keller, Demian Parkhomenko and Jurj Andrei GeorgeReacted by Yves M.51 remaining items
I don't like
info({ merging: 'object' }, 'message'), and I preferinfo('message', { merging: 'object' }).I thought Raynos did a good job arguing for message first
info('hi', {foo:'bar'})over merging-object first above. The reason "merging-object first" exists is because I wanted to support varargs-style in Bunyan at the time, to be closer toconsole.log(). Since then, (a) JS grew string templates for those that really want a varying message and (b) personally I've come to prefer a static string key for the msg (as Raynos was arguing for).Reacted by James SumnersEither of
info({ merging: 'object' }, 'message')orinfo('message', { merging: 'object' })will need to then merge the object and the message together somehow anyway, so it makes the most sense to me to just doinfo({ message: 'message', other: ... }).We can safely support two arguments, ideally with the string first.
We would need to have an internal data structure anyway:
- time (maybe this can actually be avoided, which would be best)
- level
- message
- merging
So that you know, I recommend avoiding merging at the first step and delegating this to transports / formatters. We would lose some perf compared to pino, but it would allow for a clearer standardization.
Reacted by Mert Can Altin, James Sumners, Toni Villena and Yves M.hello, thanks again for details, I sended some commits in today, first look I try implemented string first and Two arguments : https://github.com/nodejs/node/pull/60468/files
than, I try @Qard suggestion (#60468 (comment)): e149af2
and first performance results:
#60468 (comment)I don't really follow what you're proposing. My understanding is that your proposal would only provide logging from within core itself.
No, it would be the internals of the described public API.
import { createLogger, JSONHandler } from 'node:log'
// Construct a JSON output handler, attaching diagnostics_channel subscribers
// to whichever levelled channels it cares about
const handler = new JSONHandler({ level: ... })
handler.capture()const logger = createLogger()
logger.info({ msg: 'hello' }) // dispatches to a "log:info" diagnostics_channel internally
I have been thinking about this and I'm unsure about the viability when thinking about destinations. There are a few people that want to stream to different destinations based upon log level, and this would certainly make that possible, but the typical case is streaming everything to a single destination with interleaved levels. Which leads to ordering: I'm not clear if the proposed approach would be able to guarantee ordering of logs. People really don't like it when Log Line B is written before Log Line A.
Yes it would allow supporting the ordering of logs.
The question on using diagnostic_channels for this is how can we support a destination only receiving logs from a given logger.
This might not be a use case we want to support. If we don't, having a
createLogger()might be redundant.If we give each logger a unique name, we could map to channels like
log:info:grandparent:parent:childor something like that. Easy enough to support listening to specific loggers if that's a thing we want. We can also have a discovery channel likelog:newto be notified any time a new logger is constructed to decide if the channel is worth subscribing to.Reacted by Matteo Collina, Jurj Andrei George, Mert Can Altin and Vinicius LourençoThis issue has been marked as stale due to 210 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.It is actively being worked on. This is not stale.
Reacted by Mert Can Altin and Vinicius LourençoReacted by Guillaume HumbertIt is actively being worked on. This is not stale.
+1, for follow #60468
Reacted by Moshe BrevdaHi all, I'm a little late to the party here; apologies. I was wondering why closer integration with OpenTelemetry hadn't been explored further given how much the industry is converging towards it. This goes beyond the log event model and semantic field names, and ties into how logged events can be tied back to their relevant traces and spans. For instance, Deno's automatically tying logs back to spans if they occurred within one. I'm not suggesting this is the right mechanism for Node.js, just suggesting closer integration with OTel (even if through external libraries) might be worthwhile to explore.
Reacted by Guillaume Humbert, Tobbe Lundberg and Yoann MALLEMANCHE... I was wondering why closer integration with OpenTelemetry hadn't been explored further...
It has been really. In the prior PR it was discussed and there was a separate thread exploring OTel integration. I think it's far from settled.
In my new attempt the goal is to separate the concerns. The provider-based model with synchronous delivery makes it fairly easy to integrate. I think we need to first settle, however, on the question of whether Otel support should be baked into core at all. Once we settle on that, figuring out the integration points with things like a structured logging API become easier to work through.
What is the problem this feature will solve?
To have a structured log package available in stdlib so that all npm modules can log in a single fashion instead of
What is the feature you are proposing to solve the problem?
Implement something like https://go.dev/blog/slog
What alternatives have you considered?
No response