Repository navigation
Pino: in-process LogtailStream for serverless handlers - #168
Merged
Merged
Conversation
Pins a Pino destination that logs through a Logtail client the application owns, with the same level, custom level and messageKey mapping as the transport, synchronous enough for `await logtail.flush()` to cover it, and flushing when the stream ends. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A Writable Pino destination logging through a @logtail/node client the application owns, so `await logtail.flush()` covers everything logged before a serverless handler returns. The line mapping (levels, custom level names, messageKey, message_field, time) moves to a shared helper used by both the worker transport and the stream; the stream picks up Pino's configuration from the message Pino emits on its destination. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
PetrHeinz
marked this pull request as ready for review
September 30, 2026 15:59
Pins that a custom level method such as logger.notice() still gets context.runtime pointing at its caller, that a logger with `timestamp: false` delivers its logs through both the stream and the worker transport, and that dt is the time Pino wrote, for Pino's default epoch milliseconds as well as ISO strings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pino's level methods, custom ones included, are all its function named LOG, so the stack hint now matches that frame too and a call such as logger.notice() gets context.runtime pointing at its caller. The time mapping read obj.time.length, which threw when Pino writes no time (`timestamp: false`): through the stream the error reached the caller of the log method, and it killed the transport worker silently. It also skipped Pino's default epoch milliseconds, so dt was the time the log was processed. Any time Pino wrote now goes through new Date(), and dt is set when that is a valid date, for the stream and the worker transport alike. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gger.flush, sends in flight - A failed send through LogtailStream with throwExceptions must not be an unhandled rejection - Epoch seconds (pino.stdTimeFunctions.unixTime) become dt, in the stream and the worker transport - A line that is not JSON is skipped and the stream keeps logging - logger.flush(callback) flushes the Logtail client behind the stream - Node flush() waits for a send the batcher already started; core tracks sends until they settle Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…poch seconds - LogtailStream catches the rejection of a failed send under throwExceptions instead of leaving it unhandled, which killed the process (every Lambda invocation failed with Runtime.UnhandledPromiseRejection). The failure stays counted in logtail.dropped; it is not printed with console.error, because replaceConsoleMethods() on the same client would log it once more. - A line that is not JSON is skipped instead of throwing in _write, which left the stream wedged with every later log buffered. - LogtailStream.flush(callback) flushes the Logtail client, so logger.flush(callback) waits for the logs. - A numeric time below 1e11 is epoch seconds (pino.stdTimeFunctions.unixTime) and multiplied by 1000, in the stream and the worker transport; such values as milliseconds would be before March 1973. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Core keeps the sends the batcher started in a set until they settle (both outcomes handled, so no unhandled rejection). Node's flush() snapshots them, flushes, then waits for the snapshot with failures ignored, as the batcher retries or drops a failed batch itself. A serverless handler that awaits logtail.flush() is therefore not frozen while a batch sent by batchInterval or batchSize is in flight. Browser and Edge flush() are unchanged; tools is untouched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A send already in flight when flush() is called fails; its logs go back to the batcher for a retry on the batch timer, and flush() must not resolve before they were sent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a send that was already in flight fails, the batcher puts its logs back into the buffer for a retry on the batch timer, so flush() resolved while they were still unsent and a process exiting right after it lost them. flush() now notes which of the sends it waited for failed and, if any did, flushes once more: that sends the returned logs right away, and a failure of that send goes through the batcher's usual retries before flush() resolves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # packages/core/src/base.test.ts # packages/core/src/base.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A Pino transport runs in Pino's worker thread, so a serverless handler cannot wait for its logs.
pino.flush()only moves the main thread's buffer into the worker, and the worker's batch and HTTP request freeze with the function (#112).@logtail/pinonow also exportsLogtailStream, an in-process Pino destination that logs through a@logtail/nodeclient the application owns:messageKey,message_field, Pino'stime), in a helper shared by both. Pino 8.21+ shares its level names and message key with the stream through its configuration message; thecustomLevelsoption covers older Pino._write, so a flush right after a log call covers that log.logger.flush()calls the stream'sflush(callback), which flushes the client. Ending the stream flushes it too.context.runtimepoints at the caller, including for custom-level methods.logtail.dropped. WiththrowExceptionsthere is no caller to throw to, and printing the error would get it forwarded again byreplaceConsoleMethods().Two fixes in the shared mapping apply to the worker transport too:
pino({ timestamp: false })no longer throws; it used to kill the worker silently.dtis the time Pino logged, also for its default epoch milliseconds; before, it was the processing time. A numerictimebelow 1e11 is read as epoch seconds (pino.stdTimeFunctions.unixTime).@logtail/node'sflush()now also waits for sends already in flight, such as a batch the 1 s timer had already started, so everything logged before the call has been sent when it resolves. If one of those sends fails, it flushes once more, so the logs the batcher put back go out before it resolves. Core keeps the started sends until they settle. Browser and Edge flush as before.This is patch-level:
flush(callback).flush()resolves later only while an earlier send is still running or being resent, which is bounded bytimeoutandretryCount.Verified against Better Stack on Node 20, 22 and 26 and Pino 8.16 to 10.3, with a harness that freezes the process the moment the handler resolves, as Lambda does:
await logtail.flush(): 16 of 16 lines delivered before the freeze, also with a 1.5 s handler whose timer batch is still in flight. The worker transport delivers 2 of 16, with or withoutlogger.flush().logger.flush(): 16 of 16.throwExceptionswith a revoked token: the process keeps running; before, it died with an unhandled rejection.Each fix commit is preceded by a tests-only commit that is expected to fail CI.
Docs: the Pino page needs a "Serverless" section showing the stream and
await logtail.flush().🤖 Generated with Claude Code