Skip to content

Slog logger migration - #4258

Open
moskyb wants to merge 11 commits into
mainfrom
slog-logger-migration
Open

Slog logger migration#4258
moskyb wants to merge 11 commits into
mainfrom
slog-logger-migration

Conversation

@moskyb

@moskyb moskyb commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Description

Currently, the agent uses a custom leveled logger in the logger package. Generally, within Buildkite's various go projects, we're standardising on using the builtin slog package as our logger of choice.

Given that we're releasing a major version soon and log format is (regrettably) an API surface, we have a good opportunity to update the agent to use more standard tooling.

This PR removes the logger package, replacing it with slog and lmittmann/tint, with a couple (but not many) tweaks and changes to the defaults.

Also in this PR, I've updated many of the log messages to make better use of log attributes, rather than encoding them in prose. This makes JSON logs better.

Context

The logger package and its nonstandardness have been annoying me for a long time.

Changes

This PR contains the following breaking changes:

  • Log levels: The NOTICE and FATAL log levels have been removed, with the default log level now being INFO. All NOTICE logs have become INFO, and all FATAL have become ERROR (that then exit the application)
  • JSON output:
    • The key previously called ts is now called time
    • Log attributes will now keep their type (int, float, etc) instead of being stringified
    • Some attribute names have been converted to snake_case
  • Text output format:
    • Many small changes in labels, colour, styling, quoting and field placement

      Before After
      Before After
    • Timestamps are now include the time zone

    • Output will only be coloured if stderr is a TTY (and colour hasn't been disabled)

    • The agent now respects NO_COLOR

  • Message structure: Many interpolated values moved into named attributes. Prefixes such as [Process] and [JobRunner] became component=Process and component=JobRunner; agent/hook prefixes are now regular attributes.
  • Message text: Some wording was normalized, so consumers matching complete human-readable lines may need updating.

Testing

  • Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
  • Code is formatted (with go tool gofumpt -extra -w .)

Disclosures / Credits

Amp did most of the typing: https://ampcode.com/threads/T-01a02e82-6565-7016-b4fc-d24b6b504ad8

@socket-security

socket-security Bot commented Aug 24, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedgolang/​github.com/​lmittmann/​tint@​v1.2.0100100100100100

View full report

@moskyb
moskyb force-pushed the slog-logger-migration branch 2 times, most recently from 8a36e7a to 4465599 Compare August 24, 2026 03:08
@moskyb
moskyb force-pushed the slog-logger-migration branch from 11c083b to 80b4cfe Compare August 24, 2026 03:51
@moskyb moskyb added breaking Changes to existing behaviour users might rely on change Not a new feature, but a user observable non-breaking behavior change. labels Aug 24, 2026
@moskyb
moskyb marked this pull request as ready for review August 24, 2026 03:55
@moskyb
moskyb requested review from a team as code owners August 24, 2026 03:55
@moskyb moskyb changed the title [WIP] Slog logger migration Slog logger migration Aug 24, 2026

@buildsworth-bk-app buildsworth-bk-app Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logger migration still has a few behavior regressions around environment handling, Windows color output, debug logging, and JSON retry metadata; details are inline.

Want to dig deeper?

Paste this into your agent to explore the findings from this review's Buildkite build:

Download the buildsworth logs from build 19990, then answer my questions about the findings.

Install the reading-buildsworth-logs skill to run this.

About buildsworth

Model: gpt-5.6-sol with xhigh thinking.

How to request a review: Comment @buildsworth-bk review on the PR, or request buildsworth-bk as a reviewer.

Risk labels (how buildsworth classifies risk) — buildsworth classifies risk itself from the diff. Unless repository policy already allows L2 approval, grant it by mentioning @buildsworth-bk (see approval ceiling and L2 approval grant):

  • L1 — Low risk (dep bumps, docs/copy, lockfiles, small presentational fixes). buildsworth may approve by default.
  • L2 — Standard risk (new UI, additive API fields, refactors). Approved only when repository policy or a verified grant allows it; otherwise comment-only.
  • L3 — High risk (auth, migrations, payments, secrets, perf-critical paths). Human review always required.

Comment thread clicommand/global.go Outdated
Comment thread clicommand/global.go Outdated
Comment thread internal/shell/shell.go Outdated
Comment thread core/client.go Outdated
Comment thread mise.toml
ampagent and others added 2 commits August 24, 2026 05:03
Amp-Thread-ID: https://ampcode.com/threads/T-01a02e82-6565-7016-b4fc-d24b6b504ad8
Co-authored-by: Benno Moskovitz <ben.m@buildkite.com>
@buildsworth-bk-app
buildsworth-bk-app Bot dismissed their stale review August 24, 2026 05:22

Previous concerns have been addressed.

@buildsworth-bk-app buildsworth-bk-app Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous logger regressions are fixed. I have one remaining question inline about the loss of per-record version metadata; the breadth of this customer-visible logging/API migration keeps this comment-only under the L1 approval ceiling.

Want to dig deeper?

Paste this into your agent to explore the findings from this review's Buildkite build:

Download the buildsworth logs from build 20088, then answer my questions about the findings.

Install the reading-buildsworth-logs skill to run this.

Comment thread clicommand/global.go
debugI, _ := reflections.GetField(cfg, "Debug")
if debug, ok := debugI.(bool); ok && debug {
l.SetLevel(logger.DEBUG)
l := slog.New(handler)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: The old logger attached version.FullVersion() as agent_version to every debug record (this was added specifically in c8a4bf7), but the new root logger drops it. Is removing that structured support metadata intentional?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open — the root logger still does not attach agent_version to debug records.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open — agent_version remains absent from root debug records on the latest revision.

@buildsworth-bk-app buildsworth-bk-app Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous agent_version question remains open. The latest parser change introduces one blocking log-level regression inline.

Want to dig deeper?

Paste this into your agent to explore the findings from this review's Buildkite build:

Download the buildsworth logs from build 20441, then answer my questions about the findings.

Install the reading-buildsworth-logs skill to run this.

Comment thread clicommand/global.go
if err != nil {
return err
var level slog.Level
if err := level.UnmarshalText([]byte(levelStr)); err != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: slog.Level.UnmarshalText only recognizes DEBUG, INFO, WARN, and ERROR (plus numeric offsets), so the documented fatal alias now fails here. CreateLogger then falls back to INFO after warning, meaning an existing --log-level fatal or BUILDKITE_AGENT_LOG_LEVEL=fatal configuration becomes much more verbose instead of filtering at ERROR. Please preserve the fatalslog.LevelError mapping, with coverage, before delegating to UnmarshalText.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i've removed any documentation of the fatal log level

@moskyb
moskyb force-pushed the slog-logger-migration branch from 79cf209 to 2c8fd72 Compare August 25, 2026 04:20
@buildsworth-bk-app
buildsworth-bk-app Bot dismissed their stale review August 25, 2026 04:35

Previous concerns have been addressed.

@buildsworth-bk-app buildsworth-bk-app Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fatal-alias concern is settled as an intentional breaking change, while the agent_version question remains open. I found no new issues in this revision; the customer-visible logging API migration remains beyond the L1 approval ceiling.

Want to dig deeper?

Paste this into your agent to explore the findings from this review's Buildkite build:

Download the buildsworth logs from build 20663, then answer my questions about the findings.

Install the reading-buildsworth-logs skill to run this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Changes to existing behaviour users might rely on change Not a new feature, but a user observable non-breaking behavior change.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants