Skip to content

[Feat][SDK-601] Add built in scrubbing - #377

Open
buongarzoni wants to merge 22 commits into
masterfrom
feat/SDK-601/add-built-in-scrubbing
Open

[Feat][SDK-601] Add built in scrubbing#377
buongarzoni wants to merge 22 commits into
masterfrom
feat/SDK-601/add-built-in-scrubbing

Conversation

@buongarzoni

@buongarzoni buongarzoni commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Description of the change

Adds a scrubbing layer that runs on every payload, so sensitive data is redacted by default rather than only when a user wires up their own transformer.

What it does

ScrubDataTransformer runs after any user-supplied transformer and cannot be
bypassed by replacing it. It redacts values with *** in:

  • Fields whose key names a secret, with no configuration: password, passwd,
    secret, token, authorization, authentication, ^auth$, apikey/api_key/
    api-key. Case-insensitive, matched anywhere in the key (so token covers
    access_token and csrfToken); auth is anchored so author is left alone.

  • Request headers — via a built-in deny-list (Authorization, Cookie,
    Set-Cookie, X-Api-Key, X-Auth-Token, X-Access-Token, X-Secret,
    Proxy-Authorization, WWW-Authenticate). Always on, no config needed.

  • Every slot a request is serialized into: request.get, request.post,
    request.params, request.metadata, request.query_string, custom data and
    Frame.locals, including the copies carried by body.threads. Nested maps,
    collections and arrays are traversed to a depth of 8, preserving shape.

    Request.url is sanitized unconditionally by DefaultUrlSanitizer, which strips
    userinfo, query string, and fragment.

Not covered: request.body, a raw string the notifier cannot parse.

Config

ConfigBuilder.withAccessToken(TOKEN)
    .redactedKeys(Arrays.asList("ssn", "pin"))  // case-insensitive regex, additive
    .useDefaultRedactedKeys(false)              // optional: match only my keys
    .urlSanitizer(myCustomSanitizer)            // optional
    .build();

Added to both the core and reactive-streams builders. useDefaultRedactedKeys
defaults to true on CommonConfig, so third-party Config implementations get the
built-in redaction too.

Warning

Behavior change. Two things arrive differently in Rollbar after this PR:

  1. Fields matching the built-in key list are redacted, including keys that are not
    strictly secrets — substring matching means tokenCount is redacted too. Set
    useDefaultRedactedKeys(false) if that is too broad for your payloads.
  2. Query strings are stripped from request.url. Users who relied on seeing them
    need a custom urlSanitizer.

Performance

Keys without regex syntax — the whole built-in list, and most user keys — are matched
with a lowercase pass and a substring search rather than a Matcher per key per
pattern. On a ~100-key payload that is 6.3 µs and 7.5 KB allocated, down from 28.7 µs
and 171 KB. Regex semantics for redactedKeys are unchanged; keys that do carry regex
syntax still run through Pattern.

okhttp

RollbarOkHttpInterceptor now delegates its default URL sanitization to the
shared DefaultUrlSanitizer so the okhttp and notifier paths can't drift apart,
and gains a withSharedUrlSanitizer(recorder, sanitizer) factory to reuse the
sanitizer from a notifier config.

The sanitizer types live in rollbar-api (com.rollbar.api.scrubbing), not
rollbar-java, so rollbar-okhttp keeps its existing lightweight dependency.
The factory is static rather than a constructor overload — UrlSanitizer and
StringUrlSanitizer are both functional interfaces, so an overload would make a
lambda argument ambiguous and break existing callers at compile time.

Testing

  • Unit coverage of the key list, the opt-out, nesting, collections/arrays, query-string
    encoding, and literal-vs-regex matching equivalence.
  • End-to-end tests against the JSON WireMock receives, including one that sends a
    /login?password=hunter2 request through a bare ConfigBuilder and asserts the
    secret appears nowhere in the payload.
  • Docs in SCRUBBING.md

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Maintenance
  • New release

Related issues

Shortcut stories and GitHub issues (delete irrelevant)

Checklists

Development

  • Lint rules pass locally
  • The code changed/added as part of this pull request has been covered with tests
  • All tests related to the changed code pass in development

Code review

  • This pull request has a descriptive title and information useful to a reviewer. There may be a screenshot or screencast attached
  • "Ready for review" label attached to the PR and reviewers assigned
  • Issue from task tracker has a link to this pull request
  • Changes have been reviewed by at least one other engineer

@linear-code

linear-code Bot commented Jul 13, 2026

Copy link
Copy Markdown

SDK-601

@buongarzoni buongarzoni added this to the v2.4.0 milestone Jul 13, 2026
@buongarzoni buongarzoni self-assigned this Jul 13, 2026
@buongarzoni

Copy link
Copy Markdown
Collaborator Author

@claude review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 52d5d40da7

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@brianr brianr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The always-last transformer and copy-on-write implementation are a good foundation, and the current head fixes the earlier routing-param, metadata, encoded-query-key, and JVMTI thread-local gaps. I am requesting changes for the two remaining payload paths called out inline. Before release, this behavior change also needs user-facing migration documentation plus an integration test proving transformer ordering/reconfiguration (including the reactive builder) and the shared OkHttp sanitizer path.

result = new HashMap<>(map);
}
result.put(key, SCRUBBED_VALUE);
} else if (value instanceof Map && depth < MAX_SCRUB_DEPTH) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Traverse collections and arrays when scrubbing nested data

This only descends into values that are directly Map instances. Valid payloads such as custom = {"users": [{"password": "hunter2"}]} therefore ship the configured secret unchanged. The serializer supports both Collection and Object[], and the same bypass applies to nested values in Request.post, Request.metadata, and Frame.locals. Please recursively scrub maps contained in collections/arrays under the same depth cap, preserve the surrounding shape, and add regression tests for both forms.


Request scrubbedRequest = scrubRequest(originalRequest);
Map<String, Object> scrubbedCustom = scrubObjectMap(originalCustom, fieldPatterns, 0);
Body scrubbedBody = scrubBody(originalBody);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Sanitize URLs in network telemetry events

RollbarBase.recordNetworkEventFor(...) passes its URL directly to the tracker, which stores it unchanged. Those events are later carried in Body.telemetryEvents, but scrubBody() only rebuilds trace content and thread traces. A caller can therefore record https://user:pass@example.com/path?token=secret and ship both userinfo and query string despite the built-in scrubber. Please apply the configured sanitizer before recording network telemetry, or rebuild network telemetry events here, and cover the public notifier API with a regression test.

@brianr brianr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One additional security issue on the current head.

this.compressPayload = builder.compressPayload;
this.maximumTelemetryData = builder.maximumTelemetryData;
this.telemetryEventTracker = builder.telemetryEventTracker;
this.redactedKeys = builder.redactedKeys != null

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Seed field scrubbing with a meaningful default list

When redactedKeys is unset, this stores an empty list, so ScrubDataTransformer leaves GET/POST parameters, request.query_string, custom data, metadata, and Frame.locals untouched. Stripping the query from request.url does not close that gap: the web request provider serializes parsed GET parameters and the raw query string separately, so /login?password=hunter2 still sends hunter2 under both request.get.password and request.query_string with the default configuration.

That means the default behavior only protects the built-in header deny-list and URL strings; it does not satisfy the stated goal that sensitive data is redacted by default. Please seed this with an additive built-in field list (for example password/passwd, secret, token/access_token, auth/authentication/authorization), ideally with an explicit override mechanism, and add a no-configuration end-to-end test proving the secret is absent from every relevant request representation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants