Integration
sentry-spring-boot (Spring Boot 4)
Java Version
25 (Eclipse Adoptium 25.0.3)
Version
8.49.0
Steps to Reproduce
Conditions under which we hit this in production (2 replicas, both affected):
logs.enabled=true (Sentry Logs feature on), traces-sample-rate > 0.- The org/project hits its span/transaction quota — Sentry starts rate-limiting the
transaction/span/monitor categories (429 / X-Sentry-Rate-Limits). Log quota was NOT exhausted.
- As soon as envelopes start being discarded (rate limiting, and subsequently queue overflow), the SDK enters a hard CPU busy-loop: the
SentryAsyncConnection and SentryExecutorService threads each pin one CPU core at 100%, indefinitely.
Restarting the JVM does not help — a freshly booted instance re-enters the loop within seconds, as long as the rate limiting is active. Event delivery from the affected instance stops entirely. Setting logs.enabled=false prevents the loop (our current workaround, together with traces-sample-rate=0 to stop feeding the rate-limited categories).
Expected Result
Discarding rate-limited envelopes and recording client reports should be O(1)-ish and quiet — no sustained CPU burn, and no impact on delivery of non-rate-limited categories (errors).
Actual Result
Async-profiler (via Pyroscope) shows the loop clearly. CPU is dominated by exception construction — java.lang.Throwable::fill_in_stack_trace is the single largest self-time frame — reached through this path:
java.util.concurrent.ThreadPoolExecutor$Worker.run
io.sentry.clientreport.ClientReportRecorder.recordLostEnvelope
io.sentry.clientreport.ClientReportRecorder.recordLostEnvelopeItem
io.sentry.SentryEnvelopeItem.getLogs <- deserializes the log payload to count items
io.sentry.JsonSerializer.deserialize
io.sentry.SentryLogEvents$Deserializer.deserialize
io.sentry.SentryLogEvent$Deserializer.deserialize
io.sentry.JsonObjectReader.beginObject / nextListOrNull
io.sentry.vendor.gson.stream.JsonReader.beginObject
java.lang.IllegalStateException.<init> <- thrown at extreme rate
java.lang.Throwable.fillInStackTrace <- dominates the CPU profile
The JsonObjectReader recovery machinery (beginRecovery/recoverValue/recoverAfterValueFailure/skipValue) also shows up prominently, suggesting the deserializer keeps throwing and "recovering" per token over the same payload, per discarded envelope, in a tight loop.
Relative weights from a 30-minute profile window (two threads pinned the whole time):
| frame |
share of profile |
SentryLogEvents$Deserializer.deserialize (total) |
~1.9x of one core-window |
ClientReportRecorder.recordLostEnvelope (total) |
~0.97x |
Throwable.fillInStackTrace (total) |
~1.4x |
IllegalStateException.<init> (total) |
~0.73x |
Client reports from the affected day also show the feedback loop (values are per ~24h, one service): transaction/client_discard: 711850, span/client_discard: 773325, log_byte/client_discard: 8054 — far beyond what the service organically produces, consistent with the discard path re-processing items continuously.
Happy to provide the full flamegraph JSON privately if useful.
Integration
sentry-spring-boot (Spring Boot 4)
Java Version
25 (Eclipse Adoptium 25.0.3)
Version
8.49.0
Steps to Reproduce
Conditions under which we hit this in production (2 replicas, both affected):
logs.enabled=true(Sentry Logs feature on),traces-sample-rate> 0.transaction/span/monitorcategories (429 / X-Sentry-Rate-Limits). Log quota was NOT exhausted.SentryAsyncConnectionandSentryExecutorServicethreads each pin one CPU core at 100%, indefinitely.Restarting the JVM does not help — a freshly booted instance re-enters the loop within seconds, as long as the rate limiting is active. Event delivery from the affected instance stops entirely. Setting
logs.enabled=falseprevents the loop (our current workaround, together withtraces-sample-rate=0to stop feeding the rate-limited categories).Expected Result
Discarding rate-limited envelopes and recording client reports should be O(1)-ish and quiet — no sustained CPU burn, and no impact on delivery of non-rate-limited categories (errors).
Actual Result
Async-profiler (via Pyroscope) shows the loop clearly. CPU is dominated by exception construction —
java.lang.Throwable::fill_in_stack_traceis the single largest self-time frame — reached through this path:The
JsonObjectReaderrecovery machinery (beginRecovery/recoverValue/recoverAfterValueFailure/skipValue) also shows up prominently, suggesting the deserializer keeps throwing and "recovering" per token over the same payload, per discarded envelope, in a tight loop.Relative weights from a 30-minute profile window (two threads pinned the whole time):
SentryLogEvents$Deserializer.deserialize(total)ClientReportRecorder.recordLostEnvelope(total)Throwable.fillInStackTrace(total)IllegalStateException.<init>(total)Client reports from the affected day also show the feedback loop (values are per ~24h, one service):
transaction/client_discard: 711850,span/client_discard: 773325,log_byte/client_discard: 8054— far beyond what the service organically produces, consistent with the discard path re-processing items continuously.Happy to provide the full flamegraph JSON privately if useful.