Streaming Entropy, Part 3: Block-wise Entropy
Reported by candidates from OpenAI's online assessment. Pattern, common pitfall, and the honest play if you blank under the timer.
The OpenAI OA reported in July 2026 hands you Part 3 of a four-part entropy sequence, and the detail that matters is the ban on concatenating blocks or keeping the logits. You're computing softmax entropy over a stream, one block at a time, with constant state. The logits sit in [-20, 20], so overflow isn't the fight here. The fight is rewriting entropy so one pass works. If you've got this OA in a day or two, learn the identity below. StealthCoder is the backup if you blank mid-assessment, but this one is short enough to own.
The problem
Streaming Entropy Interview Sequence This is Part 3 of one four-part ML coding exercise. Streaming Entropy, Part 1: Batch Entropy Streaming Entropy, Part 2: Numerically Stable Entropy Streaming Entropy, Part 3: Block-wise Entropy Streaming Entropy, Part 4: Stable Streaming Entropy The logits of one softmax distribution now arrive in consecutive, non-empty blocks. Concatenating the blocks in order would produce the complete logits array. For the full stream, p[i] = exp(x[i]) / sum(exp(x[j])) H = -sum(p[i] * ln(p[i])) Streaming Requirements Process the blocks in order and return the entropy over every logit in the stream. Do not concatenate the blocks, retain the full logits stream, or materialize the probability vector. Keep only constant-size aggregate state. This part uses moderate logits so that block-wise processing, rather than numerical stability, is the main challenge. Your implementation must run in O(n) time and use O(1) auxiliary space, where n is the total number of logits. An answer is accepted when its absolute or relative error is at most 1e-6. Continue to Part 4: Stable Streaming Entropy. Function blockwiseEntropy(blocks: double[][]) → double Examples Example 1 blocks = [[0.0], [0.0]] return = 0.6931471805599453 The blocks form the logits [0, 0], which produce a uniform two-outcome distribution. Example 2 blocks = [[0.0], [1.0986122886681098]] return = 0.5623351446188083 The blocks form logits whose softmax probabilities are [0.25, 0.75]. Example 3 blocks = [[1.0, -1.0, 0.0], [2.0], [-2.0, 0.5]] return = 1.2511246070440085 This stream uses uneven blocks and includes positive, negative, and fractional logits. Constraints 1 <= blocks.length <= 10^4 1 <= blocks[i].length The total number of logits across all blocks is at most 2 * 10^5. Every logit is finite and lies in [-20, 20]. Use the natural logarithm. Use O(n) time and O(1) auxiliary space.
Reported by candidates. Source: FastPrep
Pattern and pitfall
The trick is algebra. With Z = sum(exp(x)), p[i] = exp(x[i])/Z, so H = -sum(p * (x - ln Z)) = ln Z - (sum(exp(x) * x)) / Z. That needs only two running sums: Z = sum(exp(x)) and S = sum(x * exp(x)). Walk every block in order, update both, then return ln(Z) - S/Z. That's O(n) time and O(1) space. The common pitfall is trying to compute p[i] per element in the first pass, which needs Z before you have it. Another is flattening blocks into a list, which breaks the stated rule. Don't normalize early. Because logits are bounded by 20, exp stays safe in doubles, so skip the max-shift trick until Part 4. If you freeze live, StealthCoder can surface the two-sum formula as a hedge, but you only need those two accumulators.
If this hits your live OA and you blank, StealthCoder solves it in seconds, invisible to the proctor.
You can drill Streaming Entropy, Part 3: Block-wise Entropy cold, or you can hedge it. StealthCoder runs invisibly during screen share and surfaces a working solution in under 2 seconds. The proctor sees the IDE. They don't see what's behind it. Built by an Amazon engineer who would have shipped this the night before his JPMorgan OA if he'd had it.
Get StealthCoderRelated leaked OAs
You've seen the question.
Make sure you actually pass OpenAI's OA.
OpenAI reuses patterns across OAs. Built by an Amazon engineer who would have shipped this the night before his JPMorgan OA if he'd had it. Works on HackerRank, CodeSignal, CoderPad, and Karat.
Streaming Entropy, Part 3: Block-wise Entropy FAQ
What's the trick for the OpenAI block-wise entropy problem?+
Rewrite entropy as ln(Z) - S/Z, where Z is the sum of exp(x) and S is the sum of x*exp(x). Both accumulate across blocks in one pass. You never need the probabilities, so no vector and no concatenation is required.
Do I need the log-sum-exp max shift in Part 3?+
No. Logits are in [-20, 20], so exp(20) is about 4.85e8 and sums over 2*10^5 elements stay far below double overflow. The statement says block-wise processing is the challenge here. Stability is saved for Part 4, so keep this version simple.
How do I verify my answer on the examples?+
Use Example 1: blocks [[0],[0]] give Z=2, S=0, so H = ln 2 = 0.6931. Example 2 should give 0.5623 for probabilities 0.25 and 0.75. Check those by hand before submitting, since the tolerance is 1e-6.
Is a two-pass approach allowed?+
A second pass over the blocks would still be O(n) time and O(1) space if the blocks are given as input. But the single-pass formula is cleaner and matches the streaming spirit. It also carries straight into Part 4, so use it.
How should I prepare in 48 hours for this OA?+
Work out the entropy identity on paper until you can derive it cold. Then code it with plain loops over blocks and test uneven block sizes. Expect Part 4 to add numerical stability, so think about how a running max would rescale Z and S.