Detecting PII in API Traffic with a Small BERT Model: Training Notes and Pitfalls

I went through more than twenty versions of this PII model. This post records the pitfalls I actually hit, the trade-offs I made, and the numbers I measured. The failed attempts are in here too, so next time nobody has to try them again.

Why build it

API traffic often has to be checked for personal data. The easy way is to send every response body to an LLM and ask “is there a phone number, ID number or card number in here?” It works, but the problems are obvious:

1. It’s expensive, and the cost grows linearly with traffic. Bodies are longer than you’d think: I measured a median of 829 tokens, p90 of 33,803, and a max of 150,547. At p90 that’s over 30k tokens per check, and the traffic never stops.
2. Latency is out of your control, and traffic checks often have to run inline.
3. Results aren’t reproducible. The same input can get a different answer at a different time or on a different model version. For compliance work that’s a real problem: you can’t prove “this request was judged no-risk last month”.

So the goal was to train a small model and turn “pay for every call” into “train once, run inference as much as you like”.

Final result first: a 307M-parameter encoder, 0.93 macro-AP, 14 of 23 labels usable. Median latency 339 ms on a single CPU process, with no per-token cost.

The latency distribution is heavily skewed, though: p90 is 16.8 s and p99 is 73 s, because 21% of the traffic is longer than 8k tokens. More on that tail later.

How is this different from an LLM

Show me. Same response body for both (all values are made up):

{"code":0,"data":{"addresses":[{"address_id":"778901","receiver_name":"Andi Saputra","phone":"+62 812-3456-7890","detail":"Jl. Melati Raya No. 42, RT.003/RW.005","city":"Jakarta Pusat","province":"DKI Jakarta","post_code":"10510","is_default":true}]}}

Ask an LLM “what personal data is in here?” and you usually get a paragraph, something like:

This data contains the following personal information:
- Name: Andi Saputra (receiver_name field)
- Phone number: +62 812-3456-7890
- Address: Jl. Melati Raya No. 42, RT.003/RW.005, Jakarta Pusat
- Postal code: 10510
Consider masking these fields.

This small model returns this (actual output):

["phone_number", "person_name", "street_address", "postal_code"]

street_address  1.0000  (threshold 0.3451)
postal_code     0.9999  (threshold 0.9756)
person_name     0.9997  (threshold 0.9560)
phone_number    0.9996  (threshold 0.9872)
other 19 labels ≤ 0.06

The verdicts are about the same. The differences are here:

	LLM	This small model
Model type	Generative (decoder), writes token by token	Encoder (BERT-style), reads the whole input and scores it
Size	Tens to hundreds of billions of parameters	307M
Output	Natural language, needs parsing	Probabilities for 23 fixed labels, ready to use
Can it point at the value	Yes, and explain why	No, only whether it’s present
Adding a label	Edit the prompt	Label new data and retrain
Cost	Pay per token, grows with traffic	Train once, runs on CPU
Same input, same result	Not guaranteed	Always
Long text	Bounded by the context window; 150k tokens is pricey	Windowed, any length

In short: an LLM can do anything, but it’s expensive, slow, and may answer differently each time. This model does one thing, deciding whether these 23 categories are present, but it’s cheap, fast and stable.

So it isn’t a replacement for everything an LLM does. It takes over one fixed, repetitive job: tagging traffic in bulk. When you need to extract values, explain a decision, or handle a new type, you still want an LLM.

A quick word on Jev

TypeSafe AI just released Jev (announcement), which they call a System One model. It doesn’t generate text: you give it some state and a few predefined questions (pick one, give a score, yes or no), and it returns structured answers with probabilities in one parallel forward pass. They quote 70–500 ms latency.

As I read it, Jev goes in the same direction as my model: don’t make the model write an answer, just score a fixed output space. That’s also the core difference between the small model and the LLM in the table above.

There are clear differences, though:

	Jev	My model
Where questions are defined	Written per request, no training	Fixed at training time: these 23 labels
Size	Not disclosed, official API only	307M, self-hosted, runs on CPU
Probabilities	Trained for calibration with RLCD	A threshold per label, fitted on a separate calibration set
Context	About 64k tokens for state plus questions	Windowed, no length limit

So, is it an LLM underneath? It’s not a GPT-style model that writes token by token. But it takes arbitrary new questions at runtime without fine-tuning, and that kind of general ability most likely still comes from large-scale pretraining. The parameter count and architecture haven’t been published. I’d describe it as a general-purpose large model that doesn’t generate text, while mine is a small model that does exactly one thing.

Put simply, mine gets a System-One-like result on one fixed task the small-model way. The idea is the same. Jev’s advantage is that a new question doesn’t need new labels or retraining; mine is cheap, runs locally, and the data never leaves.

That said, Jev isn’t fully open yet. It’s API-only, the architecture and size aren’t public, and it may well be a large model underneath, so on cost and on data leaving your environment it may not be much better than just using an LLM. My plan: keep using my own small model for now, bring Jev in to study it, run both on the same validation set, and see how accuracy and cost actually compare. I’ll decide on switching after that and write it up separately.

Task definition: narrow it down first

This took longer than I expected, but it saved a lot later.

Presence only, not position. I don’t answer “the phone number is at character N”, only “is there a phone number in this text”. What’s needed downstream is “should this endpoint be tagged”, not the value itself. That turns sequence labelling into multi-label classification and cuts labelling cost by an order of magnitude.

Labels are independent, not mutually exclusive. An order response body with products, a name, a phone number and an address all at once is normal. So it’s 23 independent binary heads on one shared encoder, each label with its own threshold.

Public label names are in English. The labelling spec is written in Chinese, but the released model uses identifiers like phone_number, person_name and street_address, with the Chinese names kept in the config’s label_names_zh. The rename only touched metadata. The head’s 23 outputs keep their order, and I checked the same input before and after: identical scores, 0.0 difference.

Explicitly out of scope: no ownership (buyer’s phone vs. seller’s phone), no value extraction, no cross-field association. That trade-off kept the architecture much simpler, which comes up again in the windowing section.

Data: where the traffic comes from and how labels were decided

Collection

Training data is real API traffic. I searched for endpoints likely to be relevant by label keywords, pulled the latest samples for each endpoint, and kept only the response body, dropping HTTP headers.

First pitfall right here: I initially collected only response bodies, not the request side. It was a carry-over from an earlier version that nobody had ever justified. Later I found that the sampling API already returned request headers, query parameters, request body, path and method together; the collection script just read one field.

After adding it I measured the gap. Among samples containing the relevant pattern, the share that appeared only on the request side and not in the response was 7% for email, 5% for phone numbers and 16% for long numeric IDs. Not huge, but not ignorable either.

Labelling

Everything was blind-reviewed row by row by AI reviewers. Blind means the reviewer sees only the text: no existing labels, no model scores, no routing hints, and no regex scripts for bulk labelling. Ten rows per batch, each written to its own file, with a validator checking that batches line up with the queue.

Provenance records the actual reviewer model name and never claims human review. That paid off later: one review pass spanned two different models, and I could record them per batch instead of writing one vague name.

Splits: group by endpoint, not random rows

This is the easiest step to get wrong, and the most costly.

One API endpoint produces hundreds or thousands of response bodies with nearly identical structure. Split by random rows and the same endpoint lands in both train and validation, so validation scores are badly inflated. The model only has to remember “what this endpoint looks like” to get it right.

So split by endpoint group, keeping every sample from an endpoint inside one split. I used hash buckets, with train, validation and calibration ranges not overlapping.

I still got bitten once: when the validation set was enlarged, the new hash range was a superset of the old one. When I used an old model as a reference on the new validation set, 49 of 303 rows were actually its training data, and it scored 1.000 AP on them. After finding that, every comparison was restricted to “clean” rows.

Lesson: before using an old model as a reference on a new split, check the sample hashes for overlap.

What each split is for

Split	Rows	Purpose
Train	13,915	Backprop only
Validation	4,359	Checkpoint selection, scores, statistical tests
Calibration	1,600	Only for fitting each label’s threshold

Calibration gets its own split because thresholds are also fitted on data. Fit thresholds on validation and report scores on validation, and the F1 is inflated.

Model and architecture

Base model

mmBERT-base, a multilingual encoder on the ModernBERT architecture: 22 layers, 768 hidden, 256k vocabulary, 8192 positions natively. Multilingual because the traffic mixes Chinese, English and Indonesian.

Long text: sliding windows + max across windows

The encoder can’t take 150k tokens in one go, so the input gets split. The flow:

long text → 512-token windows (adjacent windows overlap by 128)
          → each window through the encoder, mean-pool tokens with padding masked, linear head → 23 logits
          → for each label, max over all windows = document score
          → sigmoid, compare with the label's threshold

The max step is the core. It’s multiple-instance learning (MIL): a document is a “bag”, and if any one window sees a phone number, the whole document counts as having one. During training, the gradient only flows back through the strongest window.

Someone once asked me whether the windowing is built into the model or something I wrote. Splitting into windows is a standard feature of the HF tokenizer: return_overflowing_tokens=True plus a stride argument. What I wrote myself is just the max line and the window-selection logic for training.

One detail I got backwards at first: stride is the overlap, not the step. 512 minus 2 special tokens leaves 510 positions; stride=128 means adjacent windows share 128 tokens, so each step actually advances 382.

Training and the pitfalls

Three problems in the original training script

None of them raised an error; all of them quietly hurt results:

1. Round-robin label sampling. Each label got about the same number of updates regardless of how many positives it had. One label had a single real positive, and that one row got sampled more than two hundred times: heavy overfitting. Switched to square-root weighting by positive count, and labels with fewer than 3 positives are no longer sampling targets.
2. No checkpoint selection. Train a fixed number of steps and keep the final weights. Switched to evaluating on a validation subset every 1500 steps and restoring the best one at the end.
3. Positives only backprop through the argmax window. If a noisy window happens to score highest, the gradient reinforces the wrong window. I added an option to use the mean of the top-k windows for positives. k=3 helped on its own but hurt when combined with the other changes, so it didn’t make the recipe.

How to tell whether a change really helps

Changing only the random seed on the same recipe moves macro-AP by 0.019. So with a single seed, any gain under 0.02 could be real or could be luck.

What I did: paired bootstrap grouped by endpoint, computing a 95% confidence interval on the before/after difference. If the interval crosses zero, it’s indistinguishable, however nice the point estimate. The same test runs per label, plus a “material” threshold: a drop that’s statistically significant but smaller than 0.02 in absolute terms doesn’t count as a real regression.

Looking back with that standard, all 14 hyper-parameter arms I ran landed between 0.891 and 0.914, and none could be distinguished from the baseline. Learning rate, window stride, documents per update, head learning rate: all noise.

The four things that actually helped

In order of effect:

1. Label correctness matters more than any hyper-parameter.

I found that the “address” label in the old data followed a different convention from the current spec: rows with only a city or a postal code were labelled as addresses. The model saying “this isn’t a street address” was actually right but got scored as wrong. After one round of blind re-review, with the same model and nothing else changed, the AP for street_address went from 0.819 to 0.889, and positives dropped from 62 to 39.

That gain has nothing to do with the model; it’s just the ruler being calibrated. Put the other way: had I not caught this, I might have kept blaming model capacity and kept tuning useless hyper-parameters.

The same review exposed a real blind spot: pinyin account names (like liruiming) and real names in display_name fields should count as names under the current spec, but the old labels missed them, so the model never learned them and scored those samples near 0.

There’s a trap to watch here. After relabelling, a model trained on the new labels will of course look better on the new validation set, but that may just be fitting the reviewer’s preferences rather than being genuinely stronger. I split validation rows by “was this label changed” and scored the groups separately: the entire gain came from the 58 changed rows, and on the 2,362 rows whose definition didn’t change, the new model was slightly lower. So the accurate statement is “the model switched to the new convention”, not “the model got better at finding names”.

2. DAPT (domain-adaptive pretraining).

First a round of MLM on unlabelled text from the same distribution, then fine-tuning. I used 240k unlabelled windows; held-out MLM loss went from 13.08 to 0.743.

The effect reproduced across seeds, and the gains concentrated on the labels with the fewest positives: payment_card_number 0.82 → 0.86, national_id 0.84 → 0.87, tax_id 0.93 → 0.97. That makes sense: once the encoder knows what this kind of text looks like, rare labels don’t have to learn representations from scratch.

This step had a GPU memory pitfall. MLM predicts a distribution over the whole vocabulary at every position, the vocabulary here is 256k, and the output tensor grows with batch size and sequence length. It filled the card.

The fix is sparse prediction: compute logits only at masked positions. With a 0.3 mask rate, the output tensor drops below a third.

The catch: this switch has to be passed in at from_pretrained time; changing the config after the model is built does nothing, because the output layer is already built in dense mode. I wasted two full runs on this, and the second time I thought the batch just wasn’t small enough. The final setup was sparse prediction + gradient checkpointing, batch 8, gradient accumulation 4.

3. Unfreeze all layers. Fine-tuning only the top layers vs. all 22: the latter is 0.029 AP higher, the single biggest gain.

4. Weight averaging (model soup), with a condition. Uniformly average the weights of the same recipe across seeds. Averaging three seeds put both the AP and F1 confidence intervals above zero, and the AP interval was the narrowest of any arm. But every model being averaged has to be strong: another two-seed average of mine came out below both of its members.

Three things that didn’t help

1. LLRD (layer-wise learning-rate decay), the only clearly distinguishable negative result. Decaying the learning rate exponentially toward the bottom layers sounds good for small-data fine-tuning, but on two seeds the intervals excluded zero, averaging -0.041. In hindsight it makes sense: full unfreezing was the biggest gain, and shrinking the bottom-layer learning rate cancels it out.
2. Weight EMA: +0.001 and +0.003, very narrow intervals that cross zero.
3. Asymmetric loss (ASL), designed for class imbalance: +0.009 and +0.001, within noise.

All three switches stay in the code (off by default, no behaviour change), but not in the recipe. Writing them down is so nobody tries them again.

A bug that failed silently

I added newly collected request-side data to the training set, and the scores came out almost exactly the same as before. It turned out the training script only trains on rows whose source is on a “real source” whitelist. The source name I gave the new data wasn’t on it, so 369 rows were silently filtered out, and not one of them was trained on.

Exit code 0, gates passed, report looked fine, nothing learned.

The only visible sign was the training row count in the report. So after adding any new data source, check the number of rows that actually went into training.

Does long context actually work

The base model supports 8192 natively, so why not feed the whole document at once and drop the windowing?

My initial theory: long windows do worse because of token mean pooling inside the window. The longer the window, the more a single phone number gets diluted in the pooled vector. Switch to a pooling that doesn’t dilute, and long context should work.

The measurements say that theory was wrong:

Recipe	Window	Pooling	AP	Long-doc AP	Training time
Short windows	512	Mean	0.930	0.909	94–156 min
Long context	2048	Mean	0.896	0.825	188 min
Long context	2048	Approx. token max	0.894	0.848	189 min

Both poolings lose, so the problem isn’t pooling. It’s the length itself.

And long context does worst on long documents, the exact opposite of the intuition that longer windows capture long-range associations better.

My current understanding: a 512 window that happens to hold the phone field and its surroundings gives a very clean positive signal. At 2048, the same phone number is buried among 1,500 unrelated tokens, and the model has to find a needle in a haystack inside one pooled vector. The smaller the window, the simpler each window’s decision, and the more likely the max picks up a clean positive.

Long context is slower too

Attention is quadratic in sequence length. Measured:

Actual tokens	512 windows	2048 windows	8192 windows
514	2 windows 0.35 s	1 window 0.19 s	1 window 0.19 s
2,045	6 windows 0.97 s	1 window 1.03 s	1 window 1.01 s
4,098	11 windows 1.83 s	3 windows 2.97 s	1 window 3.07 s
8,269	22 windows 3.57 s	5 windows 5.44 s	2 windows 20.45 s
32,936	86 windows 15.87 s	19 windows 22.68 s	5 windows 52.41 s

The 8,269 row makes it clearest: because of the 33% overlap, the 512 path actually sends more tokens through the encoder (11,001 vs. 8,785), yet it’s still 5.7× faster. A quarter more tokens, a fifth of the time; the whole gap is the quadratic attention term.

Below about two thousand tokens, though, one long pass is slightly faster, because there are only 1 or 2 windows anyway and the fixed overhead of windowing dominates. The crossover is between two and four thousand tokens.

So the worry that “windowing slows throughput” actually points the wrong way.

No cross-window association

Across windows it’s a max, which is an OR, not an AND. So any decision that needs “A and B both hold” with the two more than about 510 tokens apart is beyond this architecture.

All 23 current labels can be decided locally: to tell whether a value is a phone number, the value and its field name are enough, and nothing two thousand tokens away matters. So they’re unaffected.

But relational labels in the future (say, buyer’s phone vs. seller’s phone) would need changes. A cheap approach is to build a ~30-token prefix of global context (path, method, key fields) and put a copy in front of every window, at roughly 6% extra cost.

GPU memory: one change removed the need for an A100

Training with 2048 windows kept going OOM on a 22 GB card, even with gradient checkpointing on.

The cause: during training, each document picks one strongest window per observed label, up to 23 different windows, and all of their computation graphs are kept alive at once, waiting for one backward pass. Make the window 4× longer and attention memory grows 16×; with 23 graphs in memory at once, of course it blows up.

The encoder’s built-in gradient checkpointing only saves activations between layers; it can’t do anything about 23 graphs existing at the same time.

The fix is window-level checkpointing: keep only each window’s pooled output and recompute the whole window during backward, at the cost of one extra forward per window.

Recipe	Peak memory
2048 windows, before	21.2 GiB → OOM (card capacity 21.99 GiB)
2048 windows, after	8.14 GiB
512 windows, original	11.49 GiB

After the change, 2048 uses 29% less memory than the original 512 recipe. No bigger card needed.

(All units are GiB, the same basis as the 21.99 GiB nvidia-smi reports. In decimal GB it’s 8.74 / 12.33; the ratio doesn’t change.)

I wrote a test to confirm the change doesn’t alter results: with the switch on and off, forward outputs and head gradients match within 1e-6, and the gradients are non-zero. Any optimisation that claims “saves memory, doesn’t change results” should come with a test like that.

Found right before release: labels suppress each other

This is the pitfall I remember best from the whole project, because the metrics looked fine the entire time and the model still had a hole.

While preparing the public release, I used a made-up order response body as a demo, with eight kinds of data: name, phone, address, email, user ID, product, masked card number and IP. The model reported only five, missing email, postal code, product data, IP and card number.

My first reaction was “a hand-written short JSON is out of distribution”, and I even wrote that into the docs. Then I ran a controlled experiment, and that wasn’t it.

Tested alone, every missed value scored perfectly:

• Card number: masked 4111 **** **** 1234, the full number, and last-four-only all scored 1.000
• IP: seven field names including client_ip, ip, remote_addr and X-Forwarded-For, plus a bare value and one inside a sentence, all 1.000
• Email: with 300 unrelated records appended and split into 25 windows, still 1.000

So it wasn’t field names, masking, or length dilution.

Then I added the fields back one at a time and watched the scores fall:

Document contents	Email	IP	Card number	Postal code
Only email	1.000	0.000	0.000	0.000
+ client_ip	0.999	1.000	0.000	0.000
+ card_no	0.997	1.000	0.121	0.000
+ post_code	0.997	0.999	0.221	0.991
+ name, phone, address	0.988	0.787	0.008	0.791
+ two levels of nesting	0.961	0.777	0.038	0.149

Same values, same field names, same length, and the card number fell from 1.000 to 0.008, the postal code from 0.991 to 0.149.

Why

Training is multiple-instance learning: each label backprops only through its own strongest window, but every label reads the same pooled vector. When a document has seven or eight positive labels at once, the encoder has to fit all of those signals into one 768-dimensional vector, and capacity is limited. High-frequency labels (name, phone, address, each with hundreds to thousands of training positives) crowd out the rare labels’ representation space. The card number has only 83 positives in the traffic data, so it collapses first.

Why I never noticed

Because I was always looking at macro-AP. The card number’s AP is 0.823, which looks perfectly usable. But AP is computed over the whole validation set, and most validation documents that contain a card number don’t also contain seven or eight other categories.

Aggregate metrics completely hide this kind of conditional failure. The right approach is to look at each label’s recall stratified by “how many positive labels this document has”, and I had never done that.

Every row in a multi-label model’s score table is an average. If your real data is more extreme than the validation set on some dimension (label density, document length, nesting depth), that row may not apply. Running one deliberately hard, hand-built sample before release is worth more than ten more hyper-parameter runs. That’s how I found this one, and only after I was already getting ready to publish.

Final results and scope

Macro-average over the 14 gated labels: AP 0.9297 / F1 0.8906, up from 0.7055 / 0.6497.

Two numbers here are easy to mix up: 14 labels have scores (at least 10 positives and 10 negatives in validation), and 13 labels have fitted thresholds (at least 10 positives in calibration).

“No fitted threshold” doesn’t mean “unusable” either, and I mixed that up at first too. Measured on the 10 labels at the default 0.5: IP, date of birth and age are perfect, card number F1 is 0.880, and password is 0.667, missing half. The only truly broken ones are biometric ID and house number (1 to 2 validation positives each, so they can’t be measured well anyway). Passport number, social security number and vehicle plate have zero validation positives, so their performance is completely unknown.

Tally: of 23 labels, 2 broken, 3 unknown, 1 with low recall, and the remaining 17 usable. All of that is in the model card.

Add what the previous section showed: even those 17 usable labels can nearly fail on dense documents, and aggregate metrics won’t catch it.

Scope needs to be clear too. Training and calibration were both on API traffic. I tested seven kinds of non-traffic text: CSV, SQL dump, logs, plain Chinese paragraphs, email, and Markdown résumés. High-frequency labels (name, phone, address, email) transferred surprisingly well, scoring above 0.99; rare labels basically didn’t work.

False positives on free text depend on the version, because the last round fixed them: on 20 sentences with no personal data at all, an early model flagged 6 as names (the name threshold was only 0.20 then), and the final shipped model flags 0. The reason is that after the label convention was fixed and thresholds recalibrated, the name threshold went from 0.20 to 0.96. So calibrating the ruler didn’t just change the scores; it changed how the model behaves out of distribution.

One more interesting observation: isolated values are actually missed. A phone number on its own triggers nothing; put the same number inside a sentence and it’s 1.00. The model learned “value + context”, not pure pattern matching.

My takeaways (somewhat subjective)

1. Fix labels before tuning the model. I spent far more time on hyper-parameters than they returned; two rounds of label review gained more than every hyper-parameter change combined. A new project should start with a sampled labelling-consistency check.
2. Run every change on at least two seeds. Single-seed conclusions within 0.02 can’t be trusted.
3. Split by business grouping, not random rows. Every time the split changes, check its overlap with past models’ training sets.
4. Use different data for threshold calibration and for reporting scores.
5. Record failed attempts. This round had three changes that clearly didn’t help and one that clearly hurt, and those findings are as useful as the successes.
6. Watch for failures that don’t raise errors. Silent filtering, config fields overwritten by a base class, a review guide that doesn’t match the queue: none of these throw, and only checking the intermediate counts catches them.
7. Don’t only look at aggregate metrics. My macro-AP went from 0.71 to 0.93, and it took a hand-built hard sample right before release to find that labels suppress each other.
8. Be your own reader before publishing. My release materials had three errors that would have been challenged on the spot: the benchmark code ran two forward passes in the long-context column but counted it as one, I’d reported one bucket’s mean as the median latency, and I’d mixed GB and GiB. I only caught them by re-running everything. Before any number goes public, re-run it yourself.

What’s open source

The windowed-inference code has nothing to do with PII specifically; it fits any “long documents + multiple labels + encoder” setup. I pulled it into its own repo:

https://github.com/godzeo/sliding-window-transformers

Three modules, eight tests:

• windows: splits overlapping windows and records which document each window belongs to
• aggregate: cross-window aggregation with four strategies, max / mean / top-k / log-mean-exp; when to use each and their pitfalls are in the docs
• checkpoint: window-level gradient checkpointing for training, the change above that took 21.2 GiB down to 8.14 GiB

The README has the measured numbers from this post: accuracy, speed and memory comparisons.

The model itself is packaged too: detect(text of any length) returns labels directly, and the windowing is invisible to the caller. Labels are in English (phone_number, person_name, street_address and so on), with Chinese names mapped back in the config.

In the model card I put a failing example instead of a pretty one: the order response body above, missing five labels. I listed the misses as-is and explained the label suppression. Users will test their own samples sooner or later anyway, so rather than have them find a problem and decide the model is bad, it’s better to say up front when it fails and why. Which two of the 23 labels are broken, which three were never tested, and which one has only half recall are all marked too.

Still unsolved

1. Label suppression has only been found and measured, not fixed. Possible directions are sampling training data by label density, or giving rare labels their own pooling; neither has been tried yet.
2. The latency tail. 21% of traffic is longer than 8k tokens and p99 reaches 73 s, so real use would need truncation or tiered processing first.

Finally

The model works, but there’s still some distance between “works” and “use it without worrying”. I’ll keep filling in the gaps.