ArtifactNet research · Part 7 · Jul 2026
Closing the genre gap and the false-positive rate we promised to fix
A research log covering July 2026. Part 6 left us with a promise: spend the next quarter on the hard-real false-positive rate instead of chasing another benchmark record. This is the record of keeping that promise — plus a genre we'd quietly given up on scanning, a ground-up retrain on the largest dataset we've assembled, and a deployment bug we caught only because we went looking for one.
Want to see the current model in action? Run any track through the free demo — the same forensic pipeline described below, no account required.
Part 6 ended with a specific, narrow target. The "udio weakness" we'd spent June chasing turned out to be a leaked-evaluation illusion — on a clean held-out set, udio detection was fine at 97.8%. The real, surviving weakness was false positives on genuine recordings: SoundCloud amateur uploads, FMA, lo-fi. On our 9-source hard-real benchmark, our best candidate still sat at 15.57% FPR, down from 32.25% but nowhere near production-grade.
We said the next quarter would go after that number directly, instead of collecting another benchmark win. This post is what that quarter looked like.
The first move wasn't a bigger model — it was a second opinion. Most of the false positives we were chasing weren't confidently wrong; they were borderline, sitting close to the decision boundary where the main CNN's single pass is least reliable. So we trained a small secondary model — internally called rescue-gate, a 28-feature LGBM head (verdict28) — whose only job is to take a second look at calls the main network isn't confident about, and override them when the evidence points the other way.
Early July, we integrated it into the serving path and locked in an operating threshold (τ). On its own, without touching the CNN at all, rescue-gate cut hard-real false positives meaningfully — evidence that a chunk of our FPR problem wasn't a representation problem at all, just a decision-boundary problem.
One rule we now enforce strictly: rescue-gate has to be retrained on the same generation of CNN it's gating. Mixing generations — an old rescue-gate behind a new CNN, or vice versa — isn't a small drift. We have an incident on file from April where exactly that mismatch spiked false positives to 52%. It's why every CNN promotion in this post ships with a matching rescue-gate retrain, never a reused one.
Mid-July, we ran a smaller side investigation: does each AI generator leave a distinguishable "fingerprint" in embedding space, separate from the general AI-vs-human signal? The answer was a qualified yes. Human-made music sits in a clearly distinct region from AI output regardless of generator, and the separation holds up reasonably well even on generator architectures the model has never seen — though not perfectly. It's a side result, not a shipped feature, but it's more evidence that the model is reading something structural about codec residue rather than memorizing per-generator surface cues (the same conclusion Part 6's failed experiments pointed to).
Around the same time, we started work on detecting partial-AI tracks — songs where only part of the material is AI-generated (an AI vocal over a human-performed instrumental, for example). Building that detector meant looking much more closely at genre-level behavior, and it surfaced a problem we'd been quietly working around rather than solving: electronic music had an unusually high false-positive rate, high enough that we'd stopped trying to scan it with any confidence.
The stopgap at the time was a conditional threshold bump for electronic-leaning sources — not a fix, just a way to stop the bleeding while we figured out the actual cause. Section 6 below is that cause.
By the third week of July it was clear that threshold tuning and a rescue-gate pass, however useful, weren't going to close the genre gap or push hard-real FPR the rest of the way down. Both problems shared a root cause: the CNN's training distribution didn't have enough electronic music or enough real-world SoundCloud-style re-encoded material to represent either well.
So we did the largest retrain in the project's history — 27,476 tracks across 22 sources, built from a leak-clean manifest (verified against every training log from every prior generation, not just the immediately preceding one). We call the resulting checkpoint bigset.
In the process we finally isolated what was actually driving SoundCloud-style false positives. It wasn't the platform, and it wasn't "amateur production" as a vague category — it was repeated MP3 re-compression. Tracks that had been re-encoded to MP3 multiple times (upload → re-download → re-upload, common on aggregator and remix-culture platforms) accumulate compression artifacts that resemble the codec residue we're trained to catch. Once we understood the mechanism, we could represent it directly in training instead of papering over it with a threshold.
rescue-gate was retrained on the same bigset generation (per the rule in Section 2), and we ran a full 2D sweep over the CNN and rescue-gate thresholds together to lock in an operating point — landing on τ=0.225, down from the previous generation's 0.5.
We re-ran ArtifactBench from scratch — not incrementally, the full harness — on 5,458 tracks (2.6× our previous 2,103-track test set), covering all 22 AI generators we track.
| Metric | Prior generation | bigset |
|---|---|---|
| ArtifactBench F1 | 0.984 | 0.9921 |
| Real music flagged as AI | 4.2% | 0.38% |
| Detection @ fixed 1.5% FPR budget | 97.38% | 98.59% |
Holding the false-positive budget fixed and still gaining 1.2 points of detection is the tell that this is a genuine model improvement, not a threshold move dressed up as one. At τ=0.02 specifically, the raw counts were 4,334 true positives, 4 false positives, 65 false negatives, and 1,053 true negatives against 5,458 tracks — TPR 98.52%, FPR 0.378%.
The genre breakdown is the number we actually set out to get:
| Genre | F1 |
|---|---|
| Pop | 0.995 |
| Rock | 0.993 |
| Electronic (previously our weak point) | 0.988 |
| Lo-fi | 0.987 |
| Hip-hop | 0.987 |
| Acoustic | 0.987 |
Every genre we test now clears F1 0.986. Before bigset, electronic music was the one genre that couldn't — that's the "genre wall" this post is named for, and it's the first time we've been able to say all of them are above the line simultaneously.
On the held-out real-music set we've been tracking since Part 6 (excluding SoundCloud, to isolate the effect from the re-compression fix specifically), false positives fell from 5.98% to 1.48%. That's the same benchmark family behind the ~15% figure we disclosed last time — not a like-for-like single number, since rescue-gate and threshold work had already chipped away at it in between, but the trajectory is the same line, and it's now under 1.5% instead of in the double digits.
Promoting bigset to production surfaced exactly the failure mode Section 2 warns about — just in our own infrastructure rather than a customer-facing model. Our demo app was loading rescue-gate independently of the main inference pipeline, and during the promotion it kept pointing at the previous generation's rescue-gate checkpoint while the main CNN had already moved to bigset. Same mismatch class as the April incident, caught this time by the discipline in Section 2 rather than by a production FPR spike. We fixed the demo app to load rescue-gate the same way the main pipeline does, from a single source of truth.
A second, smaller gap: our research demo server and the actual customer-facing inference service are two different systems on two different machines, and a deployment check briefly assumed they were in sync when they weren't. We caught it during final verification the same day and corrected it — but it's a reminder that "the model is promoted" and "the model is serving traffic" are two separate claims that both need checking, every time. Both the free demo and the account-based batch service are now confirmed running bigset. Our RunPod serverless batch endpoint is out of scope for this promotion and is still serving the previous generation — that overlay is scheduled separately.
We also re-ran our monthly in-the-wild scan — the AI Music Index methodology we locked down earlier this month — through bigset. The mainstream and SoundCloud slices we'd already cleared came back clean again, as expected. The interesting change was Beatport, an electronic-music catalog we'd effectively excluded from scanning because the previous generation's false-positive rate there was unusable (a median AI-probability of 0.955 across the sample — the "false-positive explosion" that motivated Section 4). Under bigset, that collapsed to a corrected estimate of 0.07%, with the confidence interval including zero. Electronic-genre catalogs are, for the first time, a domain we can scan with a straight face.
Two ambient/sleep-music tracks flagged in an earlier index run were re-scored under a completely different model generation and came back more confidently flagged (0.842→0.892 and 0.633→0.759, not less) — evidence that signal was real and not an artifact of the old model's known weaknesses. We'll say plainly that the electronic-catalog sample size (43 tracks) is still too small for our stratified Rogan-Gladen confidence interval to exclude zero, so this particular slice doesn't clear our publication bar yet. We'd rather hold it than round up.
A second opinion is cheaper than a bigger model. rescue-gate moved the needle on hard-real FPR without touching the CNN at all — but only stays safe if it's versioned in lockstep with the model it's gating, a rule we now enforce as policy rather than convention after one incident on file.
Threshold bumps are a stopgap, not a diagnosis. The conditional adjustment we shipped for electronic music in mid-July bought time; it didn't explain anything. The actual fix only arrived once we traced the mechanism (repeated MP3 re-compression) and represented it in training.
"Promoted" and "serving traffic" are different claims. We caught a demo-app generation mismatch and a demo/production server mixup in the same rollout — both minor, both caught before they mattered, both reminders to verify the whole path, not just the model artifact.
What's left: the RunPod batch overlay for customers on that path, and pushing the electronic-catalog Index sample large enough to actually publish a number instead of holding one back.
ArtifactNet Research Team · July 2026
Upload any file or paste a URL — the same forensic pipeline described in this series.