EAIM @ AAAI 2026 · PMLR 303:1–15

TS-RaMIA

TS-RaMIA: Membership Inference Attacks for Symbolic Music Generation Models

A structure-aware, debiased audit that asks whether a symbolic music work was used to train a generation model—without turning confounders into false evidence.

Yuxuan Liu, Rui Sang, Peihong Zhang, Zhixin Li, Kunyang Zhang, Shengyuan He, Ye Li, Kaiyi Xu, Shengchen Li

First author: Yuxuan Liu

Read paper View code

Membership auditing

Can a creator test whether a piece was used to train a music generation model?

INPUTCandidate music work x

A symbolic sequence under audit

SCOREs(x)

A statistical membership-risk score

TESTH₀ / H₁

H₀: x ∉ Dtrain · H₁: x ∈ Dtrain

DECISIONThreshold fixed on validation data

Then evaluated on a held-out test set

Gray-box · forward-pass only

What the auditor can access—and what remains a deployment gap

Allowed

Submit a candidate symbolic-music sequence
Obtain per-token log-probabilities through teacher forcing
Know the documented tokenization scheme

Not required

No gradients
No model weights
No optimizer state
No access to the target training set

Deployment gap

Some commercial APIs do not expose per-token probabilities.
Generation-only interfaces need sampling-based approximations.
Those approximations increase query cost and are not the paper’s primary evaluation.

The evaluation trap

Why naïve sequence loss can look convincing for the wrong reason

Raw baseline AUC—Before structural-length control

Length-matched AUC—313 nearest-neighbor pairs

Conditionally calibrated AUC—Residualized against log structural length

LengthLonger works contain more opportunities for extreme loss values.
Structure countBar, position, and tempo counts can correlate with the split.
Event densityDense passages change the loss distribution independently of membership.

Without confounder control, an apparently successful attack may only be recognizing structural complexity.

Core insight

Leakage is not uniformly distributed across musical tokens

REMI

BarPositionTempoPitchVelocity

Bar, Position, and Tempo form the structural mask; note, pitch, and velocity tokens have different functions.

ABC

Header|:[]A–G

Headers before the first body line are excluded. Body-level meter, key, or voice changes remain under the paper’s rules; normalized newlines are not structural tokens.

Observation

Per-token NLL is the base observation. Tail statistics surface sparse local anomalies that a full-sequence average can dilute.

Decision rule

A directed, standardized meta-attacker produces the final score under cross-validation. A single raw NLL is not a membership probability.

TS-RaMIA pipeline

Five stages, one controlled audit score

Select a stage to inspect it on the unchanged Figure 1 from the paper.

Published TS-RaMIA Figure 1 showing data protocol, structural masking, sample-level NLL, debiasing, tail aggregation, and meta-attacker fusion — Published Figure 1, extracted from the final PMLR paper without redrawing. The transparent outline is a webpage annotation only.

Technical details

Likelihood extraction

Teacher forcing yields per-token NLL. Non-overlapping chunks exclude each chunk’s initial token because it lacks within-chunk context.

Debiasing

Nearest-neighbor length matching and conditional calibration isolate membership evidence from structural length.

Tail statistics

Three structural top-k scores use k ∈ {32, 64, 128}; three windowed p95 statistics capture local peaks.

Meta-attacker

Optional reverse/hierarchical features complete a nine-dimensional vector when available. Each fold applies z-score normalization before L2-regularized, class-weighted logistic regression.

Evaluation

Composer-stratified five-fold cross-validation produces out-of-fold predictions, reducing stylistic leakage between training and evaluation folds.

Published evidence

Inspect the controlled Table 1 results

This explorer exposes only aggregate results published in PMLR 303. It never produces a sample-level probability.

Analysis viewMethodMetric

TPR@1%FPR—

AUC—

TPR@1%FPR—

TPR@5%FPR—

TPR@10%FPR—

—

Controlled evaluation view; not a per-sample probability.

Table 1 · REMI Transformer results. AUC includes 95% confidence intervals.
View	Method	AUC [95% CI]	TPR@1%FPR	TPR@5%FPR	TPR@10%FPR

Published TS-RaMIA Figure 2 with ROC, low-FPR zoom, structural-score distribution, and checkpoint risk scan — **Published Figure 2**Rendered from the final paper and cropped only around the figure boundary. It is not a reconstructed curve and no values are digitized from it. Final Table 1 is the numerical reporting source.

Deployment-relevant evaluation

Why low false-positive rates matter

1,000

non-member works audited

1% FPR≈ 10false alarms

False accusations are costly, so rights-holder audits need evidence at a strict false-positive budget—not AUC alone.

At the 1% FPR operating point, the length-matched fusion result detects 14.6% of true members. That is recall at a fixed error budget, not the false-positive rate and not detection of most training samples.

The decision threshold must be selected on a development split, never tuned on the test set.

Ablations and falsification

What failed—and what that taught us

Table 2 · Length-matched ablations
Method	AUC ± std.	TPR@1%FPR

Top-64 is a trade-off

It balances variance against signal dilution; it was not chosen arbitrarily.

Windowed p95 adds a small lift

Local peaks help, but they do not replace structural top-k aggregation.

Equal-N preserves the trend

Performance near the full-piece result argues against piece length being the only signal.

Note-only attack

AUC —

Score inversion can recover approximately 0.68, but only if orientation is known in advance. It is not a usable attack as-is.

EVT tail modeling

AUC —

With 314 non-member samples, fitting was unstable and did not outperform simpler top-k aggregation.

Cross-representation transfer

Does the trend survive a different symbolic representation?

Main representationREMI Transformer

Conversion pathMAESTRO MIDI → MusicXML → ABC— · successful

Transfer representationABC / NotaGen

Raw AUC—

TPR@1%FPR—

Length-matched AUC——

Privacy–utility checkpoint scan

Membership risk accumulates as training progresses

Published Figure 2d showing attack AUC increasing while training perplexity falls across checkpoints — Figure 2(d), cropped from the final paper without changing the plotted content.

The same audit pipeline is applied to intermediate checkpoints. Attack AUC rises as training continues, indicating increasing membership risk.

The scan can inform selection of a lower-risk checkpoint when utility is adequate. The page does not invent checkpoint values that the text does not report.

Limitations and responsible use

Evidence the paper supports—and claims it does not establish

Supported by this paper

Forward-pass membership auditing
Symbolic-music structural tokens
Low-FPR controlled evaluation
REMI main evaluation
ABC/NotaGen transfer evidence

Not established by this paper

Generation-only commercial API performance
Proof of copyright infringement
Prompt-injection or jailbreak defense
End-to-end AI Agent security
Universal transfer to arbitrary music models
Large-scale production latency

TS-RaMIA provides statistical auditing evidence and should be combined with provenance, licensing records, metadata, and other supporting evidence.

Conceptual transfer / future direction — not evaluated in this paper

From structured music auditing to structured Agent auditing

Structural music tokensBar / Position / Tempo

Risk-critical Agent eventsTool calls / permission changes / external transmission / file writes

Sequence confoundersStructural length and event density

Trajectory confoundersTrajectory length / tool count / task difficulty / user type

Group-aware evaluationComposer-stratified folds

Agent evaluationUser-, task-, domain-, or attack-family-stratified splits

Strict operating pointTPR@1%FPR

Safety operating pointDetect unsafe actions without excessively blocking benign users

What transfers

Audit the security-relevant parts of a structured sequence.
Control confounders before claiming an attack works.
Evaluate at deployment-relevant low false-positive rates.
Separate exploratory scores from calibrated decisions.
Use group-aware splits to prevent evaluation leakage.

What does not transfer directly

Membership inference is not prompt-injection detection.
Symbolic-music structure is not identical to Agent semantics.
The paper does not evaluate Tool Use, RAG, Memory, or mobile-device actions.
Production systems may not expose token log-probabilities.
The proposed Agent extension is future work.

Paper and resources

Verify the primary sources

Official paper pagePMLR 303 Official PDFOpen final paper Project repositoryGitHub Conference slidesEAIM @ AAAI 2026

BibTeX

@InProceedings{pmlr-v303-liu26a,
  title = {TS-RaMIA: Membership Inference Attacks for Symbolic Music Generation Models},
  author = {Liu, Yuxuan and Sang, Rui and Zhang, Peihong and Li, Zhixin and Zhang, Kunyang and He, Shengyuan and Li, Ye and Xu, Kaiyi and Li, Shengchen},
  booktitle = {Proceedings of Machine Learning Research},
  pages = {1--15},
  year = {2026},
  volume = {303},
  publisher = {PMLR},
  url = {https://proceedings.mlr.press/v303/liu26a.html}
}

The audit question

Can a creator test training-set membership?

H₀x ∉ D_train

The candidate was not used for training.

H₁x ∈ D_train

The candidate was used for training.

Use caseCreator / rights-holder audit

A statistical signal, calibrated on validation data.

Gray-box access

Forward-pass evidence—without gradients or weights

AVAILABLEPer-token log-probabilities

Teacher forcing under a known tokenizer.

NOT NEEDEDNo gradients, model weights, optimizer state, or target training data

DEPLOYMENT GAPGeneration-only commercial APIs may not expose the required probabilities.

Sampling approximations add query cost and are not the paper’s primary result.

The evaluation trap

A strong raw AUC may be measuring structural complexity

Raw baseline AUC—Apparently useful

Control length + density

Length-matched AUC—Near chance

Structural lengthEvent densityFormatting / representation

Without confounder control, an apparently successful attack may only be recognizing structural complexity.

Core insight

Membership leakage is localized; uniform averages dilute it

REMI STRUCTURAL MASK

BarPositionTempoPitchVelocity

ABC STRUCTURAL MASK

Header|:[]A–G

Base observationPer-token NLL

Localized signalStructural tail statistics

Final decisionDirected, standardized out-of-fold meta-attacker

TS-RaMIA method

Mask → NLL → Debias → Tail → Meta-fusion

Published TS-RaMIA pipeline — Published Figure 1, extracted from the final PMLR paper without redrawing. The transparent outline is a webpage annotation only.

Top-k captures sparse leakage pockets.
Length matching and calibration remove spurious correlations.
Composer-stratified out-of-fold evaluation reduces stylistic leakage.

Controlled evidence

Performance at the strict low-FPR operating point

Baseline—AUC

— TPR@1%FPR

StructTail-64—AUC

— TPR@1%FPR

StructTail+Fusion—AUC

— TPR@1%FPR

Primary metric: TPR@1%FPR313 matched pairs95% confidence intervalsComposer-stratified evaluation

Falsification, transfer, and limits

What survived—and what did not

Ablation

Top-64 is the best trade-off.
Windowed p95 adds a small lift.
Equal-N retains the trend.

Negative results

Note-only is unusable without score inversion.
EVT adds complexity without gain.

Cross-representation

— AUC · — TPR@1%FPR

NotaGen under distribution shift—not a direct replication.

Deployment limit

The method requires per-token probabilities; generation-only API performance is not established.

Conceptual transfer / future direction — not evaluated in this paper

From structured music auditing to structured Agent auditing

Music structureBar / Position / Tempo

Risk-critical Agent eventsTool calls / permission changes / external transmission / file writes

ConfoundersLength / density / composer

Agent confoundersTrajectory length / tool count / task difficulty / user

Strict evaluationTPR@1%FPR

Safety objectiveDetect unsafe actions without overblocking benign users

What transfers

Structure-aware auditing · confounder control · low-FPR evaluation · calibrated decisions · group-aware splits.

What does not transfer directly

Membership inference is not prompt-injection detection. The paper does not evaluate Tool Use, RAG, Memory, or mobile-device actions, and production systems may not expose token log-probabilities.