ICASSP 2026 · Oral · Information Forensics and Security
LSA-Probe
Membership Inference Attack Against Music Diffusion Models via Generative Manifold Perturbation
We replace a noisy endpoint signal with a behavioral stability score: the minimum time-normalized latent perturbation required to cross a fixed perceptual degradation threshold.
Xi’an Jiaotong-Liverpool University, Suzhou, China
01
Audit question
What are we auditing?
Given a candidate audio clip and an audited music diffusion model, estimate whether the clip was used during training.
Statistical audit signal — not data extraction, model inversion, or standalone legal proof.
02
Background bridge
Diffusion models in 60 seconds
Before discussing membership, we need one piece of background: a diffusion model learns to reverse a gradual noising process.
x0
Add Gaussian noise according to timestep t
xt
Rt(·; θ)
x̂0
Forward process
xt = √ᾱtx0 + √(1−ᾱt)ε, ε∼𝒩(0,I)
The forward process mixes clean audio with Gaussian noise according to timestep t. A trained diffusion model learns a reverse mapping that progressively removes this noise.
Reverse mapping
x̂0 = Rt(xt; θ)
LSA-Probe does not only inspect the final reconstruction error. It intervenes at an intermediate state and observes how stable the learned reverse mapping is around that candidate.
The paper finds the strongest membership separation around the middle of the reverse trajectory, with t_ratio=0.6 used in the main setting. This is an experimental finding in the evaluated settings, not a universal property of every diffusion model.
Method in one glance
What does LSA-Probe change?
Stability instead of endpoint loss
Replace a noisy single reconstruction signal with the minimum intervention required to destabilize the local reverse process.
Time-normalized probing
Scale the perturbation by the natural forward-noise level so that budgets are comparable across diffusion timesteps.
Minimum adversarial cost
Use inner PGD to search for a degradation direction and outer bisection to find the smallest successful budget.
The resulting score does not require an explicit likelihood or a separately trained shadow model.
03
Access assumptions
A strong, explicit white-box threat model
LSA-Probe is designed for a developer or authorized auditor who can inspect and differentiate through the model’s reverse process.
- Can run forward diffusion and deterministic reverse sampling.
- Can compute gradients with respect to an intermediate diffusion state.
- Does not need shadow models or member labels from the target model.
- Does need an independent non-member development set for calibration.
04
Why the baseline is weak
Why endpoint reconstruction error is insufficient
A single reconstruction loss is a noisy endpoint observation, especially in the strict low-FPR regime that matters for high-cost audits.
Reconstruction error changes with the diffusion state.
Injected noise and sampling choices alter the endpoint.
Timbre and musical complexity can confound raw loss.
Waveform MSE need not track what listeners perceive.
Instead of asking whether one reconstruction has a small error, LSA-Probe asks how much worst-case, time-normalized intervention is required to destabilize the local reverse process.
Correct diffusion-time direction
05
Latent stability hypothesis
Measure the intervention required to break local stability
Training members empirically occupy more stable local regions of the learned reverse mapping. They require a larger bounded perturbation to reach the same perceptual degradation target.
Geometric intuition
What do we mean by a generative manifold?
Audio latents live in a high-dimensional ambient space, but plausible musical signals occupy a structured region shaped by factors such as pitch, rhythm, timbre and dynamics. A trained diffusion model learns an approximate reverse mapping around this data-supported region.
In LSA-Probe, “generative manifold” does not mean that we analytically recover the true data manifold. It is a local geometric interpretation of how stable the learned reverse process is around a candidate intermediate state.
We therefore probe the size of the local neighborhood that can be perturbed before the reconstructed audio crosses a fixed perceptual degradation threshold.
Ambient spaceAll possible latent states.
Learned generative supportStructured states mapped toward plausible audio.
Local stability radiusNormalized budget required to reach degradation threshold τ.
Interpretation boundaryThe surface and neighborhood radii are conceptual projections. LSA-Probe measures empirical local stability; it does not recover a literal manifold or prove that every training member is more stable than every non-member.
First-order intuition: why should members require a larger budget?Expand only when needed
x̂0 = Rt(xt; θ)
x̂0δ = Rt(xt + δt; θ)
Rt(xt+δt;θ) ≈ Rt(xt;θ) + JRt(xt)δt
The local Jacobian describes how sensitive the reverse mapping is to a small intervention around the candidate state. If the learned mapping is locally smoother around a training member, the same perturbation budget produces less output degradation. Reaching the fixed threshold τ therefore requires a larger budget, producing a larger Cadv.
Lower local sensitivity → larger required budget → higher Cadv → more member-like
A critical distinction
Two thresholds, two different jobs
Perceptual degradation threshold τ
D(x̂0, x̂0δ) ≥ τ
Defines the degradation target. It is calibrated as P95 on an independent non-member development set at ηref=0.05. It is not the membership cutoff.
Adversarial cost Cadv
Measures the minimum time-normalized budget required to reach τ for each candidate.
Membership decision cutoff
A separate cutoff on Cadv defines the operating point, such as FPR=1%, and yields TPR@1% FPR.
06
Two-loop adversarial probe
Inner PGD finds direction; outer bisection finds cost
The method searches for a strong local degradation direction at each budget, then finds the smallest successful budget.
- 01
Forward diffuse the candidate to an intermediate state.
- 02
Inject a time-normalized bounded perturbation.
- 03
Use inner PGD to find the strongest degradation direction.
- 04
Use outer bisection to find the minimum successful budget.
δt = σt δ̃ σt = √(1−ᾱt)
The raw scale of an intervention is not comparable across diffusion timesteps because the forward noise variance changes with t. Scaling δ̃ by σt places the perturbation on the timestep’s natural noise scale.
Technical details
Algorithm walkthrough · Illustrative control flow · Not a measured sample
Watch the two loops coordinate
Ready. Start with the full admissible interval.
One probe, two diffusion familiesExpand only when needed
DiffWave
Waveform diffusion
- Start from waveform x₀.
- Forward-noise it to waveform state xₜ.
- Apply δₜ=σₜδ̃ directly in waveform space.
- Run the differentiable reverse operator.
- Measure degradation between reconstructed waveforms.
MusicLDM
Latent diffusion
- Encode the waveform with the VAE encoder: z₀=Enc(x₀).
- Forward-noise it to latent state zₜ.
- Apply the time-normalized perturbation in latent space.
- Run the latent reverse process.
- Decode with the frozen decoder.
- Measure perceptual distance on the resulting waveforms.
x̂0 = Dec(Rt(zt; θ))
x̂0δ = Dec(Rt(zt + σtδ̃; θ))
- The forward noise is fixed for each paired evaluation so that the measured difference reflects the added intervention rather than a new random noise realization.
- When conditioning is used, the text prompt and guidance remain fixed between the clean and perturbed reverse passes.
07
Published aggregate results
Evidence at the strict low-FPR operating point
All values below are transcribed from Table 1. No per-sample distributions or ROC points are reconstructed.
Why focus on TPR at only 1% FPR?Expand only when needed
Membership auditing is a high-cost decision problem. When the non-member population is much larger than the member population, even a seemingly small false-positive rate can produce many false alerts. The operating point must therefore control false positives before recall is interpreted.
Illustrative arithmetic—not a paper experimentAmong 10,000 non-members, an FPR of 1% corresponds to approximately 100 false positives.
- Best TPR@1% FPR is 20%.
- Roughly 80% of members remain missed.
- Precision depends on the true member base rate.
- A membership score is not standalone legal evidence.
What is LSA-Probe compared against?Expand only when needed
| Method | Signal | Limitation or difference from LSA-Probe |
|---|---|---|
| Endpoint loss | Denoising or reconstruction loss measured at a timestep or at the endpoint. | A single loss can be confounded by timestep, noise seed, content complexity and perceptual mismatch. |
| Trajectory / PIA / PIAN | Distance between a recovered diffusion-trajectory point and the model-predicted trajectory point. | Measures trajectory mismatch rather than the minimum worst-case intervention required to cross a perceptual threshold. |
| SecMI | A diffusion-process membership signal used as an existing comparison baseline. | Does not use LSA-Probe’s time-normalized adversarial-cost construction. |
| LSA-Probe | The minimum time-normalized perturbation budget required to exceed a fixed perceptual degradation threshold. | Measures local generative stability using inner PGD and outer bisection. |
All baselines were matched within ±5% per-sample wall-clock time on an A100-80GB.
Evidence Explorer
A controlled view of the same four Table 1 rows.
Verified evidence data could not be loaded. The page will not manufacture a replacement.
| Model | Dataset | Best Baseline TPR@1% / AUC | LSA-Probe TPR@1% / AUC | Δ TPR | Δ AUC |
|---|---|---|---|---|---|
| MusicLDM | MAESTRO | 0.10 (0.07–0.12) / 0.58±0.02 | 0.13 (0.10–0.15) / 0.61±0.03 | +0.03 | +0.03 |
| MusicLDM | FMA-Large | 0.08 (0.05–0.10) / 0.56±0.01 | 0.14 (0.10–0.16) / 0.59±0.02 | +0.06 | +0.03 |
| DiffWave | MAESTRO | 0.12 (0.09–0.15) / 0.63±0.02 | 0.20 (0.16–0.24) / 0.67±0.02 | +0.08 | +0.04 |
| DiffWave | FMA-Large | 0.11 (0.08–0.14) / 0.62±0.02 | 0.18 (0.14–0.22) / 0.66±0.02 | +0.07 | +0.04 |
Published Figure 2
What the ablations support
These are the original embedded raster panels extracted from the supplied paper PDF, not reconstructed curves.
08
Reliability and limitations
How the evidence was controlled
The evaluation spans waveform and latent diffusion, two music domains, leakage controls, uncertainty estimates and compute-matched baselines.
Models
DiffWave · waveform DDPM
MusicLDM · VAE latent diffusion
Datasets
MAESTRO v3 · solo piano
FMA-Large · multi-genre music
Main setup
Deterministic DDIM · t_ratio=0.6 · p=2 · ηmax=0.8 · CDPAM · 22.05 kHz training audio · in-graph resampling to 16 kHz for metrics.
Split controls
Piece/artist splits for MAESTRO; track/artist splits for FMA; Chromaprint + LSH duplicate and cover checks with manual review.
Statistical protocol
DeLong AUC intervals; sample-level bootstrap elsewhere; 10,000 resamples; Holm–Bonferroni correction.
Compute accounting
B=10 outer × K=12 inner, roughly (K+2)B reverse passes per sample; baselines matched within ±5% wall-clock on an A100-80GB.
Scientific boundary
Limitations and open questions
- Strong white-box access is required: parameters, sampler, intermediate states and gradients.
- Repeated PGD, restarts and bisection make the audit computationally expensive.
- The best TPR@1% FPR is still only 20%.
- Evaluation covers two model families and two music datasets.
- Threshold calibration depends on an independent non-member development set.
- Cross-model and cross-domain threshold transfer remain underexplored.
- Results do not directly generalize to text LLMs or multimodal agents.
- A membership score is not standalone legal evidence.
- “Generative manifold” is an empirical local-geometric description, not an analytically recovered manifold.
- Defenses need further study: differential privacy, regularization, deduplication, audit-aware training and reduced memorization.
Future evaluation
Defensive implications and open evaluation questions
Not experimentally evaluated in the current paperTraining-data hygiene
Deduplication, artist-aware splitting and provenance control reduce accidental duplication and leakage, but do not by themselves guarantee the absence of memorization.
Memorization reduction
Regularization, early stopping, audit-aware training and differential privacy are potential directions for reducing membership leakage.
Access control
LSA-Probe requires parameters, intermediate states and gradients. Restricting access changes the practical threat surface, although it does not eliminate the underlying memorization risk.
Adaptive evaluation
A meaningful defense evaluation must consider an auditor or attacker who knows the defense and can recalibrate the probe, threshold or optimization procedure.
09
Future direction
From Generative-Model Auditing to Agent Safety
A methodological transfer, not an experimental result of LSA-Probe.
LSA-Probe
Define the protected training-data asset
Agent Safety
Define protected user data, memory, retrieved content and tool permissions
LSA-Probe
Specify white-box developer-side access
Agent Safety
Build authorized internal red-team and audit infrastructure
LSA-Probe
Measure the minimum intervention needed to destabilize a trajectory
Agent Safety
Measure the minimum bounded intervention needed to change a decision or tool action
LSA-Probe
Evaluate at strict low-FPR operating points
Agent Safety
Control false alarms before blocking legitimate tasks or users
LSA-Probe
Treat membership as statistical evidence
Agent Safety
Preserve audit trails and uncertainty rather than issuing absolute claims
What transfers
Asset definition, authorized access, bounded intervention, strict operating points and uncertainty-aware audit trails.
What does not transfer directly
Audio latents are not LLM hidden states; diffusion trajectories are not agent plans. LSA-Probe does not detect prompt injection, stop jailbreaks, solve tool authorization, provide trusted execution or establish effectiveness for mobile agents.
10
Paper and resources
Verify the primary sources
The public repository currently exposes a project README rather than a complete implementation or checkpoints.
BibTeX
@inproceedings{liu2026membership,
title={Membership Inference Attack Against Music Diffusion Models via Generative Manifold Perturbation},
author={Liu, Yuxuan and Zhang, Peihong and Sang, Rui and Li, Zhixin and Tan, Yizhou and Cai, Yiqiang and Li, Shengchen},
booktitle={2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
year={2026}
}Audit question
What are we auditing?
Given a candidate audio clip and an audited music diffusion model, estimate whether the clip was used during training.
Background bridge
Diffusion models in 60 seconds
x0
+ ε
xt
Rt
x̂0
xt = √ᾱtx0 + √(1−ᾱt)ε
x̂0 = Rt(xt; θ)
Access assumptions
A strong, explicit white-box threat model
Why the baseline is weak
Why endpoint reconstruction error is insufficient
Reconstruction error changes with the diffusion state.
Injected noise and sampling choices alter the endpoint.
Timbre and musical complexity can confound raw loss.
Waveform MSE need not track what listeners perceive.
Instead of asking whether one reconstruction has a small error, LSA-Probe asks how much worst-case, time-normalized intervention is required to destabilize the local reverse process.
Latent stability hypothesis
Latent stability and adversarial cost
Audit score
Cadv(x) = min η
Minimum perturbation budget that reaches the fixed perceptual degradation target.
Cadv(member) > Cadv(non-member)
Empirical tendency in the evaluated settings
The surface and neighborhood radii are conceptual projections. LSA-Probe measures empirical local stability; it does not recover a literal manifold or prove that every training member is more stable than every non-member.
Two-loop adversarial probe
Inner PGD finds direction; outer bisection finds cost
δt=σtδ̃
- 01
Forward diffuse the candidate to an intermediate state.
- 02
Inject a time-normalized bounded perturbation.
- 03
Use inner PGD to find the strongest degradation direction.
- 04
Use outer bisection to find the minimum successful budget.
Published aggregate results
Evidence at the strict low-FPR operating point
Reliability and limitations
Reliability, limitations and defenses
Data and split controls
MAESTRO piece/artist split · FMA track/artist split · Chromaprint + LSH + manual review.
Statistical protocol
DeLong for AUC · 10,000 bootstrap resamples elsewhere · Holm–Bonferroni correction.
Compute fairness
B=10 outer · K=12 inner · roughly (K+2)B reverse passes · baselines matched within ±5% wall-clock on A100-80GB.
Future direction
From Generative-Model Auditing to Agent Safety
A methodological transfer, not an experimental result of LSA-Probe.
Protected assetTraining audio
Agent SafetyUser data, memory, retrieved content and tool permissions
Bounded interventionDestabilize a diffusion trajectory
Agent SafetyChange an agent decision or tool action
Strict operating pointTPR@1% FPR
Agent SafetyAvoid blocking legitimate users or safe tasks because of false alarms
What transfers
Asset definition · authorized internal auditing · bounded worst-case intervention · strict low-FPR evaluation · uncertainty-aware audit trails.
What does not transfer directly
Audio latents are not LLM hidden states; diffusion trajectories are not agent plans. The paper does not establish protection against prompt injection, jailbreaks, unsafe tool calls or mobile-agent attacks.