FG 2026 · Skeleton-based sign recognition

Shape each class as a connected region.

LBOR moves beyond point and pair objectives. It regularizes the within-class geometry of feature space while preserving a margin between class centers.

Peihong Zhang · Yuxuan Liu · Zhixin Li · Rui Sang · Shengchen Li

+1.62 pp
SKIM Top-1 · WLASL2000
+1.3 pp
Siformer Top-1 · NMFs-CSL
0
extra inference cost
Method in motion

Repair the class manifold, then protect its margin.

Animated reading: signer-specific fragments contract through class-restricted graph edges, while EMA class centers retain a minimum distance between different signs.

LBOR feature geometry regularizationFragmented same-class feature clusters are connected through a class-specific nearest-neighbor graph and Laplacian energy, while a center-margin term separates different sign classes.BEFORE · POINT / PAIR VIEWOne class, several centroidsSigner variation fragments the regionSAME LABEL · DISCONNECTED MODESLBOR · TRAINING OBJECTIVEClass-specific kNN graphSame-class edges only · k = 5Llap smooths local geometry+ LCM PRESERVES CENTER MARGINAFTER · CONNECTED REGIONSSmooth within, apart betweenConnectivity + center separationMARGIN MNO ARCHITECTURE CHANGE · NO INFERENCE COSTNovelty: the objective constrains class-level feature geometry rather than only isolated samples or pairs.
Figure · LBOR geometry repair and center separationCircles denote one sign class; squares denote another. Motion illustrates the loss objective, not a test-time module.
01 · Question

Why can the same sign split into several feature clusters?

Signer, speed, and viewpoint variation can create multiple local centroids for one class. Point- and pair-level objectives do not explicitly reconnect that fragmented geometry.

Intra-class

One label, many local modes

Examples from different signers can cluster separately even when they represent the same lexical sign.

Inter-class

Fine-grained neighbors

Visually similar signs require local smoothness without collapsing the gap between distinct classes.

Compatibility

Keep the backbone

The regularizer attaches to existing objectives and requires no architectural change.

02 · Method

Approximate manifold smoothness inside each class.

Graph

Class-restricted neighbors

For each mini-batch, normalized features form same-class kNN graphs with RBF affinities.

Energy

Within-class Laplacian

The Dirichlet-style loss penalizes distance along graph edges, encouraging connected and smoother class regions.

Margin

Center separation

EMA-updated class centers receive a hinge margin so smoothing does not erase between-class structure.

03 · Evidence

Improvements repeat across backbones, splits, and datasets.

SKIM · WLASL2000

45.28 → 46.90

Per-instance Top-1; Top-5 rises from 73.15 to 74.70.

Siformer · WLASL100

80.2 → 82.2

Per-instance Top-1; per-class Top-1 rises from 81.0 to 83.0.

Siformer · NMFs-CSL

76.8 → 78.1

Per-instance Top-1; Top-5 rises from 94.8 to 95.7.

Values transcribed from IEEE Table I. Table II shows that either term alone underperforms the combined LBOR objective on SKIM.

04 · Boundary

Geometry-aware, but still a controlled skeleton-only study.

Evaluation frame

  • WLASL: 21,083 samples, over 100 signers, and a 2,000-word vocabulary.
  • NMFs-CSL: 32,010 samples across 1,067 words.
  • HMA, SignBERT, SKIM, and Siformer variants under one skeleton-only protocol.

Claim boundary

  • Graph construction adds training-time computation but no inference modules.
  • The study uses 2D keypoints extracted with MMPose.
  • Absolute reproduced SignBERT results may differ from its original implementation.
Method visual · paper-based

Make the latent class a connected region, not a pile of points.

This interpretation follows the paper’s training objective: class-specific graphs connect same-label features, Laplacian energy smooths the local geometry, and a center margin keeps neighboring signs apart. The publisher PDF is not redistributed here.

Paper-based visual interpretation of LBOR: skeleton-sign observations flow into a connected same-class feature manifold separated from another class.
LBOR · paper-based visual interpretation of the training objective.It illustrates class-restricted graph regularization and center separation; it is not a test-time module.
Input skeleton sequences with signer variationWithin class kNN graph + Laplacian energyBetween classes EMA center margin

Inspect the complete method and IEEE result tables.

Open IEEE paperPublication list