We propose a tied prototype model TPM that improves few-shot medical segmentation via probabilistic modeling.

By Hyeongji Kim, Postdoctoral Research Fellow at SFI Visual Intelligence

We develop this model as a means for challenging few-shot medical image segmentation, a task which requires segmenting new anatomical structures using only a small number of annotated support examples.

Prototype-based methods are commonly adopted for this task. These methods summarize support annotations into representative feature prototypes and classify query pixels based on their similarity to these prototypes.

However, unlike the foreground, the background class in medical images can be highly heterogeneous, often comprising many unrelated tissues, organs, and anatomical structures. This makes it inherently difficult to represent the background using a fixed number of prototypes.

As illustrated in the figure below, the key idea of TPM is to use a tied (shared) prototype location for both foreground and background class distributions, while enforcing different standard deviations.

While using the same center for both foreground and background may appear counterintuitive, this yields a useful inside–outside (IO) classification behavior: features closer to the prototype (p) are assigned higher foreground probabilities, whereas features farther from the prototype are assigned lower probabilities.

This provides a principled probabilistic framework for separating the typical foreground class from heterogeneous background regions, which is difficult to achieve with a standard prototype model.

Building on this formulation, TPM can be naturally extended to multiple prototypes and multi-class classification. First, TPM supports multiple prototypes via a Gaussian mixture model, which helps capture intra-class variation within each foreground class. Second, it enables the simultaneous segmentation of multiple foreground classes, as well as multi-foreground training, extending beyond binary foreground–background segmentation. Furthermore, we propose ideal class priors to refine the decision boundary, thereby increasing segmentation accuracy.

Experiments on abdominal MRI and CT few-shot segmentation demonstrate the effectiveness of each component. Using estimated ideal class priors improves accuracy by refining the decision boundary. Multi-prototype modeling further enhances performance by capturing richer foreground variation. Additionally, multi-foreground training improves representation learning for multi-class segmentation.

In summary, TPM establishes IO-based prototypical few-shot medical segmentation within a probabilistic framework, enabling multiple prototypes, multiple classes, and adaptive class priors.

Common prototype-based medical image few-shot segmentation (FSS) methods model foreground and background classes using class-specific prototypes. However, given the high variability of the background, a more promising direction is to focus solely on foreground modeling, treating the background as an anomaly—an approach introduced by ADNet. Yet, ADNet faces three key limitations: dependence on a single prototype per class, a focus on binary classification, and fixed thresholds that fail to adapt to patient and organ variability. To address these shortcomings, we propose the Tied Prototype Model (TPM), a principled reformulation of ADNet with tied prototype locations for foreground and background distributions. Building on its probabilistic foundation, TPM naturally extends to multiple prototypes and multiclass segmentation while effectively separating non-typical background features. Notably, both extensions lead to improved segmentation accuracy. Finally, we leverage naturally occurring class priors to define an ideal target for adaptive thresholds, boosting segmentation performance. Taken together, TPM provides a fresh perspective on proto typebased FSS for medical image segmentation. The code can be found athttps://github.com/hjk92g/TPM-FSS.

‍