DETR (Carion et al., 2020) laid the foundations for Detection Transformers. DETR adopts the machine translation Transformer’s (Vaswani et al., 2017) encoder-decoder architecture to create a streamlined end-to-end object detection pipeline. However, in contrast to the machine translation Transformer, DETR predicts the final set of bounding boxes in parallel rather than in an autoregressive manner. DETR employs a ResNet-based backbone to generate a low resolution representation of the input image, which is subsequently flattened and refined in $N$ Transformer encoder blocks. The refined input sequence is in the next step forwarded to the decoder’s cross-attention module to refine a set of object queries. After the object queries have been refined in $N$ Transformer decoder blocks, they are fed into a classification and a bounding box regression head, resulting in predicted bounding boxes and class scores. During training, a Hungarian algorithm enforces one-to-one matches, which are particularly suitable for set prediction tasks, based on a set of weighted criteria. Although DETR’s simple design achieves promising 2D detection results on par with a Faster R-CNN baseline, it suffers from high computational complexity, low performance on small objects, and slow convergence.

Deformable DETR (Zhu et al., 2021) improved DETR’s detection performance while simultaneously reducing the model’s computational complexity and training time by introducing the deformable attention mechanism. The concept of deformable attention has been derived from the concept of deformable convolution (Dai et al., 2017), which increases the modeling ability of CNNs by leveraging learned sampling offsets from the original grid sampling locations. The deformable attention module utilizes the learned sparse offset sampling strategy introduced by deformable convolutions by allowing an object query to solely attend to a small fixed set of sampled key points and combines it with the relation modeling ability of the attention operation.

It is worth mentioning that many Detection Transformer variants tried to improve DETR’s initial concept over time. For example, Efficient DETR (Yao et al., 2021) eliminated DETR’s need for an iterative object query refinement process, Conditional DETR (Meng et al., 2021) introduced a conditional cross-attention module, DAB-DETR (Liu et al., 2022a) updated the object query formulation to represent stronger spatial priors, DN-DETR (Li et al., 2022) presented a denoising training strategy, and DINO (Zhang et al., 2022) combined and improved important aspects like denoising training, query initialization, and box prediction.

Even though 3D medical detection is a longstanding topic in medical image analysis (Criminisi et al., 2009, 2010), most research currently focuses on the more dominant discipline of semantic segmentation (Isensee et al., 2021; Navarro et al., 2021, 2019). This is because most clinically relevant tasks require voxel-vise predictions. Therefore, prior work on deep learning-based 3D medical detection remains relatively limited.

Following DETR, Focused Decoder relies on a feature extraction backbone to create a lower resolution representation of the input CT image $x\in\mathbb{R}^{H\times W\times D}$, which mitigates the high computational complexity of the attention operation. This feature extraction backbone is depicted in Fig. 1 and is given, similar to Retina U-Net, by an FPN. It should be mentioned that although the feature extraction backbone serves as a CNN-based feature encoder, we refrain from referring to it as an encoder to avoid confusion between the Transformer’s encoder and the FPN. The FPN consists of a CNN-based down-sampling branch (down), an up-sampling branch (up), and a final output projection (out). The up-sampling branch incorporates the down-sampled multi-level feature maps $\{x_{Cl}\}_{l=2}^{5}$, where $x_{Cl}\in\mathbb{R}^{24\cdot 2^{l}\times H/2^{l}\times W/2^{l}\times D/2^{l}}$, via lateral connections based on $1\times 1\times 1$ convolutions and combines them with up-sampled feature maps of earlier stages. A final output projection acts as an additional refinement stage, fixing the channel dimension to $d_{\text{hidden}}$. Therefore, the FPN outputs a set of refined multi-level feature maps $\{x_{Pl}\}_{l=2}^{5}$, where $x_{Pl}\in\mathbb{R}^{d_{\text{hidden}}\times H/2^{l}\times W/2^{l}\times D/2^{l}}$, encoding semantically strong information by leveraging the top-down inverted high- to low-level information flow. Even though Focused Decoder solely processes the feature map $x_{P2}$, we require multi-level feature maps for our experiments. This is because our experiments rely on the same feature extraction backbone to ensure maximum comparability of results (see Fig. 4 and Table 5).

In this paper, we generate custom atlases and refer to them as anatomical region atlases. Relying on the positional consistency assumption, we determine class-specific RoIs, describing the minimum volumes in which all instances of a particular anatomical structure are located. A small subset of RoIs is shown in Fig. 2 (left). Additionally, we estimate for each labeled anatomical structure the median, minimum, and maximum bounding box size, which will be necessary for the query anchor generation process and the concept of relative offset prediction. Given that the test set should only be used to estimate the final performance of the model, we estimate RoIs and bounding box sizes solely based on instances contained in the training and validation sets.

Since medical CT datasets typically contain different labeled anatomical structures, FoVs are inconsistent across datasets. Therefore, we generate dataset-specific anatomical region atlases. In general, however, existing anatomical atlases could be adjusted to fit the datasets at use based on a few anatomical landmarks (Potesil et al., 2013; Xu et al., 2016).

Due to dataset-specific inconsistencies and the substantial amount of patient-to-patient variability with regard to positions and sizes of anatomical structures, it is often difficult to ensure that datasets consist of images of exactly the same FoV. This is reflected in the fact that, on average, RoIs occupy 25 times larger volumes than anatomical structures. Therefore, Focused Decoder further subdivides RoIs by assigning to each class-specific RoI in our anatomical region atlas 27 fixed class-specific query anchors in the format $(c_{x},c_{y},c_{z},h,w,d)$. We denote query anchors by $q_{\text{anchors}}\in\mathbb{R}^{\#\text{classes}\times 27\times 6}$. While the spatial locations of query anchors are defined by uniformly spaced positions in their corresponding RoIs, their sizes are governed by the class-specific median bounding box sizes contained in the anatomical region atlas. Generated query anchors associated with the left adrenal gland’s RoI are shown in detail in Fig. 2 (right).

First, it should be mentioned that comparability of performances is of utmost importance and, due to the lack of 3D anatomical structure detection benchmarks, challenging to achieve. Therefore, we adapt and directly integrate detectors featured in this work into the same detection and training pipeline for a fair and reproducible comparison. For example, all featured detectors were trained until convergence on a single Quadro RTX 8000 GPU using the same AdamW optimizer, step learning rate scheduler, data augmentation techniques, and FPN feature extraction backbone. In addition, we tried to keep the configurations of featured Detection Transformers as similar as possible. We use the same head networks, set $N$ to three, $d_{\text{hidden}}$ to 384, and $d_{\text{FFN}}$ to 1024. By doing so, we ensure maximal comparability of results. Precise information about training details, hyperparameters, and individual model configurations is available at https://github.com/bwittmann/transoar. To facilitate understanding of our experimental setup, Fig. 4 depicts an overview of all featured detectors. Next, we briefly introduce the Detection Transformer baselines, which we adapted for 3D anatomical structure detection, and our RetinaNet variant.

Due to the lack of 3D anatomical structure detection benchmarks, we conduct experiments on two CT datasets of well-defined FoV that were originally developed for the task of semantic segmentation, namely the VISCERAL anatomy benchmark (Jimenez-del Toro et al., 2016) and the AMOS22 challenge (Ji et al., 2022). We transform their voxel-wise annotations into bounding boxes and class labels in the dataloader. Dataset-specific spatial sizes, approximated resolutions, and sizes of the respective training, validation, and test sets are along with the total number of subjects reported in Table 1.

The metric commonly used to evaluate object detection algorithms is the mean average precision (mAP). The mAP metric, described in (3), is defined as the mean of a selected subset of average precision (AP) values. We report $\text{mAP}_{\text{coco}}$, which is calculated based on AP values evaluated at IoU thresholds $\mathbb{T}$ = {0.5, 0.55, …, 0.95}.

To determine detection performance related to structure size, we introduce $\text{mAP}_{\text{coco}}^{\text{S}}$, $\text{mAP}_{\text{coco}}^{\text{M}}$, and $\text{mAP}_{\text{coco}}^{\text{L}}$. To this end, we categorize classes based on the volume occupancy of their median bounding boxes into the subsets S (small), M (medium), and L (large), which rely on the volume occupancy ranges of [0.0%, 0.5%), [0.5%, 5.0%), and [5.0%, 100%], respectively. Subsequently, we reevaluate $\text{mAP}_{\text{coco}}$ restricted to solely classes in the individual subsets. The dataset-specific subsets determined based on the above reported volume occupancy ranges are shown in Table 2. It should be mentioned that although both datasets have anatomical structures in common, they might be assigned to different subsets due to varying FoVs. The difference in FoV across both datasets can be observed in Fig. 7 (left).

Table 3 lists quantitative results of all detectors featured in this work. Essentially, our findings remain consistent across both datasets. Focused Decoder, possessing the least amount of trainable parameters, significantly outperforms the Detection Transformer baselines, namely DETR and Deformable DETR, and performs close to the state-of-the-art detector RetinaNet.

This demonstrates that a naive adaption of 2D Detection Transformers to the medical domain is definitely not sufficient to overcome the problem of data scarcity. However, Focused Decoder’s strong detection performances reveal that reducing the complexity of the relation modeling and thus the 3D detection task alleviates the need for large-scale annotated datasets, converting Detection Transformers into competitive and lightweight (approximately $20\%$ fewer parameters compared to RetinaNet) 3D medical detection algorithms. Focused Decoder not only performs close to or on par with its CNN-based counterpart but also provides the most accurate predictions for larger structures. On the other hand, RetinaNet outperforms Focused Decoder on smaller structures, overcoming the Detection Transformer’s well-known shortcoming of poor small object detection performance, which is subject to open research.

Even though the overall detection performance on the VISCERAL anatomy benchmark turns out to be stronger, Focused Decoder manages to narrow the gap to RetinaNet on the AMOS22 challenge. The generally stronger performances on the VISCERAL anatomy benchmark can be attributed to the absence of symmetrical and easy to detect structures such as the lungs or the rectus abdominis in the AMOS22 challenge. The narrowed performance gap, however, primarily results from the difference in FoV between both datasets. This is because the reduced FoV of the AMOS22 challenge (see Fig. 7 (left)) leads to an artificial increase in structure size, which is most beneficial for Focused Decoder.

To estimate the statistical significance of the performance differences between Focused Decoder and RetinaNet, we determined p-values using the Wilcoxon signed-rank test based on bootstrapped subsets. We indicate statistically significant performance differences in Table 3 (see footnote).

Explainability of results is of exceptional importance in the context of medical image analysis. While identifying reasons behind predictions produced by CNN-based detectors is often cumbersome and requires additional engineering susceptible to errors, Focused Decoder’s attention weights facilitate the accessibility of explainable results. Therefore, we particularly probe into intra-anatomical and inter-anatomical explainability by analyzing cross- and self-attention weights.

Qualitative results in the form of predicted bounding boxes of the two best-performing architectures, RetinaNet and Focused Decoder, are compared to the ground truth in Fig. 7. One can observe that Focused Decoder exhibits exceptional detection performance on larger structures such as the liver or the aorta, while RetinaNet remains superior in detecting small structures such as the adrenal glands.

In this subsection, we investigate the importance of Focused Decoder’s design choices by conducting detailed ablations on the validation set of the VISCERAL anatomy benchmark. Table 4 displays $\textrm{mAP}_{\textrm{coco}}$ values produced by Focused Decoder’s base configuration (first row) and three additional configurations (second to fourth row), omitting different design choices.

To evaluate the impact of our proposed query anchors, we completely omit them in the second configuration, resulting in an $\textrm{mAP}_{\textrm{coco}}$ decrease of 1.15. This proves the importance of demystifying object queries and their associated query embeddings by assigning them to precise spatial locations and thus generating diverse class-specific predictions, overcoming the issue of patient-to-patient variability. The configuration presented in the third row additionally reduces the amount of object queries per class from 27 to one. Based on the $\textrm{mAP}_{\textrm{coco}}$ reduction of 2.70, we argue that having only one object query per class fails to cover all class-specific object variations and hence leads to performance decreases. Next, we deactivate the focused cross-attention module’s restriction to RoIs in the fourth row. As expected, detection performance drastically diminishes, which is reflected in an $\textrm{mAP}_{\textrm{coco}}$ delta of -5.75. The result of this ablation study repeatedly demonstrates the necessity of simplifying the relation modeling task to achieve competitive detection performances.

Since the FPN outputs a set of refined multi-level feature maps $\{x_{Pl}\}_{l=2}^{5}$, we experiment with different input sequences represented by flattened feature maps of different resolutions and report our findings in Table 5.

One can observe that feature maps of higher resolution are clearly beneficial for detection performance, as they contain more fine-grained details, which in turn leads to more precise bounding box estimations. However, the performance improvement saturates as the feature map resolutions increase.

Finally, we prove that omitting the Transformer’s encoder leads to more precise detections when the amount of available training data is strictly limited. To this end, we experiment with DETR and Deformable DETR configurations omitting the encoder and report the results in Table 6.

Evaluation of $\textrm{mAP}_{\textrm{coco}}$ values presented in Table 6 leads to the conclusion that the Transformer’s encoder is disruptive to detection performance. This supports the hypothesis that the encoder’s intricate relation modeling task would require an extreme amount of training epochs and hence a large amount of annotated data to capture meaningful dependencies among the input sequence elements.