Luggage re-identification: A study of strategies towards a real-world application

Luggage Re-identification: A Study of Strategies Towards a Real-World Application§

This paper addresses the problem of luggage re-identification (ReID) in real-world video surveillance scenarios, where occlusions, clutter, and crowded scenes degrade performance. The authors propose an end-to-end pipeline that processes multiple video streams to detect and re-identify luggage across disjoint camera views. They systematically evaluate strategies for selecting and merging multiple shots/views of the same luggage and for training on diverse environment datasets to improve generalization.

Core Methodology§

The pipeline consists of three main stages:

1. Luggage Detection: A pre-trained object detector (e.g., YOLO or Faster R-CNN) is used to detect luggage instances in each frame. Detections are tracked across frames to form tracklets (shots) of the same luggage within a single camera view.

1. Feature Extraction and Re-identification: For each detected luggage crop, a CNN backbone (e.g., ResNet-50) extracts a feature vector. The model is trained with a combination of classification loss (cross-entropy with label smoothing) and triplet loss to learn discriminative embeddings:

$$\mathcal{L} = \mathcal{L}_{CE} + \lambda \mathcal{L}_{tri}$$

where $\mathcal{L}_{tri} = \max(0, \|f(a)-f(p)\|^2 - \|f(a)-f(n)\|^2 + m)$.

1. Multi-shot and Multi-view Fusion: Instead of relying on a single image, the system aggregates features from multiple shots (tracklets) and views (different cameras) to improve robustness. The authors compare strategies:

• Single-shot: Use one crop per identity (e.g., highest confidence).
• Multi-shot averaging: Average features from all crops in a tracklet.
• Multi-view merging: Combine features from different camera views using pooling (max, mean, or attention-based).
• Weighted fusion: Weight features by detection confidence or quality.

The final distance between two luggage identities is computed as the distance between their fused representations:

$$d(i,j) = \| \mathbf{v}_i - \mathbf{v}_j \|_2$$

where $\mathbf{v}_i$ is the fused feature for identity $i$.

Training Strategy§

To improve generalization, the authors train on a combination of datasets from different environments (e.g., airports, train stations). They investigate:

• Single-dataset training: Train on one environment and test on another (cross-domain).
• Multi-dataset training: Combine multiple datasets and train jointly.
• Fine-tuning: Pre-train on a large dataset then fine-tune on target environment.

They use standard ReID evaluation metrics: Cumulative Matching Characteristics (CMC) and mean Average Precision (mAP).

Implementation Details§

• Backbone: ResNet-50 pre-trained on ImageNet, with the last stride set to 1 to increase feature resolution.
• Input: 256x128 RGB images, normalized with ImageNet statistics.
• Data Augmentation: Random cropping, horizontal flipping, and random erasing.
• Optimizer: Adam with learning rate 3.5e-4, weight decay 5e-4.
• Training: 60 epochs with a warm-up of 10 epochs, batch size 64 (4 instances per ID).
• Triplet Loss: Margin $m=0.3$, with hard mining.
• Feature Fusion: For multi-shot, average features from all tracklets; for multi-view, concatenate and apply a linear layer or use attention.

Code Snippet: Multi-Shot Feature Fusion§

import torch
import torch.nn as nn
import torch.nn.functional as F

class MultiShotFusion(nn.Module):
    def __init__(self, feat_dim=2048, fusion='mean'):
        super().__init__()
        self.fusion = fusion
        if fusion == 'attention':
            self.attn = nn.Linear(feat_dim, 1)

    def forward(self, features, masks=None):
        # features: (batch, num_shots, feat_dim)
        if self.fusion == 'mean':
            if masks is not None:
                features = features * masks.unsqueeze(-1)
                return features.sum(dim=1) / masks.sum(dim=1, keepdim=True).clamp(min=1)
            return features.mean(dim=1)
        elif self.fusion == 'max':
            return features.max(dim=1)[0]
        elif self.fusion == 'attention':
            attn_weights = F.softmax(self.attn(features), dim=1)  # (batch, num_shots, 1)
            return (features * attn_weights).sum(dim=1)

# Example usage: fuse features from multiple shots of the same luggage
feat_dim = 2048
num_shots = 5
batch_size = 8
features = torch.randn(batch_size, num_shots, feat_dim)
fusion = MultiShotFusion(feat_dim, fusion='attention')
fused_feat = fusion(features)  # (batch_size, feat_dim)

Experiments and Results§

The authors evaluate on a new benchmark dataset captured in real-world environments (airports and train stations) with multiple cameras. They show that:

• Multi-shot fusion significantly improves ReID accuracy over single-shot, especially under occlusions.
• Multi-view fusion further boosts performance by leveraging complementary information.
• Training on a combination of datasets from different environments improves cross-domain generalization.
• Attention-based fusion outperforms simple averaging.

The paper concludes that a multi-shot, multi-view approach with diverse training data is essential for practical luggage ReID deployment.