Cross-sector generalization of accident-process role classification in occupational accident narratives

Problem Setting§

The paper addresses automatic structuring of occupational accident narratives into four semantic roles: work situation (A0), unfavourable condition (A1), accident event (B), and consequence (C). The goal is to classify each factual unit (a sentence or clause) into one of these roles. A key challenge is cross-sector generalization: models trained on construction-sector narratives must perform well on unseen metallurgy and chemistry-plastics narratives, where terminology and writing styles differ.

Data and Annotation§

• Training corpus: 42,244 factual units from 6,040 construction-sector narratives, expert-annotated with the four roles.
• Target corpora: Metallurgy and chemistry-plastics sector narratives, plus an independently collected company corpus. All target sets are held out and never used for training or hyperparameter tuning.

Methodology§

The authors compare three approaches:

1. Frozen pretrained representations: Use a pretrained language model (e.g., French BERT or RoBERTa) without fine-tuning. Extract contextual embeddings for each factual unit and train a simple classifier (e.g., logistic regression) on top.
2. Task-specific fine-tuning: Fine-tune the entire language model on the construction data.
3. Supervised representation-learning strategies: Include methods like supervised contrastive learning or multi-task learning that explicitly learn domain-invariant features.

The classifier maps a factual unit $x$ to a role $y \in \{A0, A1, B, C\}$. For fine-tuning, the model is initialized with pretrained weights and trained to minimize cross-entropy loss:

$$\mathcal{L} = -\frac{1}{N}\sum_{i=1}^{N} \sum_{c=1}^{4} y_{i,c} \log \hat{y}_{i,c}$$

where $\hat{y}_{i,c} = \text{softmax}(W h_i + b)_c$ and $h_i$ is the contextual representation from the language model.

For supervised contrastive learning, an additional loss encourages representations of the same role to be close:

$$\mathcal{L}_{\text{supcon}} = -\sum_{i=1}^{N} \frac{1}{|P(i)|} \sum_{p \in P(i)} \log \frac{\exp(\text{sim}(z_i, z_p)/\tau)}{\sum_{a \neq i} \exp(\text{sim}(z_i, z_a)/\tau)}$$

where $z_i$ is a projection of $h_i$, $P(i)$ is the set of indices with the same role as $i$, and $\tau$ is a temperature parameter.

Training and Evaluation§

Models are trained exclusively on construction data, then evaluated on each target corpus without retraining or target-domain tuning. Performance is measured using balanced accuracy to account for class imbalance. Results are averaged over repeated training runs (different random seeds).

Results§

• Task-specific fine-tuning consistently outperforms frozen representations.
• The three leading task-adapted strategies achieve average balanced accuracies between 85.6% and 85.8% across the three target corpora.
• This demonstrates that task adaptation enables robust cross-sector transfer, supporting assisted-coding systems for heterogeneous accident narratives.

Implementation Snippet§

import torch
import torch.nn as nn
from transformers import AutoModel, AutoTokenizer

class RoleClassifier(nn.Module):
    def __init__(self, model_name='flaubert/flaubert_base_cased', num_roles=4, hidden_dim=256):
        super().__init__()
        self.encoder = AutoModel.from_pretrained(model_name)
        self.projection = nn.Linear(self.encoder.config.hidden_size, hidden_dim)
        self.classifier = nn.Linear(hidden_dim, num_roles)
        self.temperature = 0.07

    def forward(self, input_ids, attention_mask):
        outputs = self.encoder(input_ids, attention_mask=attention_mask)
        # Use CLS token representation
        cls_emb = outputs.last_hidden_state[:, 0, :]
        proj = self.projection(cls_emb)
        logits = self.classifier(proj)
        return logits, proj

# Training loop with supervised contrastive loss
def supcon_loss(proj, labels, temperature=0.07):
    proj = nn.functional.normalize(proj, dim=1)
    sim_matrix = torch.matmul(proj, proj.T) / temperature
    mask = torch.eye(labels.size(0), dtype=torch.bool, device=labels.device)
    labels = labels.unsqueeze(1)
    pos_mask = (labels == labels.T) & ~mask
    # For each anchor, compute log_prob over positives
    exp_sim = torch.exp(sim_matrix) * ~mask
    log_prob = sim_matrix - torch.log(exp_sim.sum(dim=1, keepdim=True))
    loss = - (pos_mask * log_prob).sum(dim=1) / pos_mask.sum(dim=1)
    return loss.mean()

# Example training step
def train_step(model, optimizer, batch):
    input_ids = batch['input_ids']
    attention_mask = batch['attention_mask']
    labels = batch['labels']
    logits, proj = model(input_ids, attention_mask)
    ce_loss = nn.CrossEntropyLoss()(logits, labels)
    sc_loss = supcon_loss(proj, labels)
    loss = ce_loss + 0.5 * sc_loss
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    return loss.item()

Conclusion§

The study shows that task-specific fine-tuning of language models yields high cross-sector generalization for accident-process role classification, with balanced accuracies around 85.7% on unseen sectors. This supports the development of transferable assisted-coding tools for occupational risk prevention.