Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection

Methodology§

The paper introduces a comprehensive, mechanism-oriented taxonomy of Indirect Linguistic Expressions (ILE) for detecting coded language in social media posts. The taxonomy categorizes underlying operations (e.g., euphemism, metaphor, abbreviation) rather than surface forms. It is integrated into LLM prompts to guide the model's reasoning about encoding and decoding of sensitive meanings.

Core Innovation§

Instead of using a fixed list of coded terms, the taxonomy provides a structured set of mechanisms:

$$\text{ILE Mechanisms} = \{M_1, M_2, \dots, M_k\}$$

Each mechanism $M_i$ is a transformation (e.g., substitution, deletion) applied to a target meaning. The LLM is prompted to classify posts and identify spans using these mechanisms as reasoning anchors.

Evaluation§

Tested on 2,000 manually annotated posts from TikTok and Bluesky. Three LLMs (e.g., GPT-4, Llama-2, Mistral) were evaluated with five conditions: no taxonomy, four existing taxonomies (binary, topical, pragmatic, mixed), and the proposed taxonomy. Metrics:

• Document-level accuracy: $$\text{Acc} = \frac{\# \text{correct detections}}{\# \text{total posts}}$$
• Span-level F1: $$F_1 = 2 \cdot \frac{P \cdot R}{P + R}$$

The proposed taxonomy achieved 4.7% higher accuracy and 5.4% higher F1 than the best baseline.

Practical Implementation§

A prompt template includes the taxonomy definition and instructions:

# Example prompt construction for ILE detection
prompt = f"""You are a coded language detection system. Use the following taxonomy of encoding mechanisms:
{', '.join(taxonomy_mechanisms)}

Classify the post below. If it contains ILE, list the spans and their mechanisms.

Post: {post_text}
"""
response = llm.generate(prompt)

For span-level prediction, a simple PyTorch evaluation loop:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("model-name")
model = AutoModelForSequenceClassification.from_pretrained("model-name")

def evaluate_span(posts, spans_true, taxonomy_prompt):
    preds = []
    for post in posts:
        prompt = f"{taxonomy_prompt}\nPost: {post}"
        inputs = tokenizer(prompt, return_tensors="pt")
        outputs = model(**inputs)
        # ... parse logits to detect spans ...
        preds.append(parsed_spans)
    return classification_report(spans_true, preds)

Implications§

The taxonomy acts as a stable scaffold, enabling LLMs to generalize to emerging coded language without retraining. It highlights the importance of explicit encoding mechanisms over surface-form lists for content moderation.