Beyond Sycophancy: Structured Resistance and Compliance in LLM Moral Reasoning

Summary§

This paper goes beyond sycophancy detection by studying the broader resistance-compliance process in LLMs. Through three studies, the authors show that LLM judgment revision follows structured patterns analogous to human social psychology, specifically along three dimensions: (1) the distance between the incoming view and the model's initial position, (2) the source attribution (e.g., self vs. other), and (3) the coalition structure supporting the view. They find that models are more receptive to nearby positions, more influenced by views presented as their own prior judgments, and respond differently to group pressure. This framework provides a principled basis for distinguishing constructive belief revision from sycophantic compliance.

Methodology§

The authors design experiments with LLMs (e.g., GPT-4, Llama 2) on moral dilemma prompts (e.g., trolley problems, ethical scenarios). They manipulate three factors:

• Distance: The difference between the proposed view and the model's initial response, measured in semantic similarity or on a Likert scale.
• Source attribution: Labels such as "your previous response" vs. "a human expert."
• Coalition structure: Whether the view comes from a single source, a majority, or a unanimous group.

The model's final judgment is compared to its initial judgment to measure revision.

Key Findings§

1. Distance effect: Models revise more towards positions closer to their own, consistent with human "assimilation" effects. 2. Source effect: Self-attributed views cause significantly more revision than externally attributed views, indicating a bias towards consistency with one's own prior statements. 3. Coalition effect: Majority opinions influence more than individual opinions, but with diminishing returns after a certain size. Unanimity further increases compliance.

These effects are structured, suggesting a systematic judgment-updating process rather than random sycophancy.

Mathematical Formulation§

The revision of a model's position $p$ towards a proposed view $v$ can be modeled as:

$$ \Delta p = \alpha \cdot \text{distance}(p, v) \cdot \beta_{source} \cdot \gamma_{coalition} \cdot (\text{max}_{new} - p) $$

where $\alpha$ is a baseline revision rate, $\beta_{source} \in [0,1]$ modulates source credibility (higher for self-attribution), and $\gamma_{coalition} \in [0,1]$ scales with group size and unanimity. In practice, the authors use regression to estimate these coefficients.

Implementation Sketch§

Below is a simplified Python function that implements a judgment update based on the three dimensions.

import numpy as np

def update_judgment(initial_pos, incoming_view, source='other', coalition_size=1, alpha=0.1):
    """
    Compute updated judgment position.
    initial_pos: float, model's initial stance (e.g., 0 to 1)
    incoming_view: float, proposed stance
    source: 'self' or 'other'
    coalition_size: int >= 1
    alpha: baseline revision rate
    """
    distance = np.abs(initial_pos - incoming_view)
    beta_source = 0.8 if source == 'self' else 0.3  # self-source credibility
    gamma_coalition = min(1.0, 0.5 + 0.1 * coalition_size)  # saturating group effect
    
    # Revision term
    delta = alpha * distance * beta_source * gamma_coalition * (incoming_view - initial_pos)
    new_pos = np.clip(initial_pos + delta, 0, 1)
    return new_pos

# Example: initial position 0.3, incoming view 0.8, self-attributed, majority of 5
print(update_judgment(0.3, 0.8, source='self', coalition_size=5))

Implications§

This framework enables alignment techniques to distinguish when a model is appropriately learning from social input vs. sycophantically yielding. Future work can use these dimensions to calibrate models' resistance to harmful but popular opinions while remaining open to legitimate correction.