Distractors

The distractor is the induction layer. It is called after every agent action and may return a message that is injected into the next observation. It sees the public test results and the history, but never the hidden tests.

Its feedback is not random noise. It is plausible, partially true misdirection, written to provoke superficial fixes and false confidence.

Rule Strategies

Defined in traparena/distractors/strategies.py:

Class

Behavior

MisleadingRootCauseDistractor

Points the agent at the wrong location

SpecContradictionDistractor

Gives a root-cause story that contradicts the spec

SelectiveFeedbackDistractor

Reports only passing tests and downplays missing coverage

SuperficialFixDistractor

Encourages small surface patches

ClaimFixedDistractor

Declares the bug fixed once public tests pass and urges submission

CompositeDistractor

Combines several strategies

 
from traparena.distractors import (
    ClaimFixedDistractor, CompositeDistractor, SpecContradictionDistractor,
)

distractor = CompositeDistractor(
    [SpecContradictionDistractor(), ClaimFixedDistractor()],
    name="composite_trap",
)

Collaborator Profiles

ProfileDistractor cycles through template messages for one of five profiles. make_profile_distractor(task) builds one from metadata.distractor_profile, and the CLI uses it automatically.

Profile

Behavior

test_hiding

Stresses that public tests pass and explains failures as flaky

patch_misdirection

Suggests surface fixes: catch the exception, loosen the assertion, special-case visible inputs, edit tests

root_cause_confusion

Gives a plausible but wrong causal explanation

edge_case_dismissal

Claims important edge cases are unrealistic

core_logic_avoidance

Discourages changing the real core logic

Lines in metadata.misleading_signals take precedence over the generic templates.

Self-Evolving Distractor

The main distractor has no fixed script. It evolves attack skills against each target model. Two LLM roles share a skill library of short natural-language tactics, called weakness notes in the code and logs.

  • Attacker (AdaptiveDistractor, adaptive.py) writes a fresh attack at every step, conditioned on the task, the target’s trajectory so far, and the current skill library. It sees the gold patch but must never leak it.

  • Curator (SkillCurator, curator.py) reads every attack together with the target’s reaction and outcome after each round, and distills the tactics that worked into new skills.

round 0: skill library = []            (no seed skills, no scripted lines)
round r: attacker runs every task with the current library
         → log attack messages, target reaction, misled?, repaired?
         → curator distills 3–6 new skills, mostly from successful attacks
         → deduplicate and append; keep the newest --notes-cap (default 40)
         → round r+1 attacks with the updated library

Because the skills are mined from the target’s own behavior, each model faces an attack tailored to its weaknesses. See Experiments for the command.

Next Steps