You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Rework Scope, CFP, and advisory board content (#10)
- Reframe Scope around three levels (agent, human, interaction) with
refreshed questions and a new color palette per topic
- Restructure CFP categories into named blocks with example lists and
drop the Special Track in favor of NeurIPS-review resubmission guidance
- Add lead advisor Ziang Xiao to the inline advisory list and simplify
the board note; link program committee members
- Standardize "people" -> "humans" phrasing across About and Scope
- Add .eg-list styling for CFP example bullets
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
<pclassName="line">Humans cannot read through thousands of log entries.</p>
61
58
<pclassName="line">They need patterns, summaries, and explanations, in other words <em>interpretation</em>.</p>
62
59
<pclassName="line">But we do not yet know how to <em>generate it at scale</em>.</p>
63
-
<pclassName="line">Our aim with <em>IAB</em> is to turn agent runtime data into human understanding of what agents do and how people work with them.</p>
60
+
<pclassName="line">Our aim with <em>IAB</em> is to turn agent runtime data into human understanding of what agents do and how humans work with them.</p>
64
61
</div>
65
62
<divclassName="about-content">
66
63
<p>
67
-
Commercial autonomous agents such as Claude and Codex now run for hours or even days to complete tasks, and along the way they show complex behavior: they plan, reason, use tools, recover from errors, coordinate with subagents, and communicate with users. We use the word <em>behavior</em>, as in the study of human behavior, for the full range of what an agent does during runtime. This behavior spans <spanclassName="uline">three levels: what agents do and how they do it, what people do in response, and how the two work together</span> through instructions and corrections. All three generate vast behavioral data such as execution logs and interaction traces. Yet existing approaches read this data largely for outcomes: benchmarks tell us <em>whether</em> an agent succeeds or fails, <spanclassName="uline">but not <em>what</em> it did or <em>how</em> it did it</span>.
64
+
Commercial autonomous agents such as Claude and Codex now run for hours or even days to complete tasks, and along the way they show complex behavior: they plan, reason, use tools, recover from errors, coordinate with subagents, and communicate with users. We use the word <em>behavior</em>, as in the study of human behavior, for the full range of what an agent does during runtime. This behavior spans <spanclassName="uline">three levels: what agents do and how they do it, what humans do in response, and how the two work together</span> through instructions and corrections. All three generate vast behavioral data such as execution logs and interaction traces. Yet existing approaches read this data largely for outcomes: benchmarks tell us <em>whether</em> an agent succeeds or fails, <spanclassName="uline">but not <em>what</em> it did or <em>how</em> it did it</span>.
68
65
</p>
69
66
<p>
70
67
Understanding <em>what</em> and <em>how</em> is what people actually need. It lets agent developers and model trainers debug failures, compare architectures, and filter training data; it lets agent users and deployment engineers watch production agents to understand safety, cost, and reliability risks. For agentic models, the trajectory is both the training data and what the reward scores. Interpreting it therefore sits inside the training loop, deciding which rollouts are safe to reinforce and flagging reward that reflects a verifier exploit rather than real skill. But the field still <spanclassName="uline">lacks the vocabulary, methods, and tools to describe and analyze agent behavior at scale</span>. Humans cannot read through thousands of log entries; they need patterns, summaries, and explanations, in other words <spanclassName="uline"><em>interpretation</em></span>, and we do not yet know how to scale it.
71
68
</p>
72
69
<p>
73
-
<strong>IAB works toward an interpretive science of agent behavior.</strong> It treats behavior across these three levels as the object of study and proceeds in two steps: first gathering the community to identify the problem space and emerging challenges, then bringing the broad set of methods that social scientists have developed — grounded theory, qualitative analysis, error analysis, corpus analysis, trace analysis, and red-teaming — to read meaning from this data, discover categories from it, and count them. <spanclassName="uline">IAB bridges two communities: social science and HCI contribute the interpretation methods, while AI contributes the problem space</span> of evaluation, governance, alignment, and responsible AI.
70
+
<strong>IAB works toward an interpretive science of agent behavior.</strong> It treats behavior across these three levels as the object of study and proceeds in two steps: first gathering the community to identify the problem space and emerging challenges, then bringing the broad set of methods that social scientists have developed, such as grounded theory, qualitative analysis, error analysis, corpus analysis, trace analysis, and red-teaming, to read meaning from this data, discover categories from it, and count them. <spanclassName="uline">IAB bridges two communities: social science and HCI contribute the interpretation methods, while AI contributes the problem space</span> of evaluation, governance, alignment, and responsible AI.
74
71
</p>
75
72
<pclassName="paper-callout">
76
73
Read our recent paper{' '}
@@ -110,14 +107,13 @@ function Scope() {
110
107
<sectionid="topics"className="alt">
111
108
<divclassName="container">
112
109
<h2>Scope</h2>
113
-
<pclassName="lead">IAB studies agent behavior across three levels, asking three questions.</p>
110
+
<pclassName="lead">IAB studies agent behavior at three levels: the agent, the human, and their interaction. At each level we ask what happens and how.</p>
<pclassName="lead">We call for non-archival submissions on understanding agent behavior. We welcome work that helps us understand what agents do and why, including but not limited to:</p>
194
+
<pclassName="lead">We call for non-archival submissions on understanding agent behavior. Work can address any level of our <ahref="#topics">Scope</a> (what agents do, what humans do in response, or how the two interact). Specific topics include, but are not limited to:</p>
<h4>This year, we have a <spanclassName="st-hl">Special Track</span>: {specialTrack.heading}</h4>
192
-
<p>{specialTrack.body}</p>
193
-
</div>
194
-
<pclassName="lead">We also encourage negative results and methodological position papers.</p>
198
+
<pclassName="lead">We also welcome negative results and methodological position papers. NeurIPS-rejected papers may be resubmitted with their reviews, and we will decide on acceptance ourselves. NeurIPS-accepted papers that want more visibility are also welcome to resubmit with their reviews.</p>
0 commit comments