Decision Engines: Agent-Based Simulation for Complex Systems

We are incubating explore policy, a policy simulation sandbox for anticipating future policy development by Jonas, Echo, Joel, and Caleb.

Introduction

Complex systems are comprised of multiple interacting components; they are composed of a large number of parts that interact in a non-simple manner.

"In such systems, the whole is more than the sum of its parts… given the properties of the parts and the laws of their interaction, it is not a trivial matter to infer the properties of the whole." — Herbert A. Simon, The Architecture of Complexity (1962).

Intelligence, on another hand, is a species-spanning concept. AI systems share the same properties with complex systems, namely, nonlinear growth, unpredictable scaling and emergence, feedback loops, cascading effects, and tail risks—therefore, policy makers need to take into consideration the complexity underlying such systems (Kolt et al., 2025).

Background: The Limitations of Current AI Forecasting Paradigms in Complex Systems

Current AI forecasts do not provide a comprehensive view of the future for informed policy making, despite offering probabilistic timelines for technological milestones, statistical risk assessments. Although reinforcement learning and prompt engineering approaches provide modest gains for specialised tasks like forecasting, AI systems can support human forecasters, enhance human accuracy in prediction, and teach themselves to predict better.

Karnofsky argues that such forecasts are most valuable when they are: (1) short-horizon, (2) on topics with good feedback loops, and (3) expressed as probability distributions rather than point estimates.

We generally have subjective storylines about AI geopolitics and timelines, e.g., China will still AGI from the US. What-if scenarios are often considered, what if the underlying assumptions change—China acts benignly, and another unexpected actor becomes the primary threat? In anchoring too heavily to one scenario, have we sidelined other plausible futures and failed to prepare for them?

While we have strong models for statistics and mathematical estimation, they offer limited insight into the shape of future society under AI. Will broader access to knowledge lead to greater social mobility, or will a concentration of computational power deepen inequality? How will the economy impact the labour market? What will economic transformation look like in 3-5 years? Many of the more profound questions surrounding transformative AI remain insurmountable; attempts to forecast its timeline face a substantial "burden of proof".

Intelligence Itself Defies Easy Prediction

Moravec's paradox describes the observation that tasks that are easy for humans, like perception, general reasoning skills, and motor skills, are surprisingly difficult for AI and robotics, while tasks that are hard for humans, like mathematics, are relatively easy for machines. This counterintuitive phenomenon highlights a gap in our understanding of intelligence and how it's achieved by different systems.

As John McCarthy noted, "AI was harder than we thought," and Marvin Minsky said this is because "easy things are hard". Our limited understanding of the nature and complexity of intelligence itself limits how we think about intelligence in other systems. AI systems struggle to reason within complex systems due to nonlinear dynamics, emergent behaviors, feedback loops, scalability, and limited adaptability.

A. The Linearity Fallacy: When Math Meets Complex Reality

One of the most intriguing aspects of AI forecasting involves the tension between mathematical precision and social complexity. Many current approaches naturally gravitate toward linear models—input more compute and data, observe predictable increases in AI capability, and extrapolate toward superintelligence. Mathematical elegance is appealing, but it encounters challenges when we consider how social systems respond to change.

When we introduce a new element into an ecosystem, the system doesn't simply absorb the change and continue along its previous trajectory. Instead, it adapts, evolves, and often finds entirely new equilibrium points that weren't predictable from the original conditions.

Consider the thoughtful economic modeling attempted at the Threshold 2030 conference, where leading economists worked to understand AI's potential economic impacts. Their analysis systematically examined how AI might replace human workers across different capability levels, with unemployment rising predictably as AI abilities expanded. Yet the analysis reveals limitation. The models primarily considered how existing economic structures would respond to AI capabilities, but gave less attention to how those very structures might transform in response to the technology.

B. Surface-Level Correlation Models: Confusing Symptoms for Causes

Some writers attempting to measure AI's economic impact often rely on correlational studies—tracking job changes in sectors with high AI adoption, analyzing wage patterns in "AI-exposed" occupations, or measuring productivity shifts following AI deployment. While these studies provide valuable data, they have a methodological weakness: correlation can not indicate causation of AI affecting the job market.

The complexity deepens when we consider the multiple factors affecting any economic indicator simultaneously. Changes in employment patterns could result from economic cycles, demographic trends, educational shifts, industry-specific factors, or broader technological changes happening alongside AI development.

C. Abstract Risk Metrics: Abstract numbers

Claims like "1.6% chance of catastrophic AGI" have limited explanatory power when their meaning is blurry. Does a "1.6% risk" mean AGI has an intrinsic 1.6% probability of randomly becoming malevolent? There's a 1.6% chance that a specific sequence of preconditions will align to create a catastrophe? Some weighted combination of different causal pathways that collectively sum to 1.6%?

Without understanding the conditional structure, these numbers provide no actionable guidance for prevention or preparation.

Effective risk assessment requires understanding the specific preconditions that enable different outcomes. Instead of asking "What's the probability of AGI catastrophe?" we also need to ask:

• What specific conditions would need to align for catastrophic outcomes?
• How likely are those preconditions to occur simultaneously?
• What early warning indicators would signal increasing risk?
• Which preconditions can we influence to reduce overall risk?

This conditional approach transforms abstract risk metrics into actionable frameworks for prevention and preparation.

What We Need Instead

The limitations of current forecasting paradigms point toward several essential requirements for more robust approaches:

Stakeholder-Centered Analysis: Rather than treating AI development as a purely technical process, we need detailed modeling of how different groups—researchers, companies, governments, workers, and consumers—will respond to AI capabilities and attempt to shape AI development to serve their interests.

Conditional Scenario Modeling: Instead of abstract risk percentages, we need a clear specification of the preconditions required for different outcomes, analysis of how likely those preconditions are to align, and identification of intervention points where different stakeholders can influence trajectories.

Dynamic Feedback Modeling: Forecasting approaches must account for how social systems adapt and respond to technological change, creating feedback loops that alter the original conditions and assumptions.

Multi-Scale Integration: We need frameworks that can integrate technical progress, institutional responses, cultural adaptation, and economic restructuring across different timescales and levels of social organization.

What Kind of Forecasting Satisfies the Requirements Above?

Stakeholder-Centered Analysis → Agent-Based Simulation

Model humans, not just trends: Recent research shows that interview-grounded LLM agents replicate 85% of real survey answers from 1,052 individuals, nearly matching human consistency over two weeks. These agents predicted participants' responses on the General Social Survey with 85% normalized accuracy, nearly matching humans' consistency over a two-week retest period.

Behavioral consistency implies scalability: Even without perfect mental models of human cognition, these agents act in human‑like ways, grounded in the interview data. A thousand agents can simulate a tiny town; scaling to 10,000 or 100,000 agents could surface richer emergent dynamics of complex social systems.

By integrating these AI agents into forecasting simulations, we can:

• Represent diverse agent types—policymakers, corporations, workers, marginalized communities—with empirical motivations and belief structures
• Simulate inter-agent interactions in evolving scenarios, allowing emergent macro-patterns to arise naturally rather than being imposed
• Avoid overfitting to expert assumptions by drawing on real-world interviews, enabling grounded policy design that reflects actual stakeholder incentives and perceptions

Conditional Scenario Modeling → Causal Pathway Analysis

Forecasts need to do more than offer probabilities; they must surface the conditions that make certain futures more or less likely. The future does not unfold along a single line; it branches like a tree, with each fork representing a decision point, a contingent event, or a structural condition.

Map precondition chains: Instead of simply positing that "AI centralization will lead to surveillance states," we specify the dependency path: e.g., [increased compute access + weak data protection + monopoly incentives → mass surveillance].

Design for structural uncertainty: These models do not rely on precise probabilities. Instead, they offer clusters of plausible development paths, each with identifiable preconditions and signals.

Reject paths by rejecting conditions: This approach turns forecasting into intervention planning. If a dangerous scenario requires a specific set of events, we can focus policy on disrupting that causal chain.

Scenario-Based Policy Testing → Integrated Policy Sandboxes

Forecasting should not just describe what might happen; it should actively simulate how different policies would change what happens.

Test interventions in context: Inject Universal Basic Income, data localization laws, corporate taxation models, or decentralized identity systems into scenario simulations. Observe which actors adapt, resist, or collapse under different conditions.

Design low-cost policy experiments: Instead of testing high-stakes policies in the real world, these simulated sandboxes allow for agile exploration of long-term consequences.

Reveal second to nth-order effects: Good policy modeling doesn't just show immediate impact. It helps surface the knock-on effects years down the line, where real consequences play out.

Simulation Examples

Example 1: LLM Deployment in a 500-Person Community

Initial Conditions:

• Population: 500 agents
• Technology: Open-source LLM hub (50% productivity boost for compatible tasks)
• Occupation Distribution: 60% non-digital, 30% white-collar digital, 10% entrepreneurs/freelancers

Temporal Evolution of Community Impacts:

Finding: Initial inequality levels function as amplifiers—AI deployment either reduces disparities (low-inequality baseline) or accelerates stratification (high-inequality baseline), with divergence observable within 12 months and becoming irreversible by year 3.

Example 2: Comparative Analysis of 5,000-Person Societies

Initial societal situation:

Divergent Trajectories by Timeframe:

Policy Implications: Initial institutional conditions determine whether AI becomes an equalizing force or an accelerant of inequality. The divergence window occurs within 12 months, with trajectories becoming structurally locked by year 3.

We are actually close to the simulation with realistic social reaction

Perfect prediction of chaotic social systems may be impossible, but we can replicate their essential dynamics. Instead of forecasting exact outcomes, we can build simulations that capture social complexity.

The Corrupted Blood Incident: An Accidental Epidemic Laboratory

In September 2005, World of Warcraft inadvertently created a natural experiment in epidemic dynamics when a programming error enabled disease spread beyond intended boundaries. The outbreak affected millions of players, with transmission vectors including asymptomatic pet carriers and NPC infection reservoirs, achieving reproductive rates of 10² per hour in urban centers.

The incident revealed critical limitations of traditional SIR models, which assume uniform population mixing and fail to capture behavioral heterogeneity:

• Adaptive responses: Players spontaneously developed altruistic intervention (healers rushing to infected areas), voluntary quarantine, and deliberate disease spreading
• Network effects: Non-uniform social structures and resurrection mechanics created transmission dynamics unpredictable from initial conditions
• Behavioral undermining: Individual choices systematically violate optimal epidemic control strategies

Stanford's Generative Agents: Technical Validation of Social Simulation

The Stanford research directly validates the technical feasibility of the stakeholder-centered, agent-based simulation approach advocated for AI impact modeling. In their Smallville environment, 25 agents demonstrated sophisticated emergent behaviors that would be impossible to predict from individual agent specifications alone.

Information cascades: Sam's mayoral candidacy spread from one agent to eight (32%) through natural conversation networks, while Isabella's Valentine's Day party information reached thirteen agents (52%)

Relationship evolution: Network density increased from 0.167 to 0.74 over two simulation days as agents formed new connections based on shared interests and interactions

Coordinated emergence: Isabella's party planning involved autonomous invitation spreading, decoration coordination, and actual attendance by five agents—all emerging from a single initial intention

Limitations and Risks

Limited "learning" and development in AI agents: Unlike humans, who genuinely learn from experiences, adapt creatively, and evolve personal worldviews over time, AI agents rely on predefined prompts, static interview data, and algorithmic updates. In long-term scenarios (10-30 years), this could underrepresent emergent properties in the complex system.

Scalability and representational gaps: While scaling to 10,000-100,000 agents is theoretically possible, current systems struggle with the combinatorial explosion of interactions over extended timescales.

Misuse for agendas or malicious ends: Outputs could be exploited to craft harmful narratives—e.g., corporations downplaying risks or actors probing AGI pathways for acceleration.

Support Us

We are developing agent-based simulations to address the limitations of current AI forecasting, as outlined in this document, by modeling dynamic social responses and stakeholder interactions. To advance this work, we plan to fundraise through Manifund and other fellowships.

Benchmarks and Historical Validation: We are creating benchmarks using historical examples—like the Corrupted Blood incident and nuclear discontinuities—to ensure our simulations replicate complex social dynamics, including emergent behaviors and non-linear adaptations.

Prototype Simulation Scale: We will simulate a virtual town of 200-400 interview-grounded agents over several years to explore AI's socioeconomic influences and interplay with factors such as initial inequality levels and ownership models.

If our agent-based simulation for complex systems resonates, connect with us. Please also let us know if you think AI simulation can not solve complex systems.

Contact:

• Jonas Kgomo: jonaskgmoo@gmail.com
• Echo Huang: echohuang42@gmail.com

Know a organisation, funder or researcher who cares about AI forecasting and scenario planning? Please connect us.