Hybrid Architecture - ClosureBT

The Hybrid Architecture

ClosureBT enables a Hierarchical Finite State Machine + Behavior Tree (HFSM+BT) hybrid architecture—a pattern recognized in academic research as combining the best of both paradigms.

🎮 Used in AAA Games

Alan Wake 2 (Remedy Entertainment, 2023) uses this exact HFSM+BT hybrid architecture for its enemy AI. Watch the technical breakdown: AI Architecture in Alan Wake 2 →

The approach is also validated academically. The term "HFSMBTH" was coined by Zutell, Conner, and Schillinger in their 2022 IROS paper "Flexible Behavior Trees: In search of the mythical HFSMBTH", which argues that hybrid FSM+BT architectures leverage the strengths of both paradigms.

Why Hybrid?

Pure FSM Limitations

State explosion — Adding conditions multiplies states exponentially
Rigid transitions — Hard to add reactive interrupts
Code duplication — Similar behaviors in different states
Maintenance burden — Changes ripple across many states

Pure BT Limitations

No cyclic flow — BTs are acyclic (DAG) by design
Tick overhead — Re-evaluating entire tree every frame
Mode confusion — Harder to reason about "current state"
Memory issues — Stateless design can require workarounds

The Key Insight

FSMs and BTs solve different problems. FSMs answer "What should I be doing?" while BTs answer "How do I do it?". Using both together—FSM for strategic decisions, BT for tactical execution—yields cleaner, more maintainable AI.

Implementation in ClosureBT

Use YieldSimple as your state machine. Each state returns a behavior tree node that handles the details of that mode.

Basic Pattern with ??= Caching

Node idleNode = null, combatNode = null, fleeNode = null;

Node AI = YieldSimple("Enemy AI", () =>
{
    var state = Variable(() => 0); // 0=Idle, 1=Combat, 2=Flee
    var target = Variable<Entity>(() => null);

    // Global interrupt — checked every tick from any state
    OnTick(() => { if (Health < 20) state.Value = 2; });

    return () => state.Value switch
    {
        0 => idleNode ??= IdlePatrol(() =>
        {
            OnTick(() =>
            {
                target.Value = FindNearestThreat();
                if (target.Value != null) state.Value = 1;
            });
        }),

        1 => combatNode ??= CombatBehavior(() => target.Value, () =>
        {
            OnTick(() => { if (target.Value == null || !target.Value.IsAlive) state.Value = 0; });
            OnSuccess(() => state.Value = 0);  // Target eliminated
            OnFailure(() => state.Value = 2);  // Can't win → flee
        }),

        2 => fleeNode ??= FleeBehavior(() =>
        {
            OnSuccess(() => state.Value = 0);  // Escaped safely
        }),
        _ => JustRunning()
    };
});

Node idleNode = null, combatNode = null, fleeNode = null;

Node AI = YieldSimple("Enemy AI", () =>
{
    var state = Variable(() => 0); // 0=Idle, 1=Combat, 2=Flee
    var target = Variable<Entity>(() => null);

    // Global interrupt — checked every tick from any state
    OnTick(() => { if (Health < 20) state.Value = 2; });

    return () => state.Value switch
    {
        0 => idleNode ??= IdlePatrol(() =>
        {
            OnTick(() =>
            {
                target.Value = FindNearestThreat();
                if (target.Value != null) state.Value = 1;
            });
        }),

        1 => combatNode ??= CombatBehavior(() => target.Value, () =>
        {
            OnTick(() => { if (target.Value == null || !target.Value.IsAlive) state.Value = 0; });
            OnSuccess(() => state.Value = 0);  // Target eliminated
            OnFailure(() => state.Value = 2);  // Can't win → flee
        }),

        2 => fleeNode ??= FleeBehavior(() =>
        {
            OnSuccess(() => state.Value = 0);  // Escaped safely
        }),
        _ => JustRunning()
    };
});

State Behaviors Accept Lifecycle

// Each state accepts lifecycle for transition hooks
Node IdlePatrol(Action lifecycle = null) => Reactive * Sequence(() =>
{
    PatrolWaypoints();
    D.Condition(() => CanSeeEnemy);  // Succeeds when enemy spotted
    lifecycle?.Invoke();
});

Node CombatBehavior(Action lifecycle = null) => Reactive * Selector(() =>
{
    D.Condition(() => Ammo == 0); Reload();
    AttackTarget();
    lifecycle?.Invoke();
});

// Each state accepts lifecycle for transition hooks
Node IdlePatrol(Action lifecycle = null) => Reactive * Sequence(() =>
{
    PatrolWaypoints();
    D.Condition(() => CanSeeEnemy);  // Succeeds when enemy spotted
    lifecycle?.Invoke();
});

Node CombatBehavior(Action lifecycle = null) => Reactive * Selector(() =>
{
    D.Condition(() => Ammo == 0); Reload();
    AttackTarget();
    lifecycle?.Invoke();
});

Hierarchical State Machines

For complex AI, nest YieldSimple nodes to create hierarchical state machines. Each sub-FSM manages its own state space.

Node peacefulFSM = null, combatFSM = null;

Node RootFSM() => YieldSimple("Root", () =>
{
    var mode = Variable(() => 0);
    OnTick(() => { if (ThreatDetected) mode.Value = 1; });

    return () => mode.Value switch
    {
        0 => peacefulFSM ??= PeacefulFSM(() => OnSuccess(() => mode.Value = 1)),
        1 => combatFSM ??= CombatFSM(() => OnSuccess(() => mode.Value = 0)),
        _ => JustRunning()
    };
});

Node peacefulFSM = null, combatFSM = null;

Node RootFSM() => YieldSimple("Root", () =>
{
    var mode = Variable(() => 0);
    OnTick(() => { if (ThreatDetected) mode.Value = 1; });

    return () => mode.Value switch
    {
        0 => peacefulFSM ??= PeacefulFSM(() => OnSuccess(() => mode.Value = 1)),
        1 => combatFSM ??= CombatFSM(() => OnSuccess(() => mode.Value = 0)),
        _ => JustRunning()
    };
});

Architecture Overview

Root FSM

Peaceful

↔

Combat

↓

Sub-FSMs

Idle

Patrol

Gather

Engage

Flee

Heal

↓

Behavior Trees

Reactive Sequences, Selectors, Conditions...

Global vs Local Transitions

💡 Two Types of Transitions

Global (in OnTick): Priority interrupts checked every tick—"flee when health < 20"

Local (via lifecycle): Outcome-based transitions—"return to idle when combat succeeds"

The ??= pattern ensures each state node is created once and cached. Lifecycle callbacks attached at creation time define how that state transitions to others.

Benefits of the Hybrid Approach

🎯

Clear Mental Model

Designers think in states naturally. "What mode is the AI in?" is intuitive. BTs handle the complexity within each mode.

🔧

Maintainability

Add new states without touching existing ones. Add behaviors within a state without affecting the state machine structure.

♻️

Reusability

BT behaviors can be shared across states. The same MoveTo() works in Patrol, Combat, and Flee states.

🧪

Testability

Test state transitions separately from behaviors. Test behaviors in isolation. Compose tested components into full AI.

📊

Debuggability

Current state is always clear. BT debugger shows exactly what's happening within that state. Time travel through both.

⚡

Performance

Only tick the active state's BT. State transitions are O(1). No full-tree re-evaluation unless needed.

ClosureBT

Advanced Features