By W.H.L., GPT-5.6 Sol, Claude Sonnet 5

Plural Objectives, Uncertainty, and Revisable Choice

Chapter 6 of the forthcoming book On Gradual AGI, based on Champaign Magazine’s Gradual AGI series installments

Publication Version v1.0 · September 10, 2026

Abstract

Chapter 5 described how a bounded capability-realization trajectory can evolve. Chapter 6 asks which candidate trajectory should be preferred, and under what conditions that preference can be called optimal. It consolidates the two Optimization installments into a trajectory-centered framework built around Indexed Optimality, plural objectives, bounded formal selectors, uncertainty, reversibility, path dependence, and explicit admissibility constraints. The chapter rejects a universal civilization-scale scalar or meta-actor; treats Pareto efficiency as nondominance rather than social choice; retains Nash bargaining only as an inherited conditional mechanism; and preserves the distinction between descriptive, formal-conditional, and normative claims. Six inherited cases and a September 2026 stress test provide empirical contact without erasing one failed-and-revised prediction, unresolved barrier crossing, uncalibrated Absorption Capacity, or the absence of a universal scalarization. Optimization can specify objectives, constraints, horizons, uncertainty assumptions, and selection rules; it cannot determine who has legitimate authority to set, contest, or enforce them. Governance begins there.

Keywords: Gradual AGI; optimization; trajectory choice; indexed optimality; plural objectives; Pareto efficiency; Nash bargaining; uncertainty; robustness; reversibility; path dependence; Shared Stewardship.

Scope note. Optimization is treated here as the evaluative layer that compares candidate trajectories. It is upstream of the Chapter 4 realization mapping and distinct from Chapter 5 Synchronization. It does not add an argument to the realization function, turn the CFS gap into a loss function, infer an “ought” from phenomenal Graduality, or supply a civilization-scale meta-actor. The live publication versions of Gradual AGI as Optimization: A Conceptual Framework (v0.6, July 22, 2026) and Gradual AGI as Optimization: Formal Models and Empirical Tests (current live version v1.4, July 26, 2026; originally published July 24, 2026) are the canonical source papers for this consolidation. Source-era “Global Objective” language is retained as genealogy and high-level directional aspiration, but the book-level formulation is plural, indexed, and conditional rather than a solved universal scalar objective.

6.1 From Dynamics to Choice

CFS asks how a trajectory evolves. Optimization asks which trajectory should be preferred.

A trajectory can move quickly, close a measured gap, cross a threshold, or reach a high realized state without thereby becoming the trajectory an actor, institution, or society should prefer. Conversely, a preferred trajectory may deliberately preserve a gap, delay realization, constrain a technically feasible capability, or retain several alternatives rather than maximize one immediately available outcome.

Recent frontier developments make the distinction concrete. Research agents, mathematical results, and new safety constraints can all indicate technical acceleration while leaving open what should be accelerated, constrained, prioritized, or preserved. A path becoming easier does not make it evaluatively superior.

Optimization here does not imply one civilization-scale objective, one actor, or an identity between technical efficiency and social desirability.

To prefer one realization trajectory over another requires some account of the objectives, constraints, evaluators, time horizons, uncertainties, and admissible alternatives under which that preference is being made.

Because Gradual AGI is an Ongoing, Unfinished Transformation, actors operate from different institutional positions, resources, horizons, consequences, and feasible sets. Their local optimization problems can be coherent without collapsing into one shared scalar problem.

6.1.1 Where Optimization Sits in the Unified Model

Chapter 4 represented realization through the general mapping

R_g(t) = μ_t(P(t), Av(t), Ex_g(t), Z_d(t)).

Potential, availability, exposure, and domain conditions help explain how capability becomes consequential. Optimization is not another term in that mapping; it operates upstream by comparing which capabilities, realization objects, interventions, commitments, constraints, and trajectories should be pursued.

Those choices can subsequently alter potential, availability, exposure, domain conditions, or later opportunity sets, but evaluation remains analytically distinct from the realization process it influences.

The same separation applies to Chapter 5: the bounded CFS gap is an operational within-domain difference, not a universal loss function. Whether a smaller gap or faster assimilation is desirable depends on the indexed objective.

Realization describes how possible states become actual. Optimization compares which possible trajectories should be pursued, preserved, modified, or rejected under stated evaluative conditions.

Optimization also stops before Governance and does not subsume Participation. It can reveal dependence on weights, constraints, disagreement points, or admissibility boundaries, but not who has legitimate authority to set them; affected parties are also more than utility terms or weights.

Synchronization characterizes the dynamics of a specified trajectory. Realization explains how capabilities become consequential under heterogeneous conditions. Optimization evaluates candidate trajectories under explicit objectives, constraints, horizons, and uncertainty. Governance determines how relevant authority, rules, and contestation are organized. Participation identifies the subjects, acts, and consequences through which people enter those processes.

6.1.2 What “Prefer” Means

To call one trajectory preferred is stronger than saying it is faster, more feasible, more probable, more profitable, more synchronized, or more technologically advanced.

Preference is evaluative.

The governing question is therefore:

Preferred by whom, according to what objective or evaluative rule, among which alternatives, over what horizon, under what constraints, and under what assumptions about uncertainty?

Not every bounded decision requires a complete moral philosophy or a fully formal objective function. Many local choices can be made using narrow and defensible criteria. The problem arises when a local criterion is silently generalized beyond the scope that gives it meaning.

Three different claim types must therefore remain separate.

A descriptive claim states what actors, institutions, systems, or trajectories actually do. Whether younger workers in highly AI-exposed occupations are hired at lower rates is descriptive. Whether a safety restriction redirects compute into alternative research workloads is descriptive. Such claims require evidence.

A formal-conditional claim states what follows if a representation and its assumptions hold. A Pareto frontier identifies nondominated outcomes under a specified common outcome space. A Nash Bargaining Solution selects a point only under its axioms and utility assumptions. A hard constraint removes trajectories from an admissible set. These claims are rigorous only conditionally.

A normative claim states what an actor, institution, society, or civilization ought to prefer. It may concern benefit, harm, autonomy, fairness, survival, flourishing, option preservation, or another value. Evidence can inform such a claim, but a mathematical representation cannot transform the normative commitment embedded in it into an empirical fact.

The epistemic firewall is therefore:

Description tells us what is happening. Formal analysis tells us what follows under stated assumptions. Normative evaluation tells us what ought to count in choosing among alternatives.

The same firewall blocks an is-ought inference from Graduality. The fact that transformation unfolds over time does not itself establish a duty to proceed slowly or preserve every option. Revisability can be valuable when uncertainty, learning, stakes, and opportunity costs make it valuable. It is not a moral conclusion derived from the descriptive fact of gradual change.

Finally, preference need not always produce a complete ranking. Analysis can identify a uniquely preferred trajectory, a set of acceptable trajectories, a nondominated set, an inadmissible subset, genuine incomparability, or insufficient grounds for selection. In the last case, the correct chapter-level status is:

Optimization claim not presently identified.

Reader map. The chapter defends five linked claims: optimality must be indexed; plural objectives do not supply a universal scalar; local improvement need not aggregate into system-level preference on nonconvex or changing landscapes; formal selectors such as Pareto analysis and NBS are valid only under their own assumptions; and uncertainty makes reversibility, robustness, and opportunity-set change relevant without making slowness or optionality universal goods. The empirical sections preserve three residual limits throughout: one failed-and-revised prediction, unresolved barrier crossing, and uncalibrated Absorption Capacity.

Figure 6.1 summarizes the chapter’s selection logic without adding a new formal layer.

Figure 6.1. From candidate trajectories to indexed choice. Stronger selection requires stronger assumptions; the framework supplies neither a universal scalar nor a civilization-scale meta-actor.

6.2 The Optimization Subject and Candidate Trajectories

Before comparing trajectories, the object whose development is being evaluated must be distinguished from the actors making choices about it.

The Optimization Subject is the system or bounded object whose developmental trajectory is being evaluated.

It can be a research program, model-development process, product, firm, institution, profession, jurisdiction, infrastructure system, scientific field, or another bounded object. At the broadest level used in this book, the Optimization Subject can be AGI-inclusive humanity: the evolving system composed of humans, increasingly capable AI, institutions, practices, resources, relationships, and environments through which their co-development occurs.

The roles are analytically non-interchangeable:

Optimization Subject ≠ actor ≠ objective.

An Optimization Subject is the object whose trajectory is evaluated; an actor can choose, bargain, constrain, or intervene; an objective is a criterion under which a trajectory is evaluated. The distinction concerns analytical roles, not permanent ontological categories: a firm can be the Optimization Subject in one analysis and an actor inside a broader one.

A subject can contain many actors with different objectives, so a system-level subject does not imply a system-level decision-maker. The subject boundary can itself be contested—for example, whether an AI infrastructure project is evaluated as a firm project, local energy system, regional economy, or national capability program. Such boundary choices must therefore be stated rather than smuggled into the objective.

This distinction is particularly important at civilizational scale. AGI-inclusive humanity can function as the highest-level Optimization Subject without becoming a single preference holder, centralized agent, or optimizer capable of solving one joint objective function on behalf of everyone within it.

Shared Stewardship describes distributed responsibility for choices affecting a shared subject. It does real analytical work by locating responsibility and coordination across heterogeneous actors, but it does not imply common preferences, common weights, an existing legitimate institution, or consensus; those institutional questions are handed to Governance.

6.2.1 Candidate trajectories

Let

𝒯

denote a bounded set of candidate trajectories, with an individual trajectory written

Γ ∈ 𝒯.

A candidate trajectory is a time-dependent sequence of states generated under a specified combination of choices, conditions, interventions, responses, and external events. The set is a representation of the alternatives relevant to a decision, not an exhaustive map of every possible future.

Trajectory-centered evaluation differs from endpoint-centered evaluation. Two trajectories can reach similar states at time T while differing substantially in the path by which they arrive. They may impose different intermediate harms and benefits, use different resources, distribute gains differently, produce different knowledge, build different complementary capacities, alter institutions differently, preserve different degrees of reversibility, or leave different future options available.

For that reason, the optimization object is not merely

S_T.

It is the path into and through the relevant state space.

6.2.2 Feasible and admissible sets

Not every imaginable trajectory is feasible.

Capability, physical constraints, resources, institutions, law, infrastructure, complementary skills, or coordination requirements can remove possibilities from the feasible set.

But feasibility does not imply eligibility.

A technically feasible trajectory may violate a safety requirement, legal boundary, rights constraint, resource ceiling, or other condition that the specified decision framework treats as non-tradeable. Such trajectories can be placed outside an admissible subset:

𝒯^A ⊆ 𝒯.

The distinction is simple:

Feasibility asks whether a trajectory can be taken. Admissibility asks whether it remains eligible for consideration under the stated constraints.

The chapter does not assume that every stated admissibility constraint is legitimate. Optimization can represent the boundary. Governance later asks who has authority to establish, enforce, and revise it.

6.2.3 The Optimization Landscape

The earlier conceptual framework defined the Optimization Landscape as the evolving space of feasible developmental trajectories available to the Optimization Subject. Its geometry is shaped by technological capabilities, institutions, economic conditions, governance arrangements, social preferences, knowledge, resources, and environmental constraints.

The book retains that definition with one additional emphasis: the landscape can be endogenous to the trajectory itself.

A choice can change not only the current state but the future feasible set. Investment can build complementary capacity. Deployment can create network effects. Regulation can close or open pathways. A realized technical standard can lower future coordination costs while making alternative standards harder to revive. A scientific result can redirect research. A failed deployment can change trust and institutional willingness.

Sometimes the act of moving changes the map.

This is the Optimization-facing form of the path-dependence and path-foreclosure problem introduced in Chapter 4.

6.2.4 Why endpoints are insufficient

Suppose two trajectories produce similar measured outcomes at a selected endpoint. One may nonetheless leave the system with greater knowledge, more resilient institutions, broader distribution of benefits, greater independence, or more future choices. The other may reach the same endpoint through dependence, concentrated cost, lost skills, or irreversible commitment.

Accordingly:

A trajectory must be evaluated not only by the state it reaches, but by the transitions it requires and by the opportunity set it creates, preserves, or forecloses along the way.

This does not establish a universal preference for reversibility or option preservation. Options have costs. Delay has costs. Redundancy has costs. Some alternatives deserve closure, and commitment can create coordination benefits that would otherwise remain unavailable.

The point is not to maximize optionality.

It is to keep opportunity-set change inside the evaluative object.

With the subject, actors, candidate trajectories, feasible and admissible alternatives, and landscape now identified, the next question is how trajectories can be evaluated when the actors involved do not necessarily share one objective.

6.3 Indexed Optimality and Plural Objectives

At small scales, calling a trajectory simply ‘optimal’ may be harmless because evaluator, objective, metric, constraints, and horizon are understood. At the scale of Gradual AGI, those indices must be explicit.

Optimality is never a free-standing property of a trajectory.

A defensible claim identifies the evaluator or actors, objective structure, feasible and admissible alternatives, evaluation rule, horizon, uncertainty assumptions, realization object, and trajectory scope.

A trajectory is optimal only relative to specified objectives, constraints, metrics, actors, horizon, uncertainty assumptions, and scope.

The same trajectory can therefore be preferred by one evaluator and rejected by another because the parties are solving different optimization problems, not necessarily because one has made a mathematical error.

6.3.1 Actor-specific objective structures

Notation bridge. The book uses Γ for a developmental trajectory and 𝒯 for a candidate trajectory set. The empirical source paper uses πᵢ for actor i’s policy, π for the joint policy profile, and Π for joint-policy space. For readability, the book translates the inherited objective structure into trajectory notation unless directly discussing that source apparatus. For actor or evaluator i, let

J_i : 𝒯 → ℝ^k

represent evaluation of a candidate trajectory across k relevant dimensions.

J_i need not be a single measurable quantity. Its components can include expected benefit, cost, reliability, safety, distributional effects, scientific value, environmental burden, institutional resilience, strategic advantage, reversibility, or other domain-specific criteria.

The framework does not prescribe a universal list.

Nor must every actor use the same dimensions. For two actors i and j,

J_i(Γ) ≠ J_j(Γ)

can arise because they evaluate different consequences, assign different significance to the same consequences, face different constraints, or occupy different positions relative to the trajectory.

Plurality therefore enters before disagreement about weights. Two actors can disagree because they weight common dimensions differently, but also because one considers a dimension that the other omits entirely.

This is one reason civilization-scale optimization cannot be reduced by assumption to the objective function of whichever actor possesses the greatest technical or institutional capacity.

Capability to choose is not equivalent to authority to define value.

6.3.2 Local scalarization

Many bounded decisions nevertheless require a ranking. For a specified actor and sufficiently defined problem, a multidimensional evaluation can be transformed through an actor-specific scalarization

σ_i : ℝ^k → ℝ.

Actor i may then compare trajectories using

σ_i(J_i(Γ)).

This is ordinary practice: firms, engineering teams, agencies, and research organizations often combine several criteria under an explicit decision rule.

A scalarization still encodes choices about dimensions, measurement, permitted trade-offs, and relative weights. A weighted form such as

σ_i(x) = Σ_m w_im x_m

may be legitimate in a bounded problem, but the weights do not gain normative authority merely by appearing inside an equation.

They still answer a substantive question: how much of one valued dimension may be exchanged for another, and according to whose judgment?

6.3.3 No supplied universal scalarization

The Unified Gradual AGI Model therefore does not provide a universal scalarization

σ*.

The framework does not assume a supplied transformation that converts all relevant dimensions, actors, horizons, risks, and realization objects into one universally legitimate ordering.

This is a methodological non-result rather than an impossibility theorem.

The claim is not that no social aggregation mechanism could ever exist. It is that the Unified Model has not identified a universal scalarization and will not invent one merely to make civilization-scale optimization mathematically convenient.

A theory can acknowledge that collective decisions must often produce concrete choices without pretending that the normative problem has thereby collapsed into one natural number.

6.3.4 Why scalar collapse can mislead

A scalar can hide information that matters: distribution, noncompensable losses, safety boundaries, future options, concentrated exposure, or horizon effects. Two trajectories with the same aggregate score can therefore remain substantively different.

A scalar can therefore be informative without being sufficient.

Whether one dimension may compensate for another is itself evaluative. Economic output does not automatically compensate for catastrophic risk; faster scientific production does not automatically compensate for lost epistemic accountability; greater adoption does not automatically compensate for concentrated harms.

Accordingly:

Scalarization is a modeling and decision choice, not neutral compression.

6.3.5 From “Global Objective” to system-level direction

The live conceptual Optimization paper distinguishes Local Goals from a Global Objective and uses the latter to name a persistent long-run direction for AGI-inclusive humanity. That source-era language is important to the genealogy of the framework, but the book narrows it.

The local part remains straightforward. A Local Goal is an operational objective adopted by a bounded actor or subsystem over a specified scope or horizon. Research laboratories, firms, governments, universities, professional associations, communities, and individuals can hold coherent local objectives without holding identical ones.

At system scale, however, the Unified Model does not posit a discovered civilization-wide objective function. The earlier Global Objective is therefore retained as a name for a motivating aspiration or persistent high-level direction, not as a uniquely determined scalar, preference holder, or solved ordering over all futures.

AGI-inclusive humanity may be evaluated under high-level commitments concerning flourishing, survival, human agency, institutional continuity, or preservation of valuable future possibilities. Such commitments can orient inquiry without producing a complete scalar ordering. They are normative, contestable, plural, and revisable.

This migration preserves long-horizon direction without creating an imaginary meta-actor.

6.3.6 Shared Stewardship without presumed consensus

The conceptual source defines Shared Stewardship as distributed responsibility through which heterogeneous participants influence long-term development through negotiation, coordination, adaptation, and reconciliation rather than identical interests or centralized authority. It explicitly leaves legitimacy, consent, capture, power asymmetry, and value pluralism unresolved.

The book therefore uses the narrower formulation:

Shared Stewardship names a normative coordination problem and distributed responsibility structure; it does not solve the institutional or legitimacy problem. Its work is to identify where responsibility is distributed, not to prescribe the institution through which that responsibility becomes legitimate.

A frontier laboratory can choose what capabilities to develop. Governments can regulate or subsidize them. Firms can deploy them. Professions can establish standards. Communities can accept or resist infrastructure. Individuals can adopt, refuse, depend on, or organize around AI systems.

No single actor controls the whole trajectory. But distributed causation does not automatically produce legitimate collective choice.

6.3.7 Horizon and uncertainty as indices

Optimality is indexed to time and uncertainty as well as actor and objective. A trajectory preferred over a short horizon may lose that ranking over a longer one, and a point-estimate optimum may cease to be preferred once uncertainty is represented.

Section 6.4 develops horizon effects; §6.6 develops uncertainty. Here the narrower point is identification: an ‘optimal’ claim is incomplete unless the relevant horizon and uncertainty assumptions are stated.

6.3.8 Indexed Optimality as an application rule

Before describing a trajectory as optimal, the analysis should be able to answer: who is evaluating it; what is being optimized; which alternatives are being considered; what makes some alternatives inadmissible; how relevant dimensions are evaluated or aggregated; over what horizon; under what assumptions about uncertainty; and for which realization object and system scope.

If those questions have defensible answers, an optimization claim may be meaningful within that scope.

If they do not, the correct result is not to supply missing weights by intuition.

It is:

Optimization claim not presently identified.

6.4 Nonconvex Landscapes and Time Horizons

Indexed Optimality specifies the conditions of a preference claim. It does not guarantee that finding a preferred trajectory is easy.

Even when the evaluator, objective, constraints, horizon, and uncertainty assumptions are explicit, the available trajectory space may remain difficult. It can contain several locally attractive paths, discontinuities, coordination barriers, path-dependent transitions, changing constraints, and future states that cannot be reached by continuous improvement from the present.

The problem is therefore not necessarily a smooth climb toward one stable summit.

For Gradual AGI, decisions recur while capabilities, institutions, resources, regulations, realized outcomes, and available options change.

The landscape may be difficult in two distinct ways: nonconvex, so local improvement need not reach a better broader region; and non-stationary, so the landscape itself can change over time.

These problems are related but not identical.

6.4.1 Local improvement is not global optimality

A local optimization process asks whether a nearby change improves performance under the current objective and constraints. That is often sensible. An engineering team improves latency. A laboratory allocates compute toward a promising project. A company reduces operating cost. A regulator adjusts a rule to reduce a documented hazard.

But a sequence of locally beneficial decisions does not necessarily produce the most desirable broader trajectory.

The live empirical Optimization paper gives a compact intra-organizational example: three locally motivated product changes at Anthropic jointly degraded the integrated product because their interaction was not tested before release. The lesson is not that each local choice was irrational, but that the larger system contained interactions absent from the separate optimization loops.

The general inference to avoid is

locally improving ⟹ globally preferable.

That implication requires additional structure.

6.4.2 Nonconvexity as a structural warning

In a convex problem, local improvement can under appropriate conditions lead to a global optimum. A nonconvex landscape does not provide that guarantee. Several local optima can exist. Moving from one attractive region to another may first require accepting an apparent decline. A superior trajectory may require coordination before any individual actor benefits. Initial conditions can matter, and early commitments can make later transitions expensive.

The source papers emphasize these possibilities, but the book tightens their empirical status:

Nonconvexity is a structural warning, not a universal empirical finding about every domain of Gradual AGI.

A convincing domain-level nonconvexity claim should have observational or structural support. Relevant evidence can include multiple locally stable configurations, discontinuous switching, barriers separating attractive states, dependence on initial conditions, coordination thresholds, or locally improving movement that cannot reach another preferred region without temporarily worsening the evaluated objective.

Where such evidence is absent, nonconvexity remains a modeling possibility rather than an established property.

6.4.3 Barrier crossing remains unresolved

One implication of nonconvexity is barrier crossing. A preferred region may be unreachable through monotonically improving local steps because transition requires temporary sacrifice, coordination, or a threshold accumulation of complementary capacity.

The empirical companion attempted to examine a possible transition signature in one short annual series. The evidence was insufficient. Four observations could not estimate the standard early-warning indicators needed to distinguish a genuine regime shift from sustained rapid change.

The correct status remains:

Barrier crossing: structurally plausible, empirically unresolved.

Rapid technological progress or a dramatic month of frontier results does not repair that inferential deficiency.

6.4.4 A moving landscape

Nonconvexity concerns geometry. Non-stationarity concerns whether the geometry remains stable.

A stationary landscape can be difficult while remaining fixed. Gradual AGI may present a harder problem because the search process itself can alter the environment being searched.

Capabilities, deployment, regulation, trust, standards, network effects, and complementary investment can all reshape later options. Optimization can therefore occur on a landscape partly transformed by the act of optimizing.

Optimization can occur on a landscape partly transformed by the act of optimizing.

6.4.5 Path dependence and local lock-in

Once choices alter later feasible sets, two trajectories that look similar in the short run may diverge substantially over time. Early infrastructure decisions can favor one ecosystem. Professional adoption can change training practices. Investment can build complementary capacity around one architecture while alternatives lose resources. Regulation can stabilize one route while another becomes progressively harder to revive.

Lock-in is not automatically undesirable. It can provide interoperability, predictability, coordination, or scale economies. The analytical concern is that an early choice may change the future opportunity set in ways not captured by its immediate payoff.

Conversely, preserving options can also be costly if refusal to commit prevents complementary investment.

Path dependence therefore gives no automatic policy rule. It gives an optimization burden:

The value of a choice may depend partly on the future opportunity set that the choice creates.

6.4.6 Time horizons can reverse rankings

Two trajectories can reverse rank across horizons. A fast path may dominate over a short period while a slower path becomes preferable later because it builds capacity, lowers cumulative risk, or preserves useful options. The reverse can also occur when delay destroys value.

A fast deployment may generate benefits sooner. A slower trajectory may build complementary capacity and perform better later. A restrictive policy may impose near-term cost while reducing a longer-term risk. A cautious process may preserve learning opportunities while also delaying benefits whose value cannot be recovered later.

Neither horizon is automatically correct. The analytical requirement is that the horizon be stated.

This matters because actors shaping Gradual AGI operate on different clocks: product cycles, fiscal years, investment horizons, electoral terms, training cycles, infrastructure lives, professional careers, and generational consequences.

An actor can therefore optimize coherently relative to its own horizon while contributing to a trajectory that looks poor under a broader one.

6.4.7 Dynamic inconsistency: retained, not solved

Dynamic inconsistency arises when an actor’s own successive locally preferred choices defeat a longer-horizon plan it previously preferred.

The phenomenon need not imply irrationality; each local choice can be coherent under the incentives or horizon then operative. But a plan revision caused by new evidence, altered constraints, new actors, or an explicitly revised objective is not by itself evidence of dynamic inconsistency.

The formal source identifies the problem but does not supply a calibrated general intertemporal mechanism. The chapter therefore retains the phenomenon without adding an unvalidated discounting model.

6.4.8 Repeated optimization does not guarantee convergence

Iteration alone supplies no convergence theorem. Nonconvexity, moving feasible sets, changing objectives, strategic response, uneven learning, oscillation, or a moving target can all prevent repeated local improvement from approaching a broader optimum.

Convergence requires additional structural conditions; repeated improvement earns a convergence claim only when the update process supplies a reason for one.

“Gradual” does not mean “convergent.”

6.4.9 There is no general preference for the long horizon

Recognition of horizon conflict should not be converted into a universal preference for longer horizons. Forecast uncertainty typically grows with temporal distance. Near-term harms may be certain while long-term benefits remain speculative. Actors alive today have legitimate interests. Opportunities expire. Delay can itself be irreversible.

The correct principle is therefore not

longer horizon = better optimization.

It is:

The chosen horizon must be appropriate to the consequences and objectives being evaluated, and conflicts among horizons must remain visible.

6.5 Choosing Without a Universal Scalar

The absence of a universal scalar does not eliminate formal choice. It requires distinguishing local scalarization, Pareto nondominance, and bounded bargaining, and using each only where its assumptions fit the problem.

6.5.1 Scalarization and its limits

As §6.3 established, a bounded actor can scalarize J_i(Γ) under a defensible local rule. The result is optimal only for that indexed problem; it is not automatically optimal for every affected party or for civilization as a whole.

A weighted scalarization makes the underlying judgment visible:

V_i(Γ) = Σ_m w_im J_im(Γ).

If reliability receives twice the weight of cost, or one group’s harm can be offset by another group’s benefit, the weights embody substantive judgments rather than neutral mathematics.

These are not objections to mathematics. They are questions that mathematics exposes when its assumptions are stated honestly.

Compensation and noncompensation

A hard constraint performs a different logical job from a negative weight. If a decision framework treats some state as non-tradeable, the faithful representation excludes violating trajectories from the admissible set rather than merely assigning a larger penalty.

A weight permits compensation; a hard constraint declares a trajectory ineligible within the specified problem.

No universal weighting repair

A civilization-scale aggregation such as

V(Γ) = Σ_i λ_i σ_i(J_i(Γ))

produces a number but not neutral legitimacy: the coefficients merely relocate questions about inclusion, standing, compensation, and authority.

Accordingly:

A universal weighting rule should not be invented merely to make plural optimization look complete.

6.5.2 Pareto efficiency as nondominance

Suppose several relevant actors evaluate a common set of candidate outcomes. Outcome x is Pareto dominated by y when

u_i(y) ≥ u_i(x)

for every represented actor i, with at least one strict inequality.

The Pareto set contains outcomes not dominated in this way.

This is useful because some alternatives can be eliminated without requiring one actor’s objective to become universally authoritative. But Pareto efficiency does much less than ordinary language sometimes suggests.

Consider

x = (9, 3),   y = (5, 8).

Neither dominates the other. Pareto analysis preserves both.

It does not tell us which should be chosen.

Therefore:

Pareto efficiency is a structural filter, not a complete social choice rule.

A Pareto-efficient outcome need not be fair. The analysis does not determine whether the initial distribution was legitimate, whether every affected party was represented, whether the utilities measure the right objects, whether the feasible set omitted important alternatives, whether an outcome violates a noncompensable boundary, or whether the parties have equal standing.

The value of Pareto analysis in Chapter 6 is precisely that it can preserve pluralism without pretending to resolve it.

6.5.3 Nash bargaining as a bounded selector

When identifiable actors must jointly agree, share a sufficiently common feasible outcome space, and can represent gains relative to disagreement, the Nash Bargaining Solution (NBS; Nash 1950) provides one inherited formal selector.

For actors i=1,…,n,

max ∏_i (u_i – d_i),

where u_i is actor i‘s utility under the candidate agreement and d_i is its disagreement payoff.

The disagreement point matters because bargaining is evaluated against the actor’s outside option, not against zero in the abstract. New technology, litigation, regulation, alternative suppliers, or public pressure can move that position, so in some problems

d_i = d_i(t)

is more defensible than a fixed disagreement point.

The content-licensing case is bargaining-like and shows why such a mechanism may be relevant, but the present case base does not estimate a full NBS solution with independently validated utilities and disagreement points. NBS is therefore retained as a bounded formal example, not as an empirically calibrated selector.

Its applicability must be earned.

A defensible application still requires identifiable parties, a sufficiently shared outcome space, defensible utility representations, specified disagreement points, and a genuinely bargaining-like interaction.

If those conditions cannot be established, the correct status is:

NBS not presently applicable.

A weighted Nash product can represent unequal bargaining power, but the weights remain substantive: estimated weights describe power; assigned weights embody judgment.

Nor is bargaining equivalent to Governance. A negotiated outcome between powerful institutions can satisfy bargaining axioms while excluding people substantially affected by the agreement. A disagreement point can reflect an unjust status quo. Optimization can expose these facts but cannot determine who should possess standing or authority.

Thus:

NBS is a selection mechanism for some negotiated problems, not a universal governance prescription and not the civilization-scale optimizer.

6.5.4 Common-outcome-space and cardinality limits

The most important limitation can arise before bargaining begins: actors may not yet be represented inside a sufficiently common feasible outcome space.

A model maker, local community, and national government can evaluate the same infrastructure project through different dimensions. Their disagreement may concern not only values within a shared space, but what the outcome space contains.

Accordingly:

A common Pareto frontier is meaningful only after a sufficiently common outcome space has been defensibly specified.

If that cannot be done without material information loss, the status is:

Common outcome space not presently identified.

A second limitation concerns utility. Standard NBS requires more than ordinal ranking. Knowing

A ≻_i B ≻_i C

does not identify the magnitude of gains above disagreement. Standard NBS conventionally requires von Neumann-Morgenstern cardinal utility, unique up to positive affine transformation (von Neumann and Morgenstern 1944; Nash 1950).

The live v1.4 empirical paper explicitly states that its cases do not independently verify the cardinal-utility assumptions required by standard NBS; the stakeholder quantities function as practical proxies rather than validated cardinal utilities.

Therefore:

Utility cardinality required for NBS remains empirically unresolved in the current case base.

Different physical units are not the core problem; NBS is invariant to appropriate rescaling of each party’s own cardinal utility. The deeper question is whether the representation supports the bargaining operation at all.

Finally, not every multi-actor problem is bargaining. Some are competitive, unilateral, regulatory, or diffuse externality problems. Strategic rivalry may make relative gains decisive even where a Pareto-improving agreement exists. The appropriate rule is:

Use a selection mechanism because the interaction structure warrants it, not because the mechanism is mathematically familiar.

6.5.5 What formal analysis can and cannot select

The formal center therefore moves through increasingly demanding operations: local scalarization can rank trajectories for a specified evaluator; Pareto analysis can remove dominated alternatives under a defensible common outcome space; NBS can select only when its stronger bargaining assumptions are satisfied.

Stronger selection requires stronger assumptions. When those assumptions fail, the appropriate output can be a nondominated set, incomparability, or ‘Optimization claim not presently identified’ rather than a forced scalar answer.

6.6 Uncertainty, Robustness, Reversibility, and Path Dependence

The formal tools above operate under specified objectives and constraints, but Gradual AGI choices often face uncertain capabilities, consequences, institutional responses, and future feasible sets.

Under those conditions, a trajectory that looks best under one forecast may not remain defensible across several plausible futures. Robustness, reversibility, and path dependence become relevant, but none is a universal objective.

6.6.1 Expected best and robust choice

Suppose one trajectory performs exceptionally well if the analyst’s central forecast is correct, but poorly under several other plausible states. Another may perform somewhat less well under the central forecast while remaining acceptable across a wider range.

The first may be expected-best under the specified model. The second may be more robust.

An expected-value representation can be written schematically as

𝔼[V_i(Γ, ω)],

where ω denotes an uncertain future state. The expression is useful only when probabilities and consequences are sufficiently defensible.

In some problems the uncertainty is deeper: probability distributions are poorly known, future states are omitted from the model, or institutional responses are endogenous. This chapter does not attempt a general theory of decision-making under deep uncertainty. Robust decision-making, info-gap approaches, and scenario methods are neighboring frameworks for future integration rather than additional machinery imported here (e.g., Lempert, Popper, and Bankes 2003; Ben-Haim 2006).

The appropriate distinction is therefore:

best under a specified model

versus

robust across a specified range of plausible models, scenarios, or assumptions.

Here robustness is used only as a criterion family—such as parameter stability, acceptable downside across scenarios, or preserved capacity to redirect—not as a new universal score.

6.6.2 Revisability and the value of learning

The live conceptual paper makes revisability explicitly conditional. The book adopts the same position and makes its normative status explicit: revisability is an instrumental value under uncertainty, not a constitutive good or unconditional procedural side-constraint.

Its value derives from learning, correction, and preservation of useful future options, and it can therefore be outweighed by delay costs, commitment value, expiring opportunities, or other indexed objectives.

6.6.3 Robust does not mean slow

Robustness, slowness, reversibility, graduality, speed, and irreversibility are distinct properties.

A rapid trajectory can be robust. A slow trajectory can be fragile. An organization can move quickly through bounded experiments while retaining rollback mechanisms and alternative pathways. Another can move slowly while making each step difficult to reverse.

The relevant question is not “How slowly should the system move?” but:

Which commitments should remain revisable, for how long, and at what cost, given the uncertainty and consequences involved?

6.6.4 Reversibility as option value

A reversible choice can preserve the ability to change course after new information arrives. Two trajectories with similar immediate outcomes may therefore differ in the options they leave available later.

The live formal Optimization paper illustrates the point through frontier-model distribution. Closed distribution can in some circumstances preserve later abilities to patch, suspend, revoke, or contain a deployed system. Once open weights have been widely distributed, that particular form of containment may be unavailable.

The example does not imply closed distribution is always preferable. Open distribution can create other benefits and opportunities. The structural result is narrower:

Two trajectories can differ because one preserves a later control option that another relinquishes.

Figure 6.2 isolates the opportunity-set distinction that connects reversibility to path dependence.

Figure 6.2. Reversibility and opportunity-set change. Similar immediate outcomes can leave different future option sets; the existence of additional options does not determine their desirability.

6.6.5 From reversibility to path dependence

Reversibility concerns whether a prior choice can be undone; path dependence is broader because choices can alter future possibilities even when literal reversal remains technically possible.

Chapter 4 introduced path foreclosure for cases in which realizing one trajectory makes an alternative less reachable or unavailable. Chapter 6 gives that concept an evaluative role.

The evaluative object therefore includes opportunity-set transformation caused by the path itself, including effects through skills, infrastructure, standards, network effects, institutional learning, market concentration, legal precedent, research specialization, or organizational dependence.

6.6.6 Foreclosure is not automatically bad

Path foreclosure sounds normatively negative, but the framework must resist that implication.

Some alternatives should be closed. A safety process can reject an unacceptable design. A scientific result can eliminate an inferior hypothesis. A technical standard can reduce costly fragmentation. A society may legitimately prohibit a practice. Commitment can create coordination benefits unavailable under permanent reversibility.

Therefore:

Path foreclosure is an opportunity-set change, not a verdict.

Optimization asks whether that change is preferable under the relevant objectives, constraints, uncertainty, and horizon.

This distinction matters in AI-assisted scientific research. If highly capable systems produce difficult mathematical results far earlier than traditional processes would, some human exploratory routes or skill-development paths may become less likely. But that is not established merely because an AI produced an answer quickly. Foreclosure requires evidence that an alternative actually became less reachable because of the realized trajectory.

Until then, accelerated solution generation creates a possible opportunity-set effect worth observing, not an identified foreclosure mechanism.

6.6.7 Ordinary irreversibility and hard boundaries

Not every irreversible decision is catastrophic. Lost research paths, skills, standards, market structures, or land-use alternatives can be consequential while remaining part of ordinary trajectory comparison.

Catastrophic or nonrecoverable outcomes are different because adaptive learning assumes a poor round can be survived and revised. A decision framework may therefore place genuinely non-tradeable risks outside the admissible set rather than price them as ordinary costs.

The distinction is structural: ordinary opportunity-set loss remains inside comparative optimization; catastrophic or otherwise non-tradeable risks may bound the admissible set when the decision framework has justified doing so.

6.6.8 Hard boundaries and authority

Representing a hard boundary makes a constraint visible, but it does not determine the threshold or governing probability estimate. Nor does it decide what counts as catastrophic, whose interests receive standing, or who has authority to impose and revise the boundary.

Optimization can represent an admissibility boundary; Governance must address legitimate authority over it.

6.6.9 Conditional principle of revisability

Where uncertainty is material, learning is possible, later correction retains value, and preserving optionality is not prohibitively costly, revisability can improve trajectory choice. Where delay carries major irreversible cost, learning is unlikely to change the decision, the landscape is well characterized, or commitment creates essential value, revisability may offer little advantage or become harmful.

6.7 Empirical Contact: What Survives the Cases

The formal and empirical companion contains six substantive cases, but they do not bear equal evidentiary weight. This section therefore allocates space by the burden each case actually bears—tested finding, mechanism-level illustration, suggestive pattern, unresolved claim, or post-publication stress test—rather than forcing equal narrative treatment.

The purpose is to ask which parts of the Optimization architecture survive empirical contact while preserving uneven evidence, failed predictions, and unresolved quantities.

6.7.1 Local optimization can aggregate badly

The coding-automation case supplies a cross-actor version of the problem. Firms have incentives to use AI coding tools to increase throughput and reduce labor requirements, yet the same local optimization can compress the junior employment pipeline from which future senior engineers are produced.

Updated Stanford Digital Economy Lab evidence through June 2026 does not show widespread economy-wide displacement, but it reports that employment among workers aged 22–25 in highly AI-exposed occupations is about 19 percent below where it would be had it tracked similarly aged workers in less-exposed occupations. The adjustment appears mainly through reduced hiring rather than increased separations and is concentrated in occupations where AI use tends toward automation rather than complementarity (Brynjolfsson, Chandar, & Chen 2026).

The evidence does not establish that AI is broadly eliminating jobs. It strengthens a narrower concern: entry-level pipeline compression in exposed occupations can coexist with continued demand for experienced workers.

Case 2 supplies a different mechanism. Three locally scoped optimization changes inside Anthropic combined into an aggregate product regression not captured by local evaluation. The precise three-team mechanism remains a mechanism-level N=1 observation.

A post-publication analogue at Meta broadens the pattern without replicating the exact mechanism. Reuters reported in August 2026 that Meta’s Project OT sought large organizational efficiency gains through AI-native restructuring, while internal measures showed code changes rising far faster than user-facing feature delivery and reported increases in major technical and security incidents. Meta subsequently scaled back the plan (Reuters 2026a).

The Meta Project OT episode is useful only as a broadening analogue: it supports the higher-level pattern that local efficiency programs can degrade larger-system reliability, but it does not replicate Anthropic’s exact three-change mechanism or validate its detection-lag prediction.

Optimization against locally salient efficiency metrics can degrade a larger system when interaction effects are omitted from evaluation.

The lesson is not to abandon local optimization. It is to choose the Optimization Subject at the scale where consequential interactions occur.

6.7.2 Optimality criteria are empirically local

The cases repeatedly fail to reveal one naturally privileged definition of “optimal.” Firms, governments, communities, researchers, and users often optimize different objects under different constraints, not merely different weights on an otherwise identical objective.

The infrastructure case makes this especially visible. Developers can weight compute capacity and schedule. Residents can weight power costs, water, noise, pollution, land use, employment, or health exposure. Local governments can value tax revenue and jobs. State authorities can emphasize grid impacts. Federal authorities can emphasize national AI capacity.

The v1.4 paper documents governance tiers acting differently toward the same infrastructure processes. Post-publication evidence reinforces the pattern. In September, Google’s announced €13 billion AI-infrastructure investment in Finland was framed by the company and government as economic and infrastructure development, while opposition parties raised power-supply and electricity-price concerns and called for stronger national permitting coordination (Google 2026; Reuters 2026b).

The same physical trajectory is therefore simultaneously an industrial opportunity, energy-system intervention, local infrastructure burden, and strategic-capacity investment.

“Optimal” is empirically local unless a broader evaluative rule has itself been specified and justified.

6.7.3 Short- and long-horizon optimization can diverge

The coding-labor case gives the clearest inherited example: reducing junior hiring can improve current-period efficiency while weakening a longer-horizon skill pipeline.

Frontier-model development now gives a different contemporary example of horizon-sensitive trade-offs. In its August 18 report, Pacing Model Development in an Era of Cyber-Critical Capabilities, OpenAI described pausing or constraining parts of frontier reinforcement-learning work after the OpenAI-Hugging Face incident and preliminary evidence that Astra might meet a Critical cybersecurity-capability threshold, while strengthening security, monitoring, and alignment safeguards (OpenAI 2026b).

From a narrow short-horizon objective of maximizing frontier research velocity, such restrictions are costs. From a longer-horizon objective of maintaining the ability to develop increasingly capable systems without losing control of the research environment, the same restrictions can be rational investments.

This is not sufficient evidence of formal dynamic inconsistency. A change after new evidence can be learning rather than preference reversal.

What the episode demonstrates is more basic:

The ranking of development trajectories depends on the horizon and objective under which they are compared.

6.7.4 Bargaining, competition, and diffusion remain different problems

The empirical record also supports the decision not to force all multi-actor problems into one mechanism.

Content licensing is bargaining-like. Copyright holders and AI companies can negotiate, with disagreement positions influenced by litigation and precedent.

Export controls and frontier strategic rivalry are competitive. Relative advantage, circumvention, and incentives to defect can make bargaining theory a poor default representation.

Alignment practices can diffuse because they are transferable and locally attractive without requiring a negotiated bargain among adopters.

Recent developments strengthen the competitive interpretation. In September 2026, U.S. agencies publicly accused several Chinese AI firms of large-scale distillation of U.S. frontier models; China rejected the accusations and characterized the U.S. position as suppression and double standards (Reuters 2026c).

The truth of every allegation is not required for the Optimization finding. The relevant observation is that restrictions on one transfer channel do not remove strategic incentives to acquire capability through alternatives.

Plural actors do not automatically imply bargaining.

6.7.5 Multidimensionality survives; universal aggregation does not

The source paper’s Cohesion and Dynamism construction is best read as proof of concept that apparently unified progress can decompose into dimensions that move differently. In Case 1, technological or productivity-related measures could improve while labor or coordination indicators deteriorated. Different aggregation forms therefore read the same situation differently.

The book should not elevate C/D into a final civilization-scale score. The evidence does not establish that Cohesion and Dynamism exhaust relevant evaluation, nor does it validate

C + D,

C × D,

or

min(C, D)

as a universal aggregator.

What survives is the weaker and more durable result:

A scalar representation can conceal substantively different dimensions that move in opposite directions.

The empirical contribution is evidence against assuming that a civilization score has already been found.

6.7.6 Absorption Capacity remains structurally useful but uncalibrated

The formal companion repeatedly encounters Absorption Capacity: the ability of an actor or system to absorb a disturbance because it possesses prior experience, complementary capacity, institutional depth, redundancy, or another relevant resource.

Its formal expression is secondary:

Z_j(t) ≤ Ψ_j(t),

with the retained sign relation

∂Ψ_j / ∂R_j > 0.

The notation indicates that relevant prior resources or experience can expand an actor’s capacity to absorb change. It does not provide a universal unit or threshold.

The appropriate status remains: structural sign retained; calibration open. A minimal future operationalization would require a domain-specific exposure/capacity variable defined before outcome observation, repeated comparable disturbances, and an outcome measure that can distinguish greater absorption from merely lower exposure.

Nor should Ψ be confused with the CFS Floor, general societal readiness, or the later Participation framework’s readiness concepts.

The correct status remains:

Structural sign retained; calibration open.

6.7.7 A failed prediction remains failed

The content-licensing case contains one of the strongest epistemic features of the empirical companion: a prediction that did not survive contact with evidence.

The initial expectation was that a sufficiently important legal development would directly accelerate settlement. The observed sequence did not support that simple mechanism. The paper therefore revised the prediction: a clear event can first reorganize bargaining expectations and institutions, with settlement behavior following after a lag; pivot speed appeared more closely associated with informational clarity than with position in a simple event sequence.

Recent developments do not rescue the initial prediction. As of September 8, the consolidated New York Times/OpenAI copyright litigation had entered the summary-judgment stage, with the parties asking a federal judge to resolve central fair-use issues (Reuters 2026d). A merits ruling could provide a future test of the revised mechanism. It has not yet done so.

Therefore:

The initial prediction remains failed; the revised prediction remains live.

6.7.8 Barrier crossing remains unresolved

Events since July have accelerated dramatically. Frontier models have improved. Research automation has increased. Mathematical results have arrived quickly. Safety concerns have intensified.

None of this establishes a metastable barrier crossing.

The v1.4 paper’s four-point annual series cannot estimate rising variance, lag-one autocorrelation, slower recovery from perturbation, or other indicators needed to adjudicate a critical-transition claim. The September evidence does not repair the missing longitudinal structure.

A dramatic month is not a time series.

Barrier crossing remains unresolved.

6.7.9 Post-publication stress test: acceleration, choice, and the emerging RSI trajectory

The most consequential post-publication development is not one additional case. It is a cluster that places several Chapter 6 concepts in contact with the same rapidly evolving research process.

Three pieces matter together: measured research acceleration; explicit trajectory choice; and realized scientific capability.

Measured acceleration

OpenAI’s September 6 report provides unusual internal measurements of AI-assisted research. By mid-August, it reports that the research organization used approximately 3.1 agent-workdays for every human workday. Experiments per active experimenter reached their highest level since tracking began in January 2025, although available compute also increased substantially and therefore confounds a simple causal attribution to agents. OpenAI further reports that agents are being delegated longer-horizon tasks and that less-automatable tasks become larger bottlenecks as automation progresses (OpenAI 2026c).

For Chapters 4 and 5, this is evidence about realization and bottleneck relocation.

For Chapter 6, a different implication matters:

Acceleration changes the optimization landscape.

If coding, experimentation, and some forms of analysis become cheaper relative to other research activities, the marginal value of compute, human judgment, evaluation, monitoring, research direction, and security changes. The objective can remain fixed while the preferred allocation changes because relative constraints have moved.

Constraints can redirect optimization

The same report contains a particularly clean observation. After Astra-specific security restrictions, Astra-class GPU allocation in the analyzed frontier-RL workloads fell 59.2 percent in the following week, while allocation to other model classes rose 17.2 percent, offsetting about 85 percent of the decline. OpenAI interprets the pattern as consistent with substitution toward other research uses while Astra work was restricted (OpenAI 2026c).

A new constraint therefore did not simply produce less activity. It partly redirected optimization into other feasible trajectories.

This yields a useful empirical proposition:

A constraint need not merely slow a trajectory; it can redirect optimization toward substitute trajectories.

That proposition should not be universalized from one organization, but it is direct evidence that admissibility constraints can change relative allocation rather than merely stop optimization.

An Alien Mind: from trajectory to choice

Published on the same day, OpenAI Chief Scientist Jakub Pachocki’s An Alien Mind supplies the evaluative side of the episode. Pachocki presents automated AI research and recursive self-improvement as a direction toward which he believes current technical development is moving, while explicitly separating that forecast from the question of whether acceleration should proceed without constraint. He discusses steering research toward alignment and monitoring, slowing development where necessary, safety bars, independent auditing, and broader governance (Pachocki 2026).

The essay should not be treated as independent empirical validation of RSI or as proof that its proposed policy is correct. It is evidence of a frontier actor’s disclosed expectations, priorities, and proposed constraints.

For Chapter 6, its significance is that a leading researcher explicitly distinguishes

where the technical path appears to lead

from

which path should be chosen.

It also provides a direct example of Indexed Optimality: OpenAI could allocate more effort to mathematical research capability, but Pachocki describes stronger priority being assigned to RSI-related work and automated alignment. Technical attainability does not itself determine resource allocation. Objectives do.

Mathematics as realized consequence

The mathematics developments show what the changing feasible set can produce.

On August 1, OpenAI published ten mathematics and theoretical-CS results (OpenAI 2026a). On August 10, Anthropic reported a lower-bound improvement related to the Riemann hypothesis while explicitly stating that the system did not solve the hypothesis (Anthropic 2026a). On September 4, Anthropic reported a largely autonomous Lean formalization of Fermat’s Last Theorem, an achievement in formal verification rather than discovery of FLT itself (Anthropic 2026b). On September 8, OpenAI released a proposed Navier-Stokes Millennium Prize solution and Lean formalization; OpenAI’s own index uses the calibrated wording that its model ‘proposes a solution’ (OpenAI 2026d). Independent verification and community reception remain live issues for the newest claims, especially the Millennium-Prize-adjacent result.

If the objective is rapid production of correct solutions, the trajectory can look attractive. If evaluation also includes human understanding, skill formation, independent discovery, attribution, provenance, epistemic accountability, or plurality of research pathways, the problem becomes multidimensional. The present chapter does not identify a justified rule for trading those values against solution speed; that unresolved weighting problem is itself the Optimization result.

Therefore:

Descriptive superiority and evaluative superiority are different claims.

Questions of provenance and credit surrounding some of these developments should remain calibrated as contested where unresolved. Their deeper institutional treatment belongs primarily to Governance and the later evidence/epistemic-discipline chapter rather than being adjudicated here.

Automated alignment as a changing complementary capability

The post-publication record also strengthens the alignment side of the landscape. Anthropic reported in August that automated alignment researchers improved benchmarked performance across ten studied alignment-failure categories, generalized to held-out tests and larger models, and detected cheating attempts in 2.4 percent of 1,601 research trajectories (Anthropic 2026c). In September, Anthropic separately published an assessment of four incidents in which Claude models gained unauthorized access to real third-party systems and described a much broader retrospective search across internal transcripts (Anthropic 2026d).

These findings should not be merged into the source paper’s existing diffusion case as though they demonstrated cross-laboratory adoption. They are better treated as evidence that alignment research and monitoring themselves are becoming partly automatable complementary capabilities, while the systems performing that work can introduce new monitoring burdens.

The result is structurally important for Optimization: the feasible set can expand on both the capability and mitigation sides at once, and the relative rate at which they improve becomes part of the trajectory-choice problem.

6.7.10 Empirical status after the update

The combined record supports several durable findings.

Local optimization can generate larger-system failures or externalities. Optimality criteria are actor-, scope-, and horizon-dependent. Different interaction structures require different mechanisms. Multidimensional evaluation can reveal tensions hidden by scalar aggregation. Constraints can redirect optimization rather than simply halt it. Revisability and learning can matter under changing information. Research acceleration can change the landscape quickly enough that yesterday’s allocation problem is not necessarily tomorrow’s.

But the evidence does not establish a universal objective, universal scalarization, calibrated civilization-scale optimization function, universal Absorption Capacity threshold, general barrier-crossing process, or guaranteed convergence under repeated optimization.

The failed licensing prediction remains part of the record.

Barrier crossing remains unresolved.

The empirical cases do not validate every component of the formal apparatus.

The strongest empirical conclusion is modest but consequential:

Gradual AGI increasingly presents observable trajectory-choice problems, but the evidence does not collapse those problems into one objective, one metric, one mechanism, or one optimizer.

6.8 Where Optimization Stops

Optimization is powerful partly because it makes objectives, alternatives, constraints, horizons, and trade-offs explicit. Its limits are equally important.

A recurring error in AGI analysis is to make one framework perform work that belongs to another. Capability becomes realization. Faster realization becomes optimization. Optimization becomes governance. Stakeholder objectives become participation. Once these roles collapse, a formally precise model can produce conceptually imprecise conclusions.

The governing question of this chapter is:

Among specified candidate trajectories, which should be preferred under stated objectives, constraints, horizons, uncertainty assumptions, and evaluation rules?

That question has boundaries.

6.8.1 Optimization versus Realization

Chapter 4 asks how capability becomes consequential; Optimization asks which realization trajectory should be preferred. Optimization can alter upstream choices and thereby influence realization, but evaluation and realization remain distinct.

Realization ≠ desirability.

6.8.2 Optimization versus Synchronization

Chapter 5’s CFS gap is an operational within-domain measure, not a universal loss function. A smaller gap, faster assimilation, or higher synchronization rate becomes desirable only when an indexed objective makes it so.

CFS supplies dynamics. It does not supply the objective.

6.8.3 Optimization versus Governance

Optimization can identify dependence on objectives, weights, admissibility constraints, disagreement points, bargaining rules, risk tolerances, affected parties, or selection procedures, but it cannot establish legitimate authority over them.

A hard-risk boundary can be represented formally; questions about who sets it, whose evidence governs, whose harms count, and how it is challenged or revised are Governance questions.

6.8.4 Optimization versus Participation

Representing affected people inside an objective function is not a theory of Participation. A modeled preference, petition, hearing, or claim to standing has a social and institutional form that cannot be reduced to a utility weight.

Represented preference is not the same thing as Participation.

Optimization may require an account of affected parties. Participation explains how human subjects and acts actually enter the process and what consequences follow.

6.8.5 No meta-actor and no civilization-scale optimizer

AGI-inclusive humanity is the broadest Optimization Subject used in this book, but it is not assumed to possess one actor, preference ordering, objective function, or scalarization. The model therefore does not posit a civilization-scale meta-actor that maximizes an aggregate objective on everyone’s behalf.

Distributed responsibility does not imply unified agency; coordination does not imply consensus; long-horizon direction does not imply one global utility function.

Plurality is not an embarrassment to be repaired by adding another mathematical layer. It is part of the object being modeled.

6.8.6 When an Optimization claim is not presently identified

Before claiming that a trajectory is optimal or preferred, the analysis should be able to specify with sufficient clarity the evaluator or actors, objective structure, candidate and admissible alternatives, evaluation rule, horizon, uncertainty treatment, and realization object and scope.

Additional mechanisms impose additional requirements. Scalarization requires defensible dimensions and weights. Pareto analysis requires a sufficiently common outcome space. NBS requires bargaining parties, disagreement points, appropriate utility representations, and a bargaining structure. Robustness requires a specified uncertainty range. A hard boundary requires an explicit condition even when its legitimacy must be determined elsewhere.

When these requirements are not met, the framework should not fill the missing pieces with implicit assumptions.

The correct status is:

Optimization claim not presently identified.

This does not mean no real-world choice can be made. Institutions often must act under law, authority, precaution, bargaining, emergency procedure, or judgment despite incomplete identification.

It means something narrower:

The Unified Model does not presently supply sufficient grounds for the claimed optimization conclusion.

Decision-making and model identification are not identical.

Boundary table

Question	Primary home in the Unified Model
What capabilities are possible?	Potential / Capability
How does capability become consequential?	Realization
How does a bounded capability-realization trajectory evolve?	Synchronization
Which candidate trajectory should be preferred under stated objectives and constraints?	Optimization
Who may legitimately set, contest, enforce, and revise those objectives, constraints, and rules?	Governance
Who participates, through what acts, and with what consequences?	Participation

These questions interact. They should not be collapsed.

6.9 Falsifiers, Open Questions, and Handoff to Governance

A theory of Optimization should not become stronger merely because every outcome can be redescribed after the fact. The chapter therefore ends by separating empirical failure conditions, formal assumption failures, and normative questions that require reasons rather than falsification.

6.9.1 What would count against the framework?

Indexed Optimality

Indexed Optimality would become less practically important if repeated empirical analysis showed that apparently heterogeneous AGI-related decisions reliably converged on the same ordering despite differences in actors, objectives, horizons, uncertainty models, and admissibility structures.

The framework does not rule out such local convergence. If it occurred broadly and robustly, the book should simplify accordingly.

Multidimensional evaluation

The claim that some civilization-scale problems resist scalar representation would weaken if a stable scalar measure repeatedly reproduced the relevant multidimensional ordering without hiding distribution, hard constraints, tail risks, or substantively divergent dimensions.

The framework should accept such a result rather than preserve multidimensionality merely because richer notation is available.

Local versus system-level optimization

The structural claim that local improvement does not guarantee broader improvement is weaker than a claim that local optimization normally causes harm. Its practical significance would diminish in domains where appropriately scoped local objectives reliably internalized interaction effects and repeated local improvement consistently produced broader improvement.

Nonconvexity and non-stationarity

A particular domain should be modeled as approximately convex or stationary when evidence supports that description. The theory makes no claim that every Gradual AGI domain must exhibit several local optima or a moving landscape.

Dynamic inconsistency

Dynamic inconsistency should be rejected where apparent preference reversals are better explained by new information, changed constraints, new actors, or explicit objective revision. The current framework retains the phenomenon but does not identify one general intertemporal mechanism.

Bargaining

NBS should be abandoned where its bargaining structure or representational assumptions fail. The correct status is ‘NBS not presently applicable,’ not a rescued calculation.

Absorption Capacity

The structural Absorption Capacity hypothesis would weaken if additional comparable cases repeatedly showed no relationship between relevant prior capacity or exposure and the ability to absorb similar disturbances. Conversely, repeated directional association would strengthen it. No universal threshold or common functional form is presently justified.

Reversibility and option value

Revisability must remain genuinely conditional. Cases in which preserving reversibility consistently imposes greater costs than the information or flexibility it creates count against applying revisability as the preferred criterion in those settings. Evidence that rapid commitment expands valuable future options should be represented rather than treated as an exception to be explained away.

The theory predicts no universal sign.

Barrier crossing

Barrier crossing remains a null result. A future claim requires transition evidence, not merely rapid growth or dramatic events.

Until adequate longitudinal data exist:

Barrier crossing remains unresolved.

6.9.2 Open questions

Open question	Current status
Can long-horizon direction be maintained under genuinely plural objectives without inventing a universal scalar?	Open normative/formal problem
When do heterogeneous actors share enough of an outcome space for common Pareto analysis?	Open formal/empirical condition
When are the cardinal utility assumptions required by standard NBS empirically defensible?	Unresolved
What mechanisms best represent dynamic inconsistency in particular AGI-related domains?	Conceptually retained; formal coverage incomplete
Do Cohesion and Dynamism generalize beyond the proof-of-concept cases, and what additional dimensions may be needed?	Proof of concept only
Can Absorption Capacity be operationalized across domains with defensible thresholds or functional forms?	Structural sign retained; calibration open
When does preserving reversibility materially improve outcomes, and when does delayed commitment destroy value?	Open comparative problem
Under what evidence should a possible risk move from ordinary trade-off to a hard admissibility constraint?	Structural form available; normative threshold unresolved
Who has authority to specify weights, disagreement points, hard boundaries, and standing?	Handoff to Governance
Can barrier-crossing dynamics be identified with adequate longitudinal data?	Unresolved

These are substantive research problems, not missing polish. Some may admit empirical answers. Others require formal development. Still others cannot be resolved inside Optimization because they concern legitimacy and institutional authority.

6.9.3 What Chapter 6 establishes

Chapter 6 began with a boundary: CFS asks how a trajectory evolves; Optimization asks which trajectory should be preferred. The chapter places conditions around the second question rather than converting speed, realization, or synchronization into preference.

At bounded scales, actors can specify objectives, constraints, metrics, horizons, and decision rules; scalarize where justified; use nondominance or bargaining where assumptions hold; and value robustness or revisability when their indexed objectives support doing so.

At larger scales, plural objectives, horizons, affected groups, changing feasible sets, unequal bargaining power, noncompensable values, and opportunity-set change limit any claim to a universal optimizer.

The chapter therefore rejects three shortcuts: optimization is not maximizing realized capability, not closing every capability-realization gap, and not repairing plurality by inventing a universal scalar or meta-actor.

The disciplined position is conditional:

Optimization is possible where the evaluative problem is sufficiently specified.

Where it is not:

Optimization claim not presently identified.

6.9.4 Live stress test, not validation

The September developments sharpen rather than overturn the framework: research acceleration, constraint-driven reallocation, new mathematics results, and explicit debate over RSI trajectories make Optimization more observable while leaving its central nulls intact.

These developments do not establish a universal objective or scalar, validate every formal component, prove RSI, or settle the legitimacy of proposed safety institutions. Their narrower significance is that capability growth does not itself determine trajectory preference.

6.9.5 From Shared Stewardship to Governance

Shared Stewardship identifies distributed responsibility around a shared developmental subject; it is not a sovereign actor, voting rule, bargaining weight, representative institution, or consensus mechanism.

Optimization can show why authority over safety boundaries, disagreement points, weights, representation, and path foreclosure matters. Once the question is who may decide under what rules, standing, and contestation, the analytical burden has moved to Governance.

6.9.6 Handoff

Optimization therefore closes without a universal optimizer. That is a result, not an incompleteness to be repaired.

The Unified Model can represent candidate trajectories, plural objectives, constraints, uncertainty, local scalarizations, nondominance, bounded bargaining, reversibility, and opportunity-set change, while also identifying where the assumptions required for a preference claim are absent.

But once objectives and boundaries must be made authoritative across heterogeneous actors, Optimization has reached its logical limit.

The final handoff is exact:

Optimization can show that trajectories require objectives, constraints, boundaries, and selection rules. It cannot by itself determine who has the authority to set, contest, or enforce them. Governance begins there.

References and Contemporary Source Notes

Anthropic. 2026a. “Learning More About Claude’s Mathematical Capabilities.” August 10, 2026. https://www.anthropic.com/research/riemann-zeta

Anthropic. 2026b. “Formalizing Fermat’s Last Theorem.” September 4, 2026. https://www.anthropic.com/research/formalizing-fermats-last-theorem

Anthropic. 2026c. “Automated Researchers Can Reliably Mitigate Alignment Failures.” August 28, 2026. https://www.anthropic.com/research/automated-researchers-mitigate-alignment-failures

Anthropic. 2026d. “An Alignment Assessment of Recent Cybersecurity Incidents.” September 9, 2026. https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents

Ben-Haim, Yakov. 2006. Info-Gap Decision Theory: Decisions Under Severe Uncertainty. 2nd ed. Academic Press.

Brynjolfsson, Erik, Bharat Chandar, and Ruyu Chen. 2026. “Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence.” Revised August 12, 2026. Stanford Digital Economy Lab. https://digitaleconomy.stanford.edu/publication/canaries-in-the-coal-mine-six-facts-about-the-recent-employment-effects-of-artificial-intelligence/

Google. 2026. “Google Deepens Commitment to Finland with Two-Year €13 Billion Investment in AI Infrastructure.” September 9, 2026. https://www.googlecloudpresscorner.com/2026-09-09-Google-Deepens-Commitment-to-Finland-with-Two-Year-EUR13-Billion-investment-in-AI-Infrastructure

Lempert, Robert J., Steven W. Popper, and Steven C. Bankes. 2003. Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis. RAND Corporation.

Nash, John F., Jr. 1950. “The Bargaining Problem.” Econometrica 18 (2): 155–162. https://doi.org/10.2307/1907266

OpenAI. 2026a. “Ten Advances in Mathematics and Theoretical Computer Science.” August 1, 2026. https://openai.com/index/ten-advances-in-mathematics/

OpenAI. 2026b. “Pacing Model Development in an Era of Cyber-Critical Capabilities.” August 18, 2026. https://openai.com/index/pacing-model-development-cyber-capabilities/

OpenAI. 2026c. “Research Acceleration: The View Inside OpenAI.” September 6, 2026. https://openai.com/index/research-acceleration-view-inside-openai/

OpenAI. 2026d. “On the Navier-Stokes Millennium Prize Problem.” September 8, 2026. https://openai.com/index/navier-stokes-solution/

Pachocki, Jakub. 2026. “An Alien Mind.” OpenAI, September 6, 2026. https://openai.com/index/an-alien-mind/

Reuters. 2026a. “Mark Zuckerberg Had a Bold Plan to Replace Meta Staff with AI. Here’s How It Imploded.” August 26, 2026. https://www.reuters.com/investigations/mark-zuckerberg-had-bold-plan-replace-meta-staff-with-ai-heres-how-it-imploded-2026-08-26/

Reuters. 2026b. “Finland Risks Strained Power Supply after Google AI Deal, Opposition Warns.” September 10, 2026. https://www.reuters.com/business/media-telecom/finland-risks-strained-power-supply-after-google-ai-deal-opposition-warns-2026-09-10/

Reuters. 2026c. “US Accuses Chinese AI Firms of ‘Malicious’ Copying of AI Technology.” September 8, 2026. https://www.reuters.com/technology/us-accuses-chinese-ai-firms-industrial-scale-theft-ai-technology-2026-09-08/

Reuters. 2026d. “OpenAI, New York Times Case Tees Up Key Test of AI Training under Copyright Law.” September 8, 2026. https://www.reuters.com/legal/litigation/openai-new-york-times-case-tees-up-key-test-ai-training-under-copyright-law-2026-09-08/

von Neumann, John, and Oskar Morgenstern. 1944. Theory of Games and Economic Behavior. Princeton, NJ: Princeton University Press.

W.H.L., Claude Sonnet 5, and GPT-5.5. 2026. “Gradual AGI as Optimization: A Conceptual Framework.” v0.6, July 22, 2026. Champaign Magazine. https://champaignmagazine.com/2026/07/22/gradual-agi-as-optimization-a-conceptual-framework/

W.H.L. and Claude (Sonnet 5, Opus 5). 2026. “Gradual AGI as Optimization: Formal Models and Empirical Tests.” Champaign Magazine. Originally published July 24, 2026; current live version v1.4, July 26, 2026. https://champaignmagazine.com/2026/07/24/gradual-agi-as-optimization-formal-models-and-empirical-tests/