What three ambitious projects have taught us about the increasingly expensive relationship between artificial intelligence, engineering progress, computational effort, and the pursuit of completion.
The Paradox of AI-Assisted Development
One of the more remarkable developments in software engineering has been the extraordinary acceleration of the early stages of application development through artificial intelligence. Projects that would once have demanded weeks of architectural planning, implementation, and preliminary testing can now progress from a relatively informal description of their intended purpose to functioning software in a surprisingly short period. Complex subsystems materialize in hours, extensive test suites are generated almost as quickly as the code they are intended to validate, and capabilities that would previously have required substantial teams of programmers can be assembled by a single individual directing a collection of AI coding agents.
Yet our experience with this technology has revealed a curious and increasingly persistent contradiction: although artificial intelligence can dramatically reduce the time required to construct the bulk of a complicated system, the effort necessary to complete that system doesn’t appear to diminish proportionally. As development proceeds and the remaining defects become fewer, the expenditure of time, computational resources, and language-model tokens required to resolve each successive problem often increases rather than decreases.
The result is a development process characterized by an initial period of extraordinary productivity followed by an increasingly expensive struggle with integration failures, obscure defects, performance limitations, and the subtle interactions among components that were individually implemented and tested without difficulty.
This phenomenon is not entirely new, since conventional software engineering has always recognized that the final stages of development can be disproportionately difficult. What’s new is the extent to which AI-assisted development compresses the earlier stages, thereby exposing the expensive tail of the engineering process with unusual speed and clarity.
Over the past several months, we’ve encountered this pattern repeatedly while developing three substantially different projects: Snake-Eyes, a Django-based server administration platform intended to provide functionality comparable to commercial hosting control panels; Common Deformation Origin (CDO), a computational cosmology research program investigating alternatives to conventional explanations of the Hubble tension; and Heinlein, a Python-based AI novel editing system designed to analyze and revise manuscripts approaching the length of a full commercial novel.
Despite their differences in purpose, architecture, and computational demands, all three projects have followed a remarkably similar trajectory. Each began with major advances in capability, achieved through a coordinated system of AI architectural planning and autonomous coding agents, only to encounter increasingly stubborn obstacles as the initial construction phase gave way to the more demanding requirements of reliable operation and defensible results.
The experience suggests that the economics of AI development may be governed less by the cost of generating functional code than by the rising marginal cost of converting that code into a thoroughly verified system.
An Architecture Built Around AI Coordination and Execution
Our development process employs two distinct layers of artificial intelligence, organized around a division between architectural reasoning and implementation.
ChatGPT occupies the coordinating position, examining the current state of each project, evaluating technical evidence, considering architectural alternatives, and producing detailed engineering instructions intended for execution by coding agents operating within development harnesses. These agents, which we informally call our minions, are responsible for inspecting repositories, modifying source code, implementing new capabilities, executing automated tests, examining operational logs, and assembling evidence that can be reviewed during the next architectural iteration.
The human owner remains responsible for defining the mission, evaluating whether the results are useful, establishing acceptance requirements, and authorizing changes to important production environments. The AI systems perform much of the intermediate engineering labor, but their output is judged against requirements that ultimately originate outside the development harness.
This arrangement allows the two AI layers to specialize. ChatGPT can concentrate on the broader design and the interpretation of evidence without becoming entangled in every implementation detail, while the coding agents can devote their attention to the concrete work of translating instructions into functioning software.
The development cycle is therefore recursive: an initial architecture produces an implementation, the implementation produces test results and operational evidence, that evidence informs a revised architectural assessment, and the resulting instructions initiate another implementation cycle.
In practice, the process can be extremely productive, particularly during the early stages when broad architectural decisions and relatively independent capabilities dominate the workload. A single well-structured instruction set may produce an entire subsystem, complete with validation logic, automated tests, and documentation, leaving the coordinating model to evaluate results that would previously have represented many hours of conventional programming.
However, the same recursive architecture becomes more computationally expensive as the project matures, because each successive cycle must account for a larger existing codebase, a more complicated history of previous modifications, and increasingly restrictive requirements concerning what must remain unchanged.
The coding agents are asked to alter only a handful of lines, yet correctly identifying those lines and demonstrating that the modification does not disturb unrelated functionality ends up requiring extensive repository analysis, test execution, log inspection, and contextual reasoning. The amount of useful code produced per token tends to decline as the engineering process moves from construction toward integration and validation. This is where the initial productivity advantage encounters the less accommodating realities of system complexity.
Snake-Eyes: The Escalating Cost of Integration
Snake-Eyes provides perhaps the clearest illustration of the difference between building a complicated application and establishing that the application can operate reliably within an existing computing environment.
The project began with an ambitious objective: create a Python and Django administration platform capable of managing websites, WordPress installations, databases, email services, DNS records, FTP accounts, backups, service monitoring, and related server functions, while remaining compatible with installations already administered through other control panels.
The initial progress was substantial. The coding harnesses implemented major functional areas, constructed management interfaces, developed installation procedures, and introduced automated testing across much of the application. Subsequent development added the ability to recognize and import existing server resources, including WordPress installations, database relationships, DNS configurations, mail accounts, and services belonging to an established hosting environment.
By one laboratory qualification milestone, the project had recorded 149 passing tests, 24 explained skips, and 14 successful acceptance gates, demonstrating that a considerable portion of the intended functionality had been implemented and could be exercised under controlled conditions. Nevertheless, the transition from laboratory qualification to reliable production installation exposed problems that had little resemblance to the larger architectural challenges already overcome.
One issue involved installer transaction archives that accumulated approximately 18 gigabytes of data on a server with a 48-gigabyte root filesystem, while another involved excessive installation logs and backups that contributed to a subsequent production server failure. Other difficulties concerned domain ownership mappings, application deployment paths, reverse-proxy integration, service configuration, and recovery behavior during interrupted installation procedures.
These were not necessarily failures of the application’s major functional design; rather, they emerged from interactions between its installation and maintenance procedures and the operating environment in which it was expected to function. A particularly revealing example involved a WordPress administration button that, following an otherwise successful installation, directed the browser to a Snake-Eyes application route that returned a 404 error instead of reaching the existing WordPress login page.
From the user’s perspective, the required correction was almost trivial: an administrative button should open the appropriate WordPress login interface, which for a conventional installation is commonly accessible through /wp-login.php.
Yet implementing that behavior reliably requires considerably more than changing a URL. The application must recognize the existing WordPress installation, distinguish ordinary deployments from installations using nonstandard authentication paths, preserve compatibility with the surrounding web-server configuration, and demonstrate through browser and backend testing that the modification does not introduce regressions.
The contrast is striking. A considerable amount of infrastructure is created during a single productive development phase, while correcting a seemingly inconsequential navigation defect can require an entire additional cycle of architectural instructions, source-code modification, headed Selenium testing, SSH-based log inspection, and eventual production acceptance.
The difficulty is not proportional to the number of lines changed, because the expensive portion of the work consists of establishing that those changes are correct within the larger operating system. Snake-Eyes thus illustrates how the marginal cost of engineering progress rises even as the visible magnitude of the remaining problems diminishes. The application becomes more complete, but each remaining defect is increasingly likely to involve an interaction that earlier, more isolated testing did not adequately exercise.
Common Deformation Origin: Diminishing Returns in Scientific Computation
Our Common Deformation Origin research program presents a different manifestation of the same phenomenon, one in which the primary objective is not a deployable application but a computationally defensible scientific explanation.
The research investigates whether alternative cosmological constructions can reconcile observations associated with the Hubble tension, including differences between locally inferred cosmic expansion rates and those derived from early-universe cosmology. In pursuing this objective, we have explored hypothetical universes in which relationships that are ordinarily constrained by conventional cosmological assumptions can be separated, modified, or replaced by alternative mathematical prescriptions.
The AI development system has been especially valuable in constructing the computational infrastructure needed for this work. Numerical solvers, observational likelihood interfaces, distance calculations, optimization routines, parameter studies, and reproducibility mechanisms can be developed and revised much more rapidly than would be practical through conventional manual implementation alone. As a consequence, the research has progressed through numerous experimental generations, frequently producing substantial improvements in numerical agreement with selected observational datasets.
Yet the history of these experiments demonstrates how quickly the character of the remaining difficulty can change. In one Universe Transport experiment, the C1 model branch improved its chi-squared statistic from approximately 1439.445 to 1400.881, representing a reduction of about 38.564. The corresponding experiment nevertheless remained approximately 1.458 above its selected reference fit, while the C2 branch retained a larger residual difference of approximately 5.155.
Such results are instructive because they demonstrate how a relatively large numerical improvement can be followed by increasingly difficult efforts to eliminate the remaining discrepancy. Changes that initially produced substantial movement in the objective function eventually encountered narrower regions of parameter space, competing observational constraints, and uncertainty about whether additional flexibility represented meaningful explanatory power or merely a more accommodating fit.
Unlike conventional software development, however, scientific modeling cannot define completion exclusively in terms of whether the computational machinery produces the desired output. A model that reproduces selected measurements with sufficient freedom may still lack predictive value, while a mathematically elegant construction may fail when confronted with independent observational evidence.
This introduces an additional layer of difficulty, since the computational ability to improve a fit does not automatically establish the scientific validity of the underlying hypothesis. Greater freedom to modify the laws of a hypothetical universe can make observational agreement easier to obtain, but that same freedom can reduce the ability to distinguish the proposed explanation from other constructions capable of reproducing the data. As more adjustable relationships are introduced, the burden of demonstrating identifiability, consistency, and genuine predictive content becomes increasingly important.
For CDO, therefore, the expensive final stages involve more than squeezing a few additional units out of a likelihood statistic. They require establishing whether the resulting construction explains the observations in a way that survives meaningful tests beyond the data used to develop it. The scientific situation also differs from software completion in an important respect: the remaining discrepancy is not necessarily a defect that can or should be removed. A rigorous negative result, showing that a proposed mechanism cannot satisfy the relevant observational constraints, may represent a successful research outcome.
Nevertheless, the expenditure pattern remains recognizable. Early computational experiments yield substantial information and rapidly eliminate broad classes of possibilities, while later investigations concentrate increasingly large amounts of computation and reasoning on narrow residual differences, subtle degeneracies, and difficult questions about what the results actually establish.
Heinlein: When the Language Model Becomes the Limiting Resource
Heinlein, our AI-assisted novel editing system, offers a third perspective because its principal computational expense arises not only from the development process but also from the inference workload of the finished application.
The system was designed to evaluate complete manuscripts for copyediting problems, stylistic weaknesses, continuity errors, story inconsistencies, and other editorial concerns. Its architecture processes a manuscript through multiple analysis stages, preserving findings and applying validated changes while maintaining the integrity of the source document.
Initial development proceeded rapidly, as the coding agents established document extraction, text segmentation, editing passes, finding validation, and Word document generation. Early tests demonstrated that the approach could identify useful editorial issues and produce structured outputs for selected portions of a manuscript.
The difficulties became more pronounced when we moved beyond successful processing of individual chunks and demanded reliable operation across an entire novel of approximately 110,000 words. A full-manuscript execution involved 40 chunks and 344 jobs distributed across multiple editorial operations, creating a much more demanding environment for context management, streaming inference, checkpointing, error recovery, and aggregation of editorial findings.
The system encountered problems involving interrupted model responses, incomplete streaming records, recovery-state reconciliation, context accounting, and the preservation of previously completed work. These issues required increasingly elaborate mechanisms to distinguish a genuine model failure from an interrupted connection, prevent inappropriate duplication of inference, and ensure that final manuscript output could be traced to valid intermediate results.
Even after substantial improvements in reliability, a more fundamental performance limitation became evident. One full-novel diagnostic recorded approximately 2,830,192 generated output tokens, at an aggregate throughput of 33.52 tokens per second. At that generation rate, the output alone represents approximately 23.5 hours of inference, with the diagnostic attributing about 97.4% of the observed wall-clock span to token generation.
Unlike a short unit test or a targeted reproduction of a software defect, this workload has a physical computational cost that cannot be eliminated merely by improving the quality of the architectural instructions supplied to the coding agents.
Heinlein uses a locally hosted language model for important parts of its processing, which reduces dependence on paid inference services but does not remove the underlying expense. Millions of generated tokens still require GPU computation, memory bandwidth, electricity, and elapsed time, whether or not those resources are represented on an external provider’s invoice.
The distinction reveals two overlapping forms of diminishing returns. The first concerns the increasing effort required to perfect the software, while the second concerns the operational efficiency of the software once it functions correctly.
The program successfully processed every manuscript chunk, recovered from interruptions, preserved document integrity, and produced validated editorial findings, yet remained inefficient when the inference strategy generated far more tokens than are necessary to obtain useful editorial results.
Consequently, later development efforts must increasingly address questions of model behavior, prompt economy, output length, caching, and the reuse of valid intermediate work rather than simply improving functional correctness.
The program’s evolution demonstrates that the transition from prototype to dependable application is not merely a matter of eliminating bugs. It also requires bringing the computational cost of the intended workload into an economically and operationally acceptable range.
A Mathematical Model of the Development Curve
The experiences of these three projects suggest a general relationship between cumulative engineering expenditure and the fraction of a defined development objective that has been achieved.
Although no single equation can capture the irregularities of real software development or scientific research, an exponential saturation model provides a useful approximation of the diminishing returns we have observed:
P(E) = 1 − e^(−kE)
Here, P represents the proportion of a fixed completion target achieved, E represents cumulative engineering effort, and k describes the effectiveness with which that effort produces progress.
Under this model, progress is initially rapid because the earliest increments of effort resolve broad architectural problems and establish substantial functionality. As effort accumulates, however, each additional increment contributes a progressively smaller improvement, causing the completion curve to approach its limit asymptotically.
For illustrative purposes, assigning k a value of one produces the following progression:
| Cumulative effort | Completion | Additional gain |
|---|---|---|
| 0 units | 0.0% | — |
| 1 unit | 63.2% | 63.2% |
| 2 units | 86.5% | 23.3% |
| 3 units | 95.0% | 8.5% |
| 4 units | 98.2% | 3.2% |
| 5 units | 99.3% | 1.1% |
These figures are illustrative rather than measurements extracted from our development logs, but the shape of the curve closely resembles the progression encountered when complex systems move from broad implementation into increasingly specialized debugging and validation.
The implications become clearer when the equation is inverted to express the effort required to achieve a specified level of completion:
E(P) = −ln(1 − P) / k
In this model, the effort needed to progress from zero to 90% completion is equal to the additional effort needed to move from 90% to 99%, while advancing from 99% to 99.9% requires an equivalent expenditure once again.
This provides a mathematical explanation for the familiar experience of spending nearly as much effort addressing a small number of residual problems as was required to establish most of the original functionality.
More revealing still is the marginal expenditure associated with incremental progress:
dE/dP = 1 / [k(1 − P)]
As completion approaches its theoretical limit, the expenditure required for each additional unit of progress increases sharply, reflecting the growing difficulty of eliminating the remaining discrepancies.
Strictly speaking, the inverse of an exponential saturation curve is logarithmic, while its marginal expenditure increases hyperbolically as the completion target approaches 100%. The expression inverse exponential expenditure is therefore more useful as an intuitive description of the development experience than as a precise mathematical classification.
There is also an important limitation to this model: real projects rarely possess a stable or objectively measurable percentage of completion. Requirements evolve, unexpected failures reveal previously unrecognized work, and new information can alter what constitutes an acceptable result. In scientific research, progress may consist of narrowing possibilities or rejecting hypotheses rather than approaching a predetermined numerical target.
Nevertheless, the model captures an essential economic relationship: when the remaining problems become progressively harder to solve, a constant expenditure of effort produces progressively smaller gains.
Why Artificial Intelligence Intensifies the Effect
The diminishing returns associated with project completion were recognized long before generative AI entered software engineering, but contemporary coding models have altered the distribution of effort in ways that make the phenomenon unusually conspicuous.
Traditional programming places substantial demands on human attention during nearly every stage of implementation. Even a well-understood architectural decision requires a programmer to translate abstractions into source code, construct tests, diagnose ordinary errors, and integrate the resulting components into a larger system.
AI coding agents compress much of this routine work, particularly when requirements are clear and individual components can be developed with limited dependence on the rest of the application. The resulting productivity improvement is genuine, but it is not uniform across the entire engineering process. Generating an implementation is often much easier than demonstrating its correctness under every relevant operating condition, and producing a plausible numerical model is often much easier than establishing that it explains independent observations.
As a result, AI disproportionately accelerates the portions of development that are relatively well specified, easy to isolate, and amenable to automated generation. The activities least susceptible to this acceleration—complex integration, unpredictable operational failures, performance optimization, and adversarial validation—come to occupy an increasing share of total project expenditure.
This creates the appearance of an especially severe productivity collapse near completion, even when the absolute cost of debugging has not increased as much as the relative cost would suggest.
A conventional development team spends months constructing the major subsystems and additional weeks resolving integration problems. AI agents compress much of the initial construction into days while leaving the difficult integration work substantially intact. The late-stage problems then dominate the project timeline because the earlier work has become so inexpensive.
An additional complication arises from the increasing complexity of the systems that AI makes feasible. An individual developer can now undertake projects that would previously have required much larger teams, but the resulting applications will contain more components, dependencies, execution paths, and operational assumptions than the developer could reasonably have implemented alone. AI allows complexity to accumulate faster than the testing and verification systems can establish confidence in the resulting implementation.
The Hidden Expenditure of Tokens and Context
The conventional measurement of AI productivity often emphasizes the volume of code generated, the number of tasks completed, or the elapsed time required to satisfy a programming benchmark. These measurements provide useful information about model capability, but they reveal relatively little about the cumulative expense of bringing a complicated application through successive development iterations. Our two-tier architecture makes this distinction particularly important because each engineering cycle can involve substantial token expenditure across multiple models and activities.
ChatGPT must interpret the project’s existing state, review evidence from previous runs, identify likely causes of failure, and construct instructions detailed enough for the coding agents to execute without introducing unnecessary changes. The minions must then inspect relevant files, reason about dependencies, implement modifications, construct or adjust tests, and produce evidence of the resulting behavior.
When an attempted repair fails, some portion of this work is repeated, often with additional context concerning why the earlier approach was unsuccessful. As the project matures, the amount of contextual information required to make even a localized change may expand considerably. A modification that appears simple in isolation must be evaluated against accumulated compatibility requirements, earlier repairs, operational assumptions, regression tests, and the behavior of adjacent subsystems.
Consequently, the relationship between the amount of source code changed and the number of tokens expended becomes increasingly weak. A repair involving a few lines of Python now requires thousands of reasoning and context tokens, while demonstrating that the repair works correctly can require substantially more computation than generating the modification itself.
The problem is compounded when full-system tests are expensive. In Heinlein, rerunning a complete novel-editing canary to investigate a narrow orchestration failure consumes many hours of inference; in Snake-Eyes, repeating an installation acceptance cycle requires server preparation, browser automation, and backend verification; and in CDO, a small improvement in the objective function emerges only after extensive numerical exploration.
These activities represent different forms of expense, so elapsed time and token counts should not be treated as interchangeable. A scientific optimizer may consume substantial CPU time without generating many language-model tokens, while a local inference workload may generate millions of tokens with comparatively little intervention from the architectural coordinator.
A useful economic assessment must therefore account for both token expenditure and the broader computational resources required to obtain verified results.
Although we have detailed evidence from individual development and execution runs, we have not yet assembled a complete, standardized ledger of cumulative tokens, execution times, and accepted progress across all three projects. Accordingly, the exponential model should be understood as a hypothesis informed by repeated engineering experience rather than a statistically demonstrated universal law.
Measuring the Cost of Actual Progress
If the objective is to understand the economics of AI-assisted development, the most informative measurement is not the quantity of code produced but the relationship between verified improvement and total expenditure. Like I talked about before, a useful metric is the amount of accepted engineering progress obtained per unit of time, computational effort, or language-model consumption, with enough supporting detail to distinguish productive implementation from unsuccessful experiments and repeated validation.
Such measurements will need to be adapted to the nature of each project. For Snake-Eyes, accepted progress consists of capabilities demonstrated to function across the supported installation environments, with regression testing establishing that the new behavior does not damage existing services. A feature that appears complete in a laboratory environment but fails during production deployment is unfinished under this definition.
For Heinlein, useful progress would include both editorial quality and operational efficiency, since a system capable of identifying meaningful manuscript problems but requiring excessive computation to process them may not satisfy its intended purpose.
For CDO, the measurement is more subtle. Reductions in chi-squared can be informative, but they can’t serve as the sole measure of scientific advancement; an experiment that rigorously excludes a proposed mechanism, identifies an observational degeneracy, or establishes the need for independent data contributes more to the research objective than a small numerical improvement obtained through additional fitting freedom.
A standardized engineering ledger reveals important regularities across these different domains. Recording the tokens consumed by architectural analysis and implementation, elapsed execution times, tests passed, regressions introduced, computational resources used, and acceptance conditions satisfied provides a much clearer picture of the marginal cost of progress.
With enough observations, it’s possible to test whether expenditure follows an exponential saturation relationship, a power law, a sequence of plateaus interrupted by architectural breakthroughs, or some other pattern better suited to actual engineering behavior.
It also will reveal that the apparent exponential tail is partly an artifact of how development work is organized, rather than an unavoidable property of software complexity.
Managing the Expensive Tail
Our experience increasingly suggests that the largest remaining productivity gains will come not from making coding agents generate source code faster, but from reducing the amount of unnecessary reasoning, repetition, inference, and testing required to establish that their work is correct. This begins with distinguishing the broad construction phase from the more demanding acceptance phase. During initial development, maximizing implementation throughput is appropriate, provided the architecture remains coherent; as the system approaches operational readiness, however, the emphasis should shift toward reproducing specific failures, isolating their causes, and validating narrowly scoped corrections before committing resources to comprehensive regression tests.
The New Economics of Software Completion
The most important consequence of AI-assisted engineering will be that it changes which parts of development are economically scarce. Historically, implementation capacity was one of the principal constraints on ambitious software projects. Human programmers had limited time, teams were expensive to assemble and coordinate, and even well-understood functionality required substantial effort to build. Generative AI has loosened that constraint dramatically, making the construction of sophisticated applications accessible to individuals and small groups operating with resources that would once have been inadequate.
Yet the removal of one constraint has exposed others that are more resistant to automation: the difficulty of establishing correctness across interacting systems, the computational expense of large inference workloads, the uncertainty inherent in scientific interpretation, and the growing contextual burden of maintaining complicated software over many iterations.
Our development experience illustrates this transition with unusual clarity. In each case, AI enabled us to construct or investigate capabilities that would otherwise have required considerably more conventional programming labor, while the later stages concentrated increasing resources on a diminishing set of unresolved questions.
The pattern doesn’t mean that AI has failed to deliver its promised productivity improvements. On the contrary, those improvements are precisely what make the remaining bottlenecks so visible. By accelerating the construction phase, artificial intelligence has shifted the center of engineering effort toward activities where correctness, integration, performance, and independent evidence matter more than implementation speed.
This analysis is based on ongoing development work involving Snake-Eyes, Common Deformation Origin, and Heinlein, using ChatGPT for architectural coordination and AI coding-agent harnesses for implementation and testing. The mathematical examples are illustrative models of diminishing returns rather than fitted empirical measurements of the projects’ development histories.