LÍMITES INTRÍNSECOS DE LA REGULACIÓN DE LA IA ("LÍMITES CUANTITATIVOS DE CONCIENCIA" (ACEMOGLU (2026, (II))

 

ABSTRACT 

We study how generative AI, and in particular agentic AI, shapes human learning incentives and the long-run evolution of society’s information ecosystem. We build a dynamic model of learning and decision-making in which successful decisions require combining shared, community-level general knowledge with individual-level, context-specific knowledge; these two inputs are complements. Learning exhibits economies of scope: costly human effort jointly produces a private signal about their own context and a “thin” public signal that accumulates into the community’s stock of general knowledge, generating a learning externality. Agentic AI delivers context-specific recommendations that substitute for human effort. By contrast, a richer stock of general knowledge complements human effort by raising its marginal return. The model highlights a sharp dynamic tension: while agentic AI can improve contemporaneous decision quality, it can also erode learning incentives that sustain long-run collective knowledge. When human effort is sufficiently elastic and agentic recommendations exceed an accuracy threshold, the economy can tip into a knowledge- collapse steady state in which general knowledge vanishes ultimately, despite high-quality personalized advice. Welfare is generally non-monotone in agentic accuracy, implying an interior, welfare-maximizing level of agentic precision and motivating information-design regulations. In contrast, greater aggregation capacity for general knowledge—meaning more effective sharing and pooling of human-generated general knowledge—unambiguously raises welfare and increases resilience to knowledge collapse. 

Daron Acemoglu Massachusetts Institute of Technology Department of Economics and NBER daron@mit.edu 

Dingwen Kong Massachusetts Institute of Technology dingwenk@mit.edu 

Asuman Ozdaglar Massachusetts Institute of Technology Department of Electrical Engineering asuman@mit.ed

The disagreements are in part about whether AI-provided information is a complement or a substitute to human learning. If the former, then expanding of AI will make humans put their effort and attention in where it matters and use AI’s inputs with growing effectiveness. In the substitutes case, however, better and better AI will increasingly discourage human effort and learning—because most relevant information comes to be served to humans on a platter.

This paper is an attempt to contribute to a better theoretical understanding of how AI tools impact human cognition and knowledge. We build a dynamic model of learning and decision-making where AI inputs can be either complementary or substitutable to human effort. At the center of our approach is a distinction between two types of information: general and individual- (or context-) specific. To perform any task, individuals require general knowledge. For example,for investment decisions one needs a basic understanding of different financial instruments such as treasury bonds, corporate bonds, stocks, options, etc., as well as information on how world stock markets and economies have been performing, some relevant aspects of their institutional structure, an understanding of macroeconomic risks etc. But one also needs information related to an individual’s context: what is the risk tolerance and planning horizon of the individual in question? What correlation is there between their other income sources and different asset returns? Do they have information, hunches, preferences or beliefs affecting how they should invest and what types of risks they should take? And so on. Notably, human decision-makers often acquire both general and specific knowledge jointly. For example, most individuals will learn about general financial knowledge in a finance course or reading relevant financial literature, and they will come to recognize their own needs and form their preferences and beliefs relevant for investment during the same process. Put differently, often there is economies of scope in learning, with the same efforts generating both general and individual- or context-specific knowledge.

Like humans, generative AI tools can also acquire both general and context-specific knowledge.
 

But it is their ability—especially the promise of the much-anticipated agentic AI models—to develop and provide context-specific information and decision support that is most promising. Building on their architecture that enables the storage and rapid inspection of vast amounts of data and inference on connections between context and relevant information, these models are promising to find patterns that are relevant to a specific context and uniquely useful for individual decision-makers.

Indeed, at some level, the Internet already aggregated a huge amount of general knowledge that was out there, and current generative AI tools can do this quite satisfactorily in many domains.

The big next step—with both great potential and danger—is the individual and context-specificaid from AI models. It is this type of context-specific recommendation from agentic AI that we focus on in our analysis. Returning to the investment example, text books and online resources can teach the mechanics of a broad range of financial instruments, but a future agentic system might translate an individual’s particular context into a concrete portfolio choice or even autonomously execute trades subject to that person’s conditions and constraints.

A key premise underlying our approach is that good prediction or performance and tasks typically requires both general knowledge and context-specific knowledge, and that these inputs are complements in the production of successful decisions. General knowledge makes context-specific evidence interpretable and valuable; conversely, context-specific knowledge pinpoints where the decision-maker sits within that general framework. Another important feature of general knowledge is that it builds on an entire community’s learning efforts—by its nature general knowledge can be shared. This implies that most of the general knowledge an individual generates is an externality that he or she does not internalize. These elements imply that additional general knowledge raises the marginal return to an individual’s learning effort: with more general knowledge, the same unit of effort is more beneficial for understanding and usefully acting on an individual’s specific context,and this motivates more effort when there is more abundant general knowledge. Thus general knowledge is complementary to human learning effort. By the same token, with context-specific recommendations from agentic AI, there will be less impetus to exert costly effort, because one of the objectives of this effort is already well served by agentic AI. Consequently, agentic recommendations are a substitute for human effort.

Our model embeds this relationship between different types of knowledge and individual learning effort into a dynamic model of community learning. An additional important element is that community-level knowledge is itself an input into the AI models—without human efforts, experiments and discovery there would not be enough valuable information for AI models to aggregate and sift through for either distilling general knowledge or for making individual-specific recommendations.

We model the decision problem of a collection of human agents as a prediction problem—each agent’s payoff depends on the distance of a common state representing general knowledge and their prediction about this state, and on the distance of their prediction about the context- state from its true value. In making these predictions, agents use their own learning effort, which generates two signals, one correlated with their context-specific state and another one correlated with the common state. By virtue of being about the common state, the latter signal is useful to all decision-makers, thus generating learning or data externalities. Agentic AI also provides context-specific recommendations, again modeled as predictions, which agents optimally combine with their own signals and priors.

The main result of the paper is a cautionary one: a powerful agentic AI model can statically help human decision-makers, but it can dynamically harm collective knowledge building. In fact,it can lead to what we call “knowledge collapse” whereby in the long-run equilibrium all human knowledge is ultimately destroyed.

Understanding the intuition of this result helps clarify our main contribution and the interpreta-tion of our results. It is not surprising that, statically, individuals gain from additional information,in the absence of any misspecification in their model of the world or other biases. They only reduce their effort because they already receive a fairly good recommendation from agentic AI, which is naturally a substitute to their effort. This substitution when done optimally cannot harm their utility statically. However, human effort feeds into collective knowledge, and this externality is not internalized by the agents. As they reduce their learning effort, the amount of information that either the community or AI models can aggregate starts diminishing. Then the long-run welfare effects depend on the balance of these two forces—better static decision-making versus less dynamic collective knowledge. We show that the negative effect is likely to dominate when agentic AI is very accurate. On the other hand, the ability of agents to learn from the general knowledge of others—either via community aggregation or more traditional AI or Internet-type tools—always improves welfare.

These same comparative statics also apply to the likelihood that the system converges to the knowledge-collapse steady state. Specifically, in domains where learning effort is sufficiently elastic,the system can exhibit multiple steady states: a high-knowledge one and a knowledge-collapse trap with zero general knowledge. As agentic AI improves, the basin of attraction of the knowledge-collapse steady state expands. Moreover, once agentic AI is accurate enough, a “complete collapse”occurs in which the high-knowledge steady state disappears and the system converges to the knowledge-collapse trap regardless of initial conditions. On the other hand, better aggregation of human-produced general knowledge has the opposite effect: it shrinks the basin of attraction of the knowledge-collapse steady state and can offset part of the discouragement effect of agentic AI.

We also consider several extensions to show both the flexibility of the model and the robustness of our main findings. First, in our baseline model, we assume that even without AI, human-generated general knowledge is relatively well aggregated and thus AI does not lead to a significant improvement in the aggregation of general knowledge. In our first extension we relax this assumption and establish that similar results apply even when AI simultaneously improves the aggregation of general knowledge relative to what the community was able to do and introduces the agentic element of providing individual specific recommendations to human decision-makers. Our results readily generalize to this case. Second, we investigate the extent to which “synthetic data,” generated byAI models without relying on human learning and experimentation can substitute for human effort.

We show that the same qualitative insights apply even with synthetic data, provided that such data is not a perfect substitute to human effort. However, in this case the knowledge-collapse steady state still features some positive amount of general information about the common state, since even without human effort synthetic data generates new knowledge. Third, we study a version of the model in which individuals can decide the direction of their learning effort, determining the balance between acquiring general knowledge and context-specific knowledge. Provided that general and context-specific knowledge cannot be perfectly separated, our results continue to apply in this case.

Conclusion

In this paper, we introduced a simple framework to study the implications of new generative AI technologies that promise to provide context-specific information and recommendations to human decision-makers. Our framework is based on three core ideas:

1. Good human decisions combine general knowledge with context-specific information.
 

2. Human effort directed at improving cognition generates both types of information, with the primary private return coming from context-specific information.

3. Individual contributions to general knowledge create externalities on others who build on this general knowledge.

These three observations together imply that the main motive for individual effort is often the acquisition of context-specific information, while the general knowledge an individual generates is primarily an externality. Consequently, better general knowledge in society is a complement to human learning effort, while better context-specific recommendations are substitutes. Because generative AI promises to provide this kind of context-specific information, it can be such a substitute, and reduce human effort. But then as lower human effort reduces general knowledge-building, generative AI (especially agentic AI) can dynamically push a society towards lower effective information and even lead to a knowledge-collapse steady state.

Our analysis is purely theoretical and shows that such a framework is tractable and yields a number of new and intuitive comparative statics. It also clarifies the conditions under which a knowledge-collapse steady state emerges and what determines how large its basin of attraction is.

The tractability of our model also enables us to consider a number of extensions, aimed at showing that our main qualitative insights are robust.

There are several interesting areas for future research, including on incorporating synthetic data and other new AI capabilities, and exploring whether they can break the strong substitution between context-specific AI recommendations and human effort. The tractability of our model makes a range of other theoretical applications and extensions possible, for example, looking at different types of efforts on the side of humans and the implications of different AI technologies that provide varying mixes of general and context-specific recommendations.

Our framework also provides guidelines on different types of effects that need to be measured empirically to evaluate the overall welfare impacts of new AI advances.

INTELIGENCIA ARTIFICIAL Y REGULACIÓN (TIM O'REILLY: "NO PUEDES REGULAR LO QUE NO ENTIENDES")

An idea for AI regulation: Start by formalizing and requiring detailed disclosure about the measurement and control methods already used by those developing and operating advanced AI systems. https://t.co/8h1FLVRgaM

— timoreilly (@timoreilly) April 14, 2023

 

 

 The world changed on November 30, 2022 as surely as it did on August 12, 1908 when the first Model T left the Ford assembly line. That was the date when OpenAI released ChatGPT, the day that AI emerged from research labs into an unsuspecting world. Within two months, ChatGPT had over a hundred million users—faster adoption than any technology in history.

The hand wringing soon began. Most notably, The Future of Life Institute published an open letter calling for an immediate pause in advanced AI research, asking: “Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?”

In response, the Association for the Advancement of Artificial Intelligence published its own letter citing the many positive differences that AI is already making in our lives and noting existing efforts to improve AI safety and to understand its impacts. Indeed, there are important ongoing gatherings about AI regulation like the Partnership on AI’s recent convening on Responsible Generative AI, which happened just this past week. The UK has already announced its intention to regulate AI, albeit with a light, “pro-innovation” touch. In the US, Senate Minority Leader Charles Schumer has announced plans to introduce “a framework that outlines a new regulatory regime” for AI. The EU is sure to follow, in the worst case leading to a patchwork of conflicting regulations.

All of these efforts reflect the general consensus that regulations should address issues like data privacy and ownership, bias and fairness, transparency, accountability, and standards. OpenAI’s own AI safety and responsibility guidelines cite those same goals, but in addition call out what many people consider the central, most general question: how do we align AI-based decisions with human values? They write:

“AI systems are becoming a part of everyday life. The key is to ensure that these machines are aligned with human intentions and values.”

But whose human values? Those of the benevolent idealists that most AI critics aspire to be? Those of a public company bound to put shareholder value ahead of customers, suppliers, and society as a whole? Those of criminals or rogue states bent on causing harm to others? Those of someone well meaning who, like Aladdin, expresses an ill-considered wish to an all-powerful AI genie?

There is no simple way to solve the alignment problem. But alignment will be impossible without robust institutions for disclosure and auditing. If we want prosocial outcomes, we need to design and report on the metrics that explicitly aim for those outcomes and measure the extent to which they have been achieved. That is a crucial first step, and we should take it immediately. These systems are still very much under human control. For now, at least, they do what they are told, and when the results don’t match expectations, their training is quickly improved. What we need to know is what they are being told.

What should be disclosed? There is an important lesson for both companies and regulators in the rules by which corporations—which science-fiction writer Charlie Stross has memorably called “slow AIs”—are regulated. One way we hold companies accountable is by requiring them to share their financial results compliant with Generally Accepted Accounting Principles or the International Financial Reporting Standards. If every company had a different way of reporting its finances, it would be impossible to regulate them.

Today, we have dozens of organizations that publish AI principles, but they provide little detailed guidance. They all say things like  “Maintain user privacy” and “Avoid unfair bias” but they don’t say exactly under what circumstances companies gather facial images from surveillance cameras, and what they do if there is a disparity in accuracy by skin color. Today, when disclosures happen, they are haphazard and inconsistent, sometimes appearing in research papers, sometimes in earnings calls, and sometimes from whistleblowers. It is almost impossible to compare what is being done now with what was done in the past or what might be done in the future. Companies cite user privacy concerns, trade secrets, the complexity of the system, and various other reasons for limiting disclosures. Instead, they provide only general assurances about their commitment to safe and responsible AI. This is unacceptable.

Imagine, for a moment, if the standards that guide financial reporting simply said that companies must accurately reflect their true financial condition without specifying in detail what that reporting must cover and what “true financial condition” means. Instead, independent standards bodies such as the Financial Accounting Standards Board, which created and oversees GAAP, specify those things in excruciating detail. Regulatory agencies such as the Securities and Exchange Commission then require public companies to file reports according to GAAP, and auditing firms are hired to review and attest to the accuracy of those reports.

So too with AI safety. What we need is something equivalent to GAAP for AI and algorithmic systems more generally. Might we call it the Generally Accepted AI Principles? We need an independent standards body to oversee the standards, regulatory agencies equivalent to the SEC and ESMA to enforce them, and an ecosystem of auditors that is empowered to dig in and make sure that companies and their products are making accurate disclosures.

But if we are to create GAAP for AI, there is a lesson to be learned from the evolution of GAAP itself. The systems of accounting that we take for granted today and use to hold companies accountable were originally developed by medieval merchants for their own use. They were not imposed from without, but were adopted because they allowed merchants to track and manage their own trading ventures. They are universally used by businesses today for the same reason.

So, what better place to start with developing regulations for AI than with the management and control frameworks used by the companies that are developing and deploying advanced AI systems?

The creators of generative AI systems and Large Language Models already have tools for monitoring, modifying, and optimizing them. Techniques such as RLHF (“Reinforcement Learning from Human Feedback”) are used to train models to avoid bias, hate speech, and other forms of bad behavior. The companies are collecting massive amounts of data on how people use these systems. And they are stress testing and “red teaming” them to uncover vulnerabilities. They are post-processing the output, building safety layers, and have begun to harden their systems against “adversarial prompting” and other attempts to subvert the controls they have put in place. But exactly how this stress testing, post processing, and hardening works—or doesn’t—is mostly invisible to regulators.

Regulators should start by formalizing and requiring detailed disclosure about the measurement and control methods already used by those developing and operating advanced AI systems.

In the absence of operational detail from those who actually create and manage advanced AI systems, we run the risk that regulators and advocacy groups  “hallucinate” much like Large Language Models do, and fill the gaps in their knowledge with seemingly plausible but impractical ideas.

Companies creating advanced AI should work together to formulate a comprehensive set of operating metrics that can be reported regularly and consistently to regulators and the public, as well as a process for updating those metrics as new best practices emerge.

What we need is an ongoing process by which the creators of AI models fully, regularly, and consistently disclose the metrics that they themselves use to manage and improve their services and to prohibit misuse. Then, as best practices are developed, we need regulators to formalize and require them, much as accounting regulations have formalized  the tools that companies already used to manage, control, and improve their finances. It’s not always comfortable to disclose your numbers, but mandated disclosures have proven to be a powerful tool for making sure that companies are actually following best practices.

It is in the interests of the companies developing advanced AI to disclose the methods by which they control AI and the metrics they use to measure success, and to work with their peers on standards for this disclosure. Like the regular financial reporting required of corporations, this reporting must be regular and consistent. But unlike financial disclosures, which are generally mandated only for publicly traded companies, we likely need AI disclosure requirements to apply to much smaller companies as well.

Disclosures should not be limited to the quarterly and annual reports required in finance. For example, AI safety researcher Heather Frase has argued that “a public ledger should be created to report incidents arising from large language models, similar to cyber security or consumer fraud reporting systems.” There should also be dynamic information sharing such as is found in anti-spam systems.

It might also be worthwhile to enable testing by an outside lab to confirm that best practices are being met and what to do when they are not. One interesting historical parallel for product testing may be found in the certification of fire safety and electrical devices by an outside non-profit auditor, Underwriter’s Laboratory. UL certification is not required, but it is widely adopted because it increases consumer trust.

This is not to say that there may not be regulatory imperatives for cutting-edge AI technologies that are outside the existing management frameworks for these systems. Some systems and use cases are riskier than others. National security considerations are a good example. Especially with small LLMs that can be run on a laptop, there is a risk of an irreversible and uncontrollable proliferation of technologies that are still poorly understood. This is what Jeff Bezos has referred to as a “one way door,” a decision that, once made, is very hard to undo. One way decisions require far deeper consideration, and may require regulation from without that runs ahead of existing industry practices.

Furthermore, as Peter Norvig of the Stanford Institute for Human Centered AI noted in a review of a draft of this piece, “We think of ‘Human-Centered AI’ as having three spheres: the user (e.g., for a release-on-bail recommendation system, the user is the judge); the stakeholders (e.g., the accused and their family, plus the victim and family of past or potential future crime); the society at large (e.g. as affected by mass incarceration).”

Princeton computer science professor Arvind Narayanan has noted that these systemic harms to society that transcend the harms to individuals require a much longer term view and broader schemes of measurement than those typically carried out inside corporations. But despite the prognostications of groups such as the Future of Life Institute, which penned the AI Pause letter, it is usually difficult to anticipate these harms in advance. Would an “assembly line pause” in 1908 have led us to anticipate the massive social changes that 20th century industrial production was about to unleash on the world? Would such a pause have made us better or worse off?

Given the radical uncertainty about the progress and impact of AI, we are better served by mandating transparency and building institutions for enforcing accountability than we are in trying to head off every imagined particular harm.

We shouldn’t wait to regulate these systems until they have run amok. But nor should regulators overreact to AI alarmism in the press. Regulations should first focus on disclosure of current monitoring and best practices. In that way, companies, regulators, and guardians of the public interest can learn together how these systems work, how best they can be managed, and what the systemic risks really might be.

Post topics: Artificial Intelligence