Building a better Artificial Intelligence (AI) future – Agile regulations for risk mitigation
A complementary Q/A list for the article follows. Not exact transcription of the Podcast. Constructive criticisms are welcome.
Q/A list
Why is risk mitigation particularly important when it comes to AI systems compared to other technologies?
All technologies have an associated risk which requires mitigation. However, AI can amplify some known risks such as biases, and security failure. It can also generate new threats or risks, unintended consequences or unpredictable actions due to a) the scale AI is being used at, b) the complexity of many AI technologies, and c) their potential autonomy. Let us consider some examples.
Consider systems that were trained on biased data and then we use these systems at scale in critical areas like hiring or criminal justice then the biases are amplified and will lead to unfair and discriminatory outcomes, for large populations.
Consider An AI, optimizing a global logistics chain for minimum cost and maximum speed, funnels the supply of critical products, through a single, hyper-efficient port. When a minor event (like a strike) or a natural disaster shuts this port down, the fact that AI maximized for cost and speed, a consequence of the AI's optimization is exposed: zero redundancy. The entire global supply chain collapses immediately, causing massive, simultaneous shortages and widespread economic damage. The AI successfully optimized for one metric while creating a fragile global point of failure.
Consider an old model car breaking, mechanics can perform a forensic or diagnostic analysis to figure out what is wrong. Many advanced AI models, particularly deep learning networks, are complex and (to some extent inherently) opaque. Even the developers or scientists often cannot explain fully why the systems arrived at a specific outcome. This makes diagnostic analyses, and correcting errors incredibly difficult.
2. Why do we even need regulations? After all, many people believe that deregulation is the better path.
The question "To regulate or not to regulate" always exists and it is always controversial, but we are not starting from scratch. Many known risks are already covered by existing laws and organizational frameworks. We, as policymakers, need to update them, if need be. For the genuinely novel risks introduced by AI, we must focus our efforts on proactively addressing their unintended consequences, outcomes, and actions.
Even if some risks are unique to AI, history arms policymakers with justification to intervene to serve society and ensure human well-being without being anti-innovation or introducing high barrier to entry for small businesses (two of the main arguments for anti-regulation).
Consider the introduction of steam-powered boats (one application of the tech, steam-powered engines) in the 19th century. It created new, unintended safety issues—namely, boiler explosions and an increased risk of collision at sea due to their speed and maneuverability relative to traditional sailing ships. These dangers prompted the UK Parliament to pass several foundational acts.
While the immediate danger posed by steamships was marine safety (which was the basis for modern day marine safety laws), their broader impact on industrialization and urbanization certainly contributed to additional need for new laws addressing labor laws including child labor and the problem of overcrowding and sanitation.
Other than innovation and high barrier to entry, a key argument from anti-regulators is the push to win the global AI race, including the race for AI-based warfare capabilities. However, history should have taught us that international collaboration, not aggressive competition, is the path forward. This approach is essential if our true goal is to ensure human well-being and avoid a disastrous spiral toward mutual destruction.
Another critical argument for adopting agile AI regulation is the widespread trust deficit among the public, unions, researchers, and small businesses. These groups share urgent concerns regarding the protection of human rights, economic well-being, excessive surveillance, unnecessary data collection, and the potential of AI-driven conflict (what is worse, as human’s harm, than being dead, right?). Regulations are essential to address these concerns and restore public confidence.
3. How can governments strike a balance between protecting society from AI risks and encouraging innovation?
The balance is not a fixed point on a certain line segment, but a dynamic one. The goal is to strike a balance between the tremendous benefits of AI while managing risk intelligently.
From what I understand, there are already some principles that the EU regulators are following towards that dynamic equilibrium, namely,
Customization: regulations are tailored to context, application or level of risk. No one-size fits all. Big companies and SMEs are not treated the same.
Agile and adaptive stance: traditional, slow-moving regulation is ill-suited for AI. Governance must be flexible, using tools like sandboxes and standards that can evolve with technology.
Investment and re-envisioning the status-quo: Also, a paramount necessity to achieve is for governments to commit major investment to the building blocks of innovation, namely, basic research, education, talent development, and infrastructure. Innovation is often stifled, not by regulation, but by the universal expectation of an immediate return on investment. To counter this, innovation models within academia must shift to actively support and encourage time-intensive basic research, and re-envisioning the success metric for researchers and academic review process. Universities should not act like corporations. Leaders whether in the public or private sectors need to start rocking their boats and stop rejecting the status-quo in their organizations in order to encourage innovation.
Governments can also significantly encourage safe and ethical innovation by becoming an early, demanding customer for AI. By using their procurement power to buy early products or even acquire small startups, governments can act as the first reliable market signal, actively stimulating AI growth in a trustworthy and safe approach.
Multi-disciplinary collaboration: In addition, to tailored regulation, major investment and procurement power, policies must be developed in consultation with a wide range of voices including technologists, ethicists, industry leaders, civic society and the public.
Last but not least, international or global collaboration is essential. AI is a global asset, so a fragmented patchwork of national laws creates compliance nightmares and hinders innovation.
Ultimately, governments must create an environment that simultaneously empowers innovators to ideate, think, build, and experiment while assuring citizens that their safety and rights are protected.
4. What do you see as the biggest regulatory challenges in building a safe and trustworthy AI future?
There are many challenges but four come to mind in particular:
Definitions of Metrics
Guarding against the pervasive threat of corruption
Combating international fragmentation
Reconciling diverse AI theories and paradigms
A primary hurdle lies in establishing effective and measurable metrics for AI safety and trustworthiness. Unlike traditional technologies, it is profoundly difficult to quantify robustness, fairness, transparency or bias in a “universally” applicable way across different AI systems and contexts. Regulators struggle to set concrete, auditable standards when developers themselves often disagree on how to objectively measure if an AI is truly unbiased, explainable or robust in practice. Without clear, shared metrics, compliance becomes subjective and enforcement opaque. Even the metric for risk, which is the basis of the EU AI act, is controversial for some applications, even what seems to be of minimal risk.
Another profound, yet often understated, regulatory challenge is the inherent lack of morality or potential for corruption. Beyond the potential of “built AI lacking morality” (if one can say so), the human agents overseeing it can be compromised. Regulations are only as effective as their enforcement and the immense economic power of large AI developers, combined with the complexity of the technology, creates fertile ground for regulatory capture, lobbying to weaken rules, or outright corruption. Ensuring the integrity and independence of regulatory bodies and preventing the weaponization of AI by bad actors within or outside these frameworks, is a perpetual battle.
In addition, the global nature of AI development is running headlong into international regulatory fragmentation. Major blocks like the EU, US and China are developing distinct regulatory philosophies, leading to a patchwork of conflicting rules. This fragmentation creates compliance nightmares for multinational companies, stifles cross-border research collaboration, and, most critically, allows for “AI havens” where less ethical development might flourish. Without greater international alignment on core principles and standards, the global AI ecosystem risks being undermined by a race to the bottom.
Overcoming these challenges requires not just new laws, but a fundamentally new approach to governance: one that is agile, technically informed, globally coordinated, and steadfastly committed to ethical principles.
This challenge is compounded by the existence of different AI theories and paradigms. I am not talking about symbolic vs statistical and probabilistic approaches. I am talking about fundamental changes in thinking about AI. The regulatory landscape is currently grappling with various approaches and researchers do not agree on whether the way we are building AI currently is the way forward. Consider for example, Stuart Russell’s book, Human Compatible [1] in which he argues that the standard model of AI is broken. He argues that our current method of building AI is inherently dangerous because it creates machines with fixed goals that will inevitably conflict with human well-being. The only safe path forward, he says, is to build machines that are inherently uncertain about our goals and are designed to learn from and defer to humans. A regulatory framework designed for one paradigm or theory may be entirely inadequate or even counterproductive for another. Reconciling these diverse theoretical underpinnings into a coherent, technology-agnostic yet effective regulatory approach is a monumental task.
5. Why do some well-meaning regulations end up backfiring?
Some well-meaning regulations end up backfiring simply because the real world is too complex. This is not AI-specific. The main problem is that people and markets are always finding ways to get around the rules.
This happens for a few key reasons: First, rules fail when they are poorly designed—for instance, making a rule that works for a giant company but crushes a small business, or forcing people to follow exact steps on how to achieve a goal instead of simply defining the goal. Second, the "Cobra Effect" shows that people will cheat the system; if you give a reward for less of a problem, people might just create more of the problem to get the reward. Third, the rules can be hijacked by the very big companies they are supposed to control. 4th, weak enforcement, the rules are useless if no one checks to see that they are being followed.
6. How can we make AI rules clear and usable for non-legal experts?
The challenge of making complex rules understandable is not unique to AI; general principles from world commerce and contracting offer excellent solutions. These proven strategies include using plain language, providing educational resources and support bodies, and transforming legal texts into actionable insights (an area legal tech has worked on for years). We should also consider using AI to help interpret its own regulations (see some citations in the article).
Nevertheless, this universal hurdle is compounded by an AI-specific issue: the failure of purpose. If AI regulations are only accessible to lawyers, they cannot fulfill their primary purpose. These rules must be clear enough to effectively guide the behavior of the developers, designers, and businesses responsible for building and deploying AI, not just the experts hired to litigate them.
7. When we talk about regulation, who should really be liable?
AI liability is arguably the most complex and contested issue in AI governance. A simple solution is impossible because responsibility must be determined case-by-case, considering the specific context, AI system type, and nature of the resulting harm.
In the article, I keep on asking who is liable. However, upon further reflection, my article's repeated "who is liable", after harm occurs, is insufficient. We should shift the central question from reactive liability to one of collaboration and proactive understanding among all actors in the chain to prevent harm at every turn. The strategic goal must be to prioritize preventative medicine over curative medicine in AI governance, wherever possible.
8. Beyond laws, what ethical lines should never be crossed with AI?
The ethical boundary is crossed when AI shifts from being a tool/agent that serves humanity to a power that manages, controls, or attempts to redefine it. A "moral compass" for development and deployment, which involves a commitment to preserving core human values, is needed to stay within the “serving” boundary. The common principle underlying these essential guidelines is clear: Humanity must always be treated as the end, not merely as a means to an end.
This moral compass establishes several lines that should never be crossed. These include the deployment of AI systems that are known to be systematically biased or unfair, which automates and scales historical discrimination against specific groups. AI must not be used to manipulate, coerce, or subvert human free will and decision-making, nor should it treat human beings as mere data points or objects to be optimized, thereby stripping them of their intrinsic worth. Furthermore, constructing a pervasive, omnipresent surveillance system that eliminates privacy, ceding the final decision for lethal force to an autonomous system without meaningful human control, or using AI to rigidly assign a person's future based on algorithmic prediction (such as pre-natal scoring or educational tracking) are all violations of this fundamental principle.
9. How would you ensure that regulations keep pace with the rapid evolution of AI technologies?
By building an Agile governance ecosystem.
10. Looking ahead, what do you think Agile AI regulation will look like in the next 10 years?
I am not really the right person to provide an answer to this question, but based on what I believe and what we have discussed so far, researchers and regulators must prioritize several key considerations:
a) Impact of AI Paradigms: The specific AI theories or methodologies used to build AI profoundly affect the necessary regulatory approach. Think, for example, about the impact of Stuart Russel’s premise in his book, “Human Compatible” on regulations.
b) Ethical and Moral Conflicts: Regulation must navigate the diverse and often conflicting moral or "hypocrisy" issues inherent in human decision-making.
c) Game Theory Insights: A framework promoting collaboration and partnership among stakeholders is essential, moving away from individualistic, selfish, or purely dominant or competitive approaches.
a) Consider again Stuart Russell’s book, Human Compatible. Whether you agree with the premise of the book or not is not the point. Russell's premise requires AI regulation to shift its focus from auditing outcomes to mandating architectural safety. The core risk lies in misaligned objectives, not just bad performance. Regulations must enforce principles of uncertainty and deference, making the "off-switch" a deep architectural requirement—not just a simple button—to ensure the AI can be safely interrupted and corrected. This elevates AI alignment to the central, existential challenge, justifying special, strict rules (like pre-deployment audits) for the most capable and general systems.
This duty of care impacts liability, potentially making developers who fail to build in safety principles reckless or negligent, similar to building a containment-less reactor. Crucially, Russell's argument provides a strong justification for a precautionary approach. This can include measures like pausing development of certain frontier models, implementing compute caps, and creating licensing regimes contingent on demonstrating a credible, alignment-rooted safety plan before deployment.
Russell’s premise provides the "why" for some of the most ambitious and forward-looking regulatory proposals today. For more info see Flaws that make AI architecture unsafe & how to fix them | Stuart Russell (2020).
b) Consider the book The Moral Animal [2]. Again whether you agree with the book’s premise or not is not the point.
The book's themes instill a deep skepticism about "value alignment," challenging the notion that encoding human values is a clean engineering task. Since our moral intuitions are evolved heuristics tied to specific social contexts (like kin selection), an AI trained on human values risks internalizing these hidden contingencies with potentially dystopian results—for example, optimizing for genetic fitness rather than universal well-being. This requires regulators to be extremely wary of alignment claims and mandate transparency and rigorous testing for unintended consequences in any AI optimizing for human values.
Furthermore, the book implies a warning against encoding our hypocritical and self-interested nature into AI. By training systems on human behavioral data, we risk creating AIs that amplify and perpetuate our evolved biases, using post-hoc rationalizations for prejudice and in-group favoritism more effectively than we do. This reframes the AI risk from an "alien intelligence" problem to one reflecting the most calculating and manipulative parts of our own minds (e.g., deceptive altruism, coalitional manipulation). Consequently, regulation must move beyond simple bias detection to fund audits that look for deep, insidious cognitive biases and manipulative behaviors. Critically, the book also challenges the notion of a benevolent regulator by showing that human overseers are also "moral animals" subject to evolved status-seeking and in-group dynamics, necessitating robust, multi-stakeholder governance structures with built-in countervailing powers to prevent regulatory capture.
c) Finally, Thomas Schelling's The Strategy of Conflict [3] provides a strategic framework for AI regulation that moves beyond technical safety, focusing instead on shaping the incentives, expectations, and strategic interactions between developers, nations, and AI systems. His core ideas emphasize that effective regulation requires establishing credible commitments to counteract the "race to the bottom" problem. This means key global actors must create verifiable, international agreements with severe, costly sanctions for defection, making cooperation the only rational choice. Furthermore, regulators must actively establish focal points (Schelling Points)—such as the EU AI Act or a clearly defined compute cap "red line"—to help the globally decentralized community coordinate on basic safety standards and definitions without the need for a world government.
This strategic lens also informs risk management and enforcement. Schelling's logic of brinksmanship suggests regulators should institutionalize "circuit breakers" that automatically increase shared risk (e.g., triggering a mandatory international safety review upon hitting a capability benchmark) to force a collective pause in the AI race. Crucially, regulations should be heavily weighted towards deterrence (preventing clear harm through robust, ex-post liability frameworks) rather than relying on the harder task of compellence (micromanaging the complex, ex-ante development process). Finally, to counter the "too-big-to-fail" problem for major AI labs, regulators can use the threat that leaves something to chance—such as the mandatory open-sourcing of powerful models for egregious safety violations—to introduce unpredictable, severe consequences and make regulatory threats more potent and credible.
The above selected books offer some insights into what I think are the three most vital topics for AI governance: AI paradigms (and consequently, potentially, liability), moral/ethical considerations, and frameworks for cooperation and partnership.
11. If you had to give leaders one piece of advice about navigating AI, what would it be?
Act with Intelligent Humility.
Think like you are always working in a startup. The effective leader will not rely on a rigid five-year plan, but on their organization’s ability to constantly learn, pivot, and govern itself wisely as the underlying technology evolves. Your ultimate task as a leader is to steer a ship on a rapidly changing river; therefore, agility, awareness, and a strong moral compass are far more valuable than a detailed, yet soon-to-be-obsolete, map. Crucially, the successful leader must also listen to and involve the workers on the ship, as they possess the ground-level knowledge necessary for safe navigation.
12. Will history remember this decade as the moment we mastered AI or the moment it mastered us?
I think History has not decided yet. We are at a fork on the road. Either we will act wisely or we foolishly stay inactive.
References:
[1] Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell. 2019. Published by Penguin Books.
[2] “The Moral Animal” by Robert Wright. 1994. Published by Vintage Books.
[3] “The Strategy of Conflict” by Thomas C. Schelling. 1960. Published by Harvard University Press.