Why AI Regulation Needs to Look Beyond the Model

The Model Isn't the Whole Picture

When regulators and boards talk about AI risk, the conversation almost always starts — and too often ends — with the model itself. Which algorithm is running? How was it trained? What's in the training data? These are important questions, but they miss a critical reality: in production, the model is just one component of a much larger system.

As researchers like Ethan Mollick have pointed out, the way we think about AI safety and regulation is fundamentally incomplete when we focus solely on model capabilities. A less capable open-source model connected to the right tools, skills, and automation pipelines can pose very different risks than a more capable closed system that's tightly constrained. The regulatory challenge isn't just technical — it's architectural.

What Makes AI Systems Truly Complex

Consider two scenarios. In one, a cutting-edge frontier model is deployed behind a narrow API with strict rate limits, no tool access, and no memory. In another, an older, less capable open model is given access to a database, a code execution environment, the ability to call external APIs, and a long-running agent loop. Which one poses greater systemic risk?

The answer isn't obvious — and that's precisely the problem. The risk profile of an AI system depends on at least four layers beyond the model itself:

• Harnesses and guardrails: The same model behaves differently when wrapped in safety constraints versus given free rein. Harnesses can both increase and decrease capability depending on their design.
• Skill and tool ecosystems: A model that can browse the web, execute code, and query internal databases has a fundamentally different risk surface than one that only generates text.
• Connected system topology: How the AI integrates with existing infrastructure — cascading API calls, data pipelines, human-in-the-loop checkpoints — determines failure modes.
• Evolving capability boundaries: As models are fine-tuned, prompted, and augmented, their effective capabilities shift. Static benchmarks capture only a snapshot.

Why Bright-Line Rules Fall Short

Regulators naturally gravitate toward bright-line rules — clear thresholds that separate acceptable from unacceptable. A model with X parameters trained on Y data under Z conditions is safe; anything beyond is not. These approaches are appealing because they're administrable, but they fail to capture the combinatorial complexity of real AI deployments.

The European AI Act's tiered approach — categorizing systems by risk level — represents genuine progress. But even this framework struggles with systems whose risk profile changes as they acquire new tools, connect to new data sources, or are deployed in new contexts. A system classified as "limited risk" at deployment might behave very differently after six months of iterative augmentation.

A Framework for Enterprise AI Governance

For enterprise decision-makers, the regulatory uncertainty around AI isn't an excuse for inaction. It's a call to build governance frameworks that match the complexity of the systems they're governing. Here's a practical starting point:

1. Map the full system, not just the model. Document every component — models, API integrations, data pipelines, tools, memory stores, human checkpoints, and escalation paths.
2. Assess capability drift over time. Models change through fine-tuning, prompt evolution, and tool augmentation. Regular reassessment is not optional.
3. Build layered guardrails with watchpoints. No single safeguard is sufficient. Design overlapping controls — input validation, output filtering, human review, and automated anomaly detection.
4. Document provenance and change history. When something goes wrong, the ability to trace exactly which component changed and when is the difference between a fixable incident and a regulatory crisis.
5. Engage proactively with emerging frameworks. The regulatory landscape is still forming. Enterprise teams that participate in standards development and pilot programs will shape the rules rather than just complying with them.

Regulatory Complexity as a Competitive Signal

The organizations that treat AI governance as a strategic function — not just a compliance checkbox — will have a structural advantage as regulations mature. When bright-line rules prove inadequate, the companies with the most comprehensive system-level documentation and the most mature governance processes will face the least regulatory friction.

The complexity of AI regulation isn't a bug; it's a reflection of the technology's genuine sophistication. Models matter, but the systems we build around them matter just as much. The enterprises that understand this distinction will be best positioned to innovate responsibly.

Building Your AI Governance Strategy

Navigating the regulatory landscape requires more than compliance expertise — it requires a deep understanding of how AI systems actually behave in production. Otonomi helps enterprise teams build governance frameworks that match the real complexity of their AI deployments, from model selection through full-system architecture review.

Book a strategy consultation to discuss your AI governance needs.