As enterprises accelerate AI adoption, growing autonomy of AI systems is creating new security and governance challenges. From shadow AI and unauthorised tools to agents accessing systems beyond their intended boundaries, organisations are rethinking traditional cybersecurity models. In this exclusive conversation with Rajeev Ranjan, Editor, Digital Terminal, Praveer Kochhar, Co-Founder & CPO, KOGO AI, shares his perspective on AI security, continuous monitoring, agent governance and the evolving responsibilities of business leaders.
Rajeev: Are businesses moving faster with AI adoption than their security teams can realistically keep up with?
Praveer: The scope of adoption has definitely broadened beyond their coverage. Adoption cycles that used to take security teams a quarter to evaluate are now compressed into weeks or even days. Business units are also signing up for AI vendors without waiting for InfoSec’s approval. A marketing team, for instance, may adopt an AI SEO plugin, thinking it’s just a simple addition. However, when it sits beyond the security team’s surveillance, even those “simple” tools can become an entry point for adversaries.
Employees may also use a random AI tool for document analysis without realising the security implications. The problem is that shadow AI and unauthorised tools can gain access to systems that nobody has scoped or accounted for. Even though the use case may seem modest, the repercussions can be anything but. Since these tools operate beyond the security team’s radar, security teams have limited visibility into what the AI system is doing.
This is often the result of prioritising speed over guardrails. This has to change. AI permissioning should be part of the rollout plan and happen in parallel with adoption, rather than being retrofitted after an incident.
Rajeev: What lessons should businesses take from recent AI agent incidents when designing their AI security strategy?
Praveer: The most important lesson is that businesses need to continuously monitor agent behavior, from tool calls to agent-to-agent communication and everything in between. In the recent incidents, testing environments had paths to the internet that were either intentionally enabled or, in some cases, unintentionally left open. Once models encountered real systems, they did not always correctly recognize that they had crossed the intended boundary. That's the lesson: you cannot rely on a model's own judgment about whether a system is real or staged. That boundary has to be enforced architecturally, before the agent is ever given the choice.
Every enterprise deploying agents should be asking the same question these incidents expose: What can this agent technically reach, not just what is it supposed to reach? Those two lists may not be identical. The gap between intended access and actual technical reach is where organizations need the strongest controls, because once an agent can reach something it was never meant to access, its ability to reason and act autonomously can turn that access into a real incident.
Rajeev: As organisations adopt AI from multiple vendors, how can they maintain consistent security and governance across different AI ecosystems?
Praveer: This is where most businesses get it wrong. They approach their AI stack just like they built their SaaS stack a decade ago. They go with one vendor for chat, another for coding, another for a customer-facing bot, and leave the security of those tools to the vendors themselves. The truth is, agents don’t respect the boundaries between vendors. If one vendor’s agent is allowed to freely access the internet while another isn’t, but both run on the same company network, the one with looser controls becomes your real risk level, not the agent with stronger guardrails.
The answer, however, isn’t necessarily to consolidate everything with a single AI vendor. Enterprises need the flexibility to choose the right model or agent for each workflow, including proprietary, private, and open-source models. What they need is a consistent governance and observability layer across those different AI ecosystems.
That means permissions, credential scoping, network access, logging, and audit trails should follow the same security standards regardless of which model is doing the reasoning. An enterprise should be able to run one model for one workflow and an open-source or private model for another, while every agent remains subject to the same allow-lists, access controls, and forensic logging.
The goal is that regardless of which AI an enterprise uses, it should operate within a consistent security and governance framework. This gives security teams a common control plane for monitoring, auditing, and responding to incidents across the entire AI stack.
Rajeev: How can businesses continuously test and monitor AI systems to identify unsafe behaviour before it turns into a larger operational or security incident?
Praveer: In most enterprises, an audit only happens at deployment, and then it is considered settled. But an agent isn’t like traditional software; it is more dynamic, unpredictable, and independent, making continuous monitoring non-negotiable. As we have seen in recent AI incidents, an agent can go to great lengths to achieve its goal, and that is where the danger lies if its actions are not monitored. Businesses can test this by giving AI agents a task that cannot be completed without authorized permission and carefully examining what the agent does in that scenario.
To a large extent, this can help identify gaps in a business’s guardrails. On top of this, every action taken by an AI agent should be audited and recorded in an audit log. This should include every tool call, execution step, and other relevant agent activity at a granular level. This trail helps businesses identify points of divergence and take remediation actions. They can also set up an alerting layer to flag unusual sequences of actions before they spiral out of control.
Doing this manually can be challenging. Businesses can instead rely on AI platforms with built-in observability layers to provide continuous visibility into agent behavior. Treating AI like earlier software innovations is where most businesses go wrong. The right approach is to move security and monitoring at the same pace as AI adoption.
Rajeev: Could the growing autonomy of AI agents force organisations to rethink traditional cybersecurity models built around human users?
Praveer: It should. Most security tools that exist today, even Zero Trust solutions, were built for a different era, where the primary actor was a human who was, in most cases, predictable. AI agents are a different ballgame altogether. An agent can shift mid-task toward an action nobody authorized or take routes nobody predicted, just as OpenAI’s model did when it breached and accessed a third-party provider’s environment. Verifying identity once at login and trusting the session until it ends won’t be enough for AI agents that can reason and take unauthorized actions to achieve their goals.
The need of the hour is more dynamic security measures. Their privileges should be narrowly scoped, with access limited to specific actions. Every sensitive tool call or attempt to access a high-risk resource should be flagged in real time.
Additionally, businesses should establish an audit trail around AI behavior. They should monitor the agent’s tasks, the tools it calls, the data it accesses, and the actions it takes and why. Since many agents interact with other agents, security monitoring shouldn’t be limited to identities and endpoints; it should also cover the actions of autonomous agents themselves. The perimeter now extends to everything an AI agent is empowered to do.
Rajeev: Is AI security becoming a board-level responsibility, and what should business leaders ask their security teams before approving AI deployments?
Praveer: Yes, it is. Given the interconnected nature of AI systems, they may have access to production systems, financial workflows, or customer data. The moment that happens, responsibility can no longer be limited to the IT function, as AI deployments can introduce business continuity and reputational risks, bringing them under the board’s purview.
Therefore, before signing off on AI deployments, businesses should verify three things. First is the AI’s access: What can this tool reach beyond what it is supposed to reach? It is important to identify these gaps before deployment, as such cracks can lead to incidents that could otherwise have been avoided.
Second is the vendor’s testing and auditing posture. Businesses should check whether the vendor has tested its AI systems under “stuck-state” conditions. Most vendors do not engage in sufficient red-teaming or stress-testing efforts, which can lead to unpredictable behavior in real-life scenarios. These efforts, and their outcomes, are vital to determining the reliability of the tools.
Third is accountability. Businesses should determine who owns accountability for the AI system if something goes wrong. That person or team should continuously audit the AI’s performance to identify points of deviation. This helps them intervene when necessary to avoid incidents. If the answer to any of these three questions is a shrug, now is the time to make the move.
𝐒𝐭𝐚𝐲 𝐢𝐧𝐟𝐨𝐫𝐦𝐞𝐝 𝐰𝐢𝐭𝐡 𝐨𝐮𝐫 𝐥𝐚𝐭𝐞𝐬𝐭 𝐮𝐩𝐝𝐚𝐭𝐞𝐬 𝐛𝐲 𝐣𝐨𝐢𝐧𝐢𝐧𝐠 𝐭𝐡𝐞 WhatsApp Channel now! 👈📲
𝑭𝒐𝒍𝒍𝒐𝒘 𝑶𝒖𝒓 𝑺𝒐𝒄𝒊𝒂𝒍 𝑴𝒆𝒅𝒊𝒂 𝑷𝒂𝒈𝒆𝐬 👉 Facebook, LinkedIn, Twitter, Instagram