AI used to sit and wait for instructions. Not anymore. Now it acts. It remembers what you’ve told it, and it fills in what you haven’t. Then it takes those guesses and uses them to book, pay, recommend, filter, or decide. Banking, healthcare, travel, shopping. There aren’t many corners left where this isn’t happening. The real issue, though, is not “whether AI can do these things” ; often it can. The real challenge that we haven’t sorted out is Accountability: who answers for it when it gets things wrong.




Trust Lags AdoptionTrust hasn’t kept pace with AI adoption. The University of Melbourne and KPMG found in 2025 that just 46 per cent of people were willing to trust AI systems. A year on, a report picked up by MES Computing laid the contradiction out plainly: 91 per cent were using AI, but only 33 per cent trusted AI agents with real work. Then came the PocketOS incident, which was hard to shrug off. An AI agent reportedly wiped a company’s production environment, backups included, in nine seconds. No breach, no attacker. Just an agent with too much access and not enough restraint. That’s what may result when autonomy is handed out faster than accountability.




Knowing you is not permissionAn AI system might know your favourite airline. It still shouldn’t book your ticket unless you asked it to. Sounds obvious, but in practice the line gets blurry. Systems pick up preferences from your behaviour and then treat them as grounds for action. A recommendation turns into a purchase. A reminder turns into a payment. A profile turns into a decision. There’s something else most people don’t see. The assistant may not be working only for you. It might be steering you towards a partner product, a higher-margin option, or whatever keeps you engaged. That doesn’t make it useless. But you should be told when a suggestion has someone else’s commercial interest behind it.




Someone must own OutcomeWhen an AI system causes damage, responsibility can’t just dissolve somewhere along a chain of vendors, platforms, and integrations. Builders own the design risks. Deployers own whether the system was right for the job, what it could get at, and who was watching. Users shouldn’t be left to stumble onto technical risks by themselves. Right now, too many organisations can’t say who owns what. OneTrust’s 2026 AI-ready governance report found only 5 per cent had clear coordination and accountability across the AI lifecycle. Just Five per cent. That’s not a small gap.




In the University of Melbourne and KPMG study, 76 per cent of respondents said they were willing to trust AI. Yet 78 per cent worried about negative outcomes. Both make sense, really. People want convenience. They also know it can come with a price. Companies, meanwhile, need to know which AI systems are running inside them. That includes the ones employees slipped in without asking anyone. Every agent needs an owner, defined permissions, and limits that hold up in practice.




Know who’s running it?NIST raised a related concern in September 2026. When agents share credentials, accountability gets hard to pin down. Put several systems under one login and work out who did what turns into guesswork. Separate identities and clearly defined authority aren’t extras. They’re the basics. Then there’s the data behind all the personalisation, and much of it is inferred. The system doesn’t only know what you told it, it draws conclusions from what you did. Only 48 per cent of people trusted companies using AI to protect their personal data, according to the Ipsos AI Monitor 2025. Cisco’s 2026 Data and Privacy Benchmark Study suggests one simple thing that really helps: being upfront about how data gets used. That’s a big part of what builds confidence. People should be able to see what’s been stored, challenge it, delete it, and decide what the system gets to remember. Not after a dispute but right from the start.




Approval is not oversightA click on “approve” doesn’t make a decision accountable. Picture a reviewer who has no context, no time, and no way of actually stopping anything. Their “approve” click means very little. It’s there so someone can say a human was involved. Real oversight is messier. You get to ask what the system did, and why. You get to say no, and find out what that does. And when the stakes go up, the checks have to get tougher, not stay the same.




MeitY’s AI Governance Guidelines , released in November 2025, push for meaningful human control, clear accountability and closer oversight in sensitive sectors. If an AI system makes a decision that matters, someone should be able to go back and work out what information it used, which model version was running, what it was permitted to do, and who signed off. Most organisations aren’t there yet. NASSCOM’s State of Responsible AI in India 2025 reportedly found that just 30 per cent of organisations had mature responsible-AI practices. Not a lot. And it’s not for lack of direction, to be fair. There’s the DPDP Act 2023 and Rules 2025, RBI’s FREE-AI framework, and MeitY’s guidelines, all guiding principles pointing in a unified direction.




Conclusion: Launch is not the endA system that passed testing last month might not be safe today. Models change. Memory changes. New tools get plugged in. Permissions creep outward without anyone making a fuss. Each of those changes can alter what the system can do, and what it might get wrong. So testing can’t stop at launch. It has to keep going. Security, privacy, manipulation, bias, drift, unauthorised actions : all of it. And when harm does happen, people need somewhere to turn. A way to challenge the outcome. A responsible human they can actually reach. And a fix, not just an apology. That process should exist before deployment, not after the first complaint lands. The more an AI system knows about someone, and the more it can do for them, the higher the bar should be. AI can make the decision. It can’t be held accountable for it. That part still belongs to people.



(Shubhradeep Nandi, Co-founder, ResponCible AI)

Contact to : xlf550402@gmail.com


Privacy Agreement

Copyright © boyuanhulian 2020 - 2023. All Right Reserved.