One recent incident involved an experimental OpenAI system that gained unauthorised access to an Australian government website while carrying out a research task.
The website was the Medicare Statistics Reporting Service, run by Services Australia. The AI was supposed to find public information about government spending on medicines for skin conditions in Victorian communities.
That sounds harmless enough.
But when the AI could not find the information through the normal public website, it kept looking for another way in.
OpenAI later said the system found a way to gain non-public access. It viewed technical information and source code, retrieved internal files, credentials and statistics, and even ran commands and wrote files.
Importantly, OpenAI says there is no evidence personal Medicare records, medical histories or individual patient information were accessed.
But while reassuring, it does not remove the bigger concern.
Most of us understand if we reach a locked door, password screen or security barrier, we should stop unless we have permission to continue.
An AI system does not necessarily understand that in the same way.
It may simply see the barrier as another problem standing between it and the goal it was given.
Imagine telling a researcher to find a document. They search the library, cannot find it, discover a locked office and decide the quickest way to finish the job is to get inside.
The problem is not AI became angry or evil, it was simply trying to complete the task.
Which is what makes this worrying.
AI agents are becoming more capable. They can search websites, use computer tools, interact with software and decide what to try next.
Safety systems in products such as ChatGPT are designed to stop obvious harmful requests. If someone asks an AI to hack Medicare, it should refuse.
But what happens when the original request is innocent?
That is the lesson from this incident.
A safe instruction can still lead an AI somewhere unsafe if the system decides breaking through a barrier is the easiest way to complete its goal.
Other AI safety tests have produced uncomfortable results.
Anthropic, the company behind Claude, has reported experiments where AI models in simulated workplace situations chose actions such as blackmail when they believed they were about to be shut down. These were controlled tests, not reports of an AI blackmailing someone in the real world, but the behaviour shows why researchers are concerned.
Experimental AI systems have also reached real internet systems during security testing and taken actions they were not authorised to take.
This is why stronger rules and oversight are becoming increasingly important.
AI companies are calling for clearer government frameworks for powerful AI systems. OpenAI has also acknowledged the Australian incident should not have happened and says it has introduced stronger restrictions, monitoring and human review.
AI can work much faster than a person and examine many sources in a short time.
That makes it extremely useful.
It can also make mistakes extremely quickly.
The important question is no longer simply, “Will AI refuse a bad instruction?”
We also need to ask, “Will AI recognise when it has gone too far?”
For now, that may be one of the most important questions facing the technology industry around the world.
There is no podcast version of Ask A Tech this week, as I am taking a short break from the podcast.
If you enjoyed this article or have a technology question, you can reach me at askatech@mmg.com.au.
26°C