The company's AI agent broke into the Medical Statistics Reporting Service database while researching public medicine spending during a test run on June 18.
OpenAI discovered the hack in August during a review of its out-of-control models, but did not tell the government until September 10 when an email landed in the public mailbox of Services Australia.
It took five days for that to be referred to the Australian Signals Directorate (ASD), Australia's cybersecurity agency, which along with a task force launched on Thursday is now charged with investigating the breach.
The delay was due to the need to first verify the notification was not a hoax, according to Finance Minister Katy Gallagher.
"It was made very clear by both ASD and Services Australia that this should not have gone to an email address, it should have been escalated," she said.
No personal details of Medicare customers are believed to have been accessed, with the information amounting to aggregate health statistics and file names, according to OpenAI.
"Our models attempted to look up answers ... (in) the course of that, our models took actions we did not intend," an OpenAI spokesperson said in a statement on Thursday.
The agent interacted with but did not hack three other government websites, including the Australian Institute of Health and Welfare, the NSW Bureau of Crime Statistics and Research, and the Victorian Department of Health.
During a phone conversation with OpenAI boss Sam Altman while in New York on Wednesday local time, Prime Minister Anthony Albanese said the company's head had "clearly accepted" it had not done enough on the issue.
Whether it broke Australian laws will now be determined by the newly announced task force, which is also expected to inform an AI framework that's being drafted and set to become law within the next 12 months.
A push to decommission outdated government websites, including the one that was breached, and move their data to newer, more secure platforms would now be accelerated, Senator Gallagher said.
Defence Minister Richard Marles on Thursday stressed the agent had "climbed a fence" to break into the low-security database, saying more sensitive information was walled in by a "fortress".
''(The agent) did scale it, and the point is it was unintended, it wasn't asked to, that's our concern here," he told reporters in Sydney.
The government was still determined to work with OpenAI on remedying the hacking incident and into the future, Mr Marles said.
"It matters to have a relationship with OpenAI so that we can have that cooperation ... but I'm also acknowledging that this is a situation which is fundamentally unacceptable," he said.
Mr Marles said Australia was a world leader on AI regulation, was armed with a first-class cybersecurity agency, and that people's sensitive data was not in danger.
OpenAI's global policy head Ann O'Leary was in Canberra when the company notified the government about the breach and when it was escalated to the ASD on September 15.
On September 14, at a conference also attended by Labor frontbenchers, Ms O'Leary touted allyship with Australia on training AI, prudence around erecting guardrails, and granting access to its most advanced models.
She did not mention the breach publicly or make further efforts to notify the government.
Services Australia also was not able to ask specific technical questions of the breach to OpenAI until Tuesday.
The revelations coincide with AI executives, including Mr Altman and Anthropic chief executive Dario Amodei, pleading with governments to help hammer the brakes on the technology's development over safety concerns.
Mr Albanese was also one of more than 20 world-leader signatories in a joint statement calling for more restraints of AI at the United Nations in New York this week.
In response to news of the breach, Opposition Leader Angus Taylor accused the prime minister of being ill-suited to safeguarding against AI.
"We have a prime minister who who has never used AI by his own admission, so how can he possibly understand it?" he said.
