The 2032 Games authority on Monday officially took possession of the Victoria Park site in Brisbane after demonstrators were moved on by police in the early hours of the morning.
A handful of activists regrouped outside the park's fenced-off perimeter as construction begun.
The Games authority is racing the clock to complete the 63,000-seat, $3.8 billion stadium that will be the 2032 centrepiece, almost five years after Brisbane won the Games bid.
Premier David Crisafulli turned the first sod on Monday morning, saying Queenslanders had been urging the government to "get on with it" after years of delays.
"It's game on - we have a plan to deliver for the 2032 Games and beyond, and today, we get cracking on delivering it," he said.
"Victoria Park will be the beating sport, cultural and green heart of Brisbane, and a place all Queenslanders can be proud of."
Mr Crisafulli said more than two‑thirds of Victoria Park would remain free green public space once the project was complete.
However, protesters and Traditional Owners had warned the project would destroy a culturally and ecologically significant green space.
Tensions boiled over in the countdown to the official start of the project with five people arrested on Friday as police began to move on protesters in what critics described as a surprise raid.
It was less eventful when protesters took advantage of the last day they could access the site, with hundreds rallying at the sprawling park in the city's north on Sunday.
The last of the protesters were moved on early Monday morning as the Games Independent Infrastructure Coordination Authority took possession of the park.
Fencing was installed and preparatory works began, including service decommissioning, testing and earthworks.
Local resident Josie - who did not want to provide her last name due to privacy reasons - said she felt compelled to return to the park on Monday after attending Sunday's protest.
"I have hope," she told AAP as about 20 police officers blocked public access to the park's main entrance.
"I want to show up for the First Nations people," she said.
Save Victoria Park organiser Andrea Lunt said protesters were disappointed but still considering their next steps.
"This space with its heritage and its history is such a special asset for Brisbane," she said.
"We're not an anti-Olympics group but we don't support the Olympics in its current form, which is going to take away precious green space from future generations."
Ms Lunt said the park held deep significance for Traditional Owners, with dozens of ancient trees she described as "ecologically irreplaceable".
Indigenous elders have declared Victoria Park a place of importance for many First Nations peoples and are pressing for their concerns to be heard as national Aboriginal heritage laws are tested.
Federal Environment Minister Murray Watt said he had received 10 applications seeking to protect a significant Aboriginal area "under threat of injury or desecration" at the park.
Two have been declined while the others remain under consideration.
Senator Watt said his powers over the proposed stadium were limited, stressing that cultural heritage declarations are not designed to stop the project outright.
"People seem to think that these are stop work orders, and they're not," he told ABC Radio Brisbane.
Any declaration he makes would instead set out what must be done "to preserve or protect an area from being injured in some way or desecrated in some way", he said.
Mr Crisafulli said while people had a right to protest but they could not remain inside an active work zone.
Deputy Premier Jarrod Bleijie said major building contracts would be awarded within months.
