The federal government will introduce its changes to the taxes to parliament on Thursday, in a bid to make it easier for more first home buyers to enter the market.
Under the changes, the 50 per cent discount on capital gains tax will be replaced from July 2027 with a rate based on inflation.
The government is still consulting with industry groups about potential carve-outs from the capital gains tax regime for startups, which could face their top marginal tax rate doubling to near 47 per cent when they sell their business.
But Dr Chalmers said wider changes still needed to apply to ensure the reforms had their intended effect.
"We are applying this change broadly and not just to the housing market, for example, as some have suggested that we should,'' he told reporters in Canberra on Wednesday.
''It doesn't make a lot of sense to replace one big distortion with another big distortion.
"There's a legitimate conversation going on with the small business sector."
The treasurer did not say what concessions were on the cards for the tax changes.
"In order for the consultation on implementation details to be meaningful, we'll take as much time as is, as is necessary after that to do that," he said.
"Ideally we're talking about weeks and months rather than months and years to bed down some of these other pieces."
The capital gains changes, along with restricting negative gearing to new properties from July 2027, are expected to allow 75,000 people to enter the housing market over the coming decade.
But the capital gains measures should only apply to housing, and businesses should be spared entirely, Australian Chamber of Commerce and Industry chief executive Andrew McKellar said.
"This is all stick and no carrot and that's the problem," he told reporters in Canberra.
"It impacts Australia's competitiveness as a destination for investment. We are urging the government to rethink, to pause."
After the tax reforms are introduced to parliament, the subsequent carve outs are set to be legislated at a later date once the measures are in place.
Mr McKellar said that approach was not good enough.
"There are fundamental issues with that headline legislation. Those need to be addressed," he said.
Former prime minister Paul Keating said the government needed to stick with its overhaul, arguing the 50 per cent discount implemented by the Howard government in 1999 had negatively impacted the economy.
"The government has done the right thing on housing but it is imperative that the CGT change doesn't create a new and further distortion to the economy by exempting all other assets, particularly commercial ones," he told The Guardian.
"The shift in capital taxation under the new arrangements is so marginal that no entrepreneurial initiative is likely to be thwarted by it."
Housing Minister Clare O'Neil said the tax changes would have a small impact on property prices.
"This will have a modest affordability effect on house prices in Australia, but at the end of the day, the thing that is driving house prices is actually not our tax settings," she told ABC TV on Wednesday.
"It's a fundamental mismatch between how many homes we're building and how many homes we need."
