Al Carns quit his position as armed forces minister in June, following John Healey's exit as defence secretary in a dispute with Downing Street about the Defence Investment Plan (Dip).
The shock resignations proved a consequential moment in Sir Keir Starmer's downfall as premier.
Healey has since been appointed Chancellor by Burnham.
Carns said the issue of defence funding has "not yet been settled", and that he had declined the offer of a role.
Earlier on Wednesday, Natalie Fleet resigned as safeguarding minister saying she found the role "overwhelmingly triggering".
The Labour MP for Bolsover, who was groomed and raped as a teenager, said she had done the "bravest thing" in telling Burnham she was stepping down.
Fleet has previously spoken about her experience of being groomed at 15 by an older man, who raped her, which resulted in her becoming pregnant.
Elsewhere, Mike Tapp said he will no longer serve as the Home Office minister for migration.
He said he was "disappointed to be leaving government", and had remained "loyal to Keir Starmer until the end".
The junior minister found himself embroiled in a row with Home Secretary Shabana Mahmood towards the end of Starmer's tenure, after he suggested exempting care workers from her indefinite leave to remain reforms.
Environment minister Mary Creagh also announced she would be stepping down.
Early education minister Olivia Bailey also announced she was leaving the government.
Scottish Labour leader Anas Sarwar is expected to be given a peerage and a job in the revamped Department for Business, Innovation, Science and Trade.
Sarwar, who has been Scottish Labour leader since 2021, was among the first to call for Starmer to stand down in February.
Downing Street has already confirmed appointments including Dan Jarvis, who stepped in as Starmer's defence secretary when Healey resigned and has been returned to his previous post of security minister.
James Murray, who was made health secretary when Wes Streeting quit, has been made Financial Secretary to the Treasury and Paymaster General.
Anna Turley is no longer Labour Party chairwoman and has instead been made a Home Office minister.
Florence Eshalomi has become a housing minister and Andy Slaughter Solicitor General, freeing up their roles as heads of parliament's housing and justice committees.
There were also 17 reappointments for ministers to remain in their roles, including Sarah Sackman as justice minister and Luke Pollard as defence minister.
