Liberal Democrat Leader Nick Clegg has replaced the Scottish Secretary in the first move of a busy day of reshuffles by all three main party leaders.
The Deputy Prime Minister replaced Michael Moore with Alistair Carmichael, saying “different experience” was needed ahead of next year’s independence referendum.
David Cameron is also overhauling his ministerial team, although any changes are expected to be further down the Government ladder.
And Labour leader Ed Miliband is mixing up his shadow cabinet, with Andy Burnham and Liam Byrne said to be frontbenchers in danger of demotion.
Two Tory Government ministers – Deputy Chief Whip John Randall and Cabinet Office Minister Chloe Smith – have already resigned ahead of the changes.
The departures increase the scope for Mr Cameron to bring in new faces to his top team, which is tipped to feature more women.
They follow Simon Burns’ resignation as Transport Minister, who has stood down to launch a bid to become a Commons Deputy Speaker.
Mr Randall, Tory MP for Uxbridge and South Ruislip, said he had indicated at the end of last year he wished to step aside.
The 58-year-old was embroiled in the “Plebgate” row last year, threatening to resign if Andrew Mitchell refused to quit.
In his resignation letter, he said it had been “a great privilege and honour” to serve for 13 years in the Whips’ Office in opposition and Government.
“I have nothing but the deepest admiration for you as a person, leader and Prime Minister. I will never forget the kind note that you wrote to me when my mother died last year,” he said.
“You can be assured that I will do whatever I can for you personally as well as for the party, the Government and of course the country.”
The Prime Minister said he “could not have wished for a more loyal, discreet, patient, trustworthy and committed colleague” and that he “had rather hoped this day would never come”.
“You have been a rock, not just in the Whips Office since 2000 where you have served with great distinction as Assistant and then Deputy Chief Whip, but for the whole Parliamentary Party,” he added.
“Your wit and humour are well known across the Party, but so too is your compassion; your dedication to the Party and to Parliament; and your steadfast reliability in good times and bad.”
Ms Smith, MP for Norwich North, was only elected to the Commons in 2009 at the age of just 27 but was quickly promoted to the ministerial ranks.
As economic secretary, she endured a toe-curling interview with Jeremy Paxman on BBC Newsnight when she struggled to explain a fuel duty freeze.
In her letter to the Prime Minister, she said: “I have been privileged to serve my country and my party under your leadership.
“However, for the remainder of the Parliament I want to be able to spend more time serving my constituents.”
Sky Political Correspondent Anushka Asthana said: “David Cameron has talked before about wanting to have a third of his cabinet female and so far he only has four Cabinet secretaries are women.
“What he will want to do today is move women up the ranks ready to go into Cabinet. Today they are focusing on middle-minister rankings. I expect we will see people like Liz Truss, perhaps Esther McVey and others start to move up.
“We are also likely to see northern and working-class MPs moving up. People are calling this a flat-cap reshuffle.”
[adrotate group=”14″]