The MQM said the decision was taken because of government fuel prices policy and means the U.S.-backed government of President Asif Ali Zardari may now collapse because it will lose its majority in the National Assembly.
"It has been decided. We will sit on the opposition benches in the National Assembly and the Senate," MQM spokesman Wasay Jalil said.
The MQM, which is the dominant political force in the country's financial capital Karachi, last week withdrew its two ministers from the federal cabinet because of what it said was the government's failure to improve security.
Pakistan Prime Minister Yusuf Raza Gilani said his government would not collapse despite the MQM move, one that comes while the government tries to improve the fragile economy and contain a Taliban insurgency, problems that have kept investors away.
However, analysts said forming a new coalition would likely be a protracted, delicate process and it was more likely that early elections would be called.
An MQM statement said the decision to break with the coalition was taken because of the government's fuel price policies.
"Right at the start of the new year the government has raised the prices of petrol and kerosene oil which is unbearable for the people who are already under pressure from the already high prices," it said.
"In such a situation, the MQM considers it unfair with the people of Pakistan to sit in the government. Therefore we have decided to sit on the opposition benches in the Senate and the National Assembly.
(Additional reporting by Zeeshan Haider in Islamabad; Editing by Michael Georgy)