The TransAsia Airways plane, a 70-seat ATR 72, crashed near the runway on the island of Penghu, west of the mainland, with 54 passengers and four crew on board, they said. No one was killed or hurt in the buildings.
Eleven injured people on the plane were taken to hospital, the government said.
The TransAsia Airways aircraft took off from Taiwan's southern city of Kaohsiung, headed for the island of Makong, but crash-landed in Huxi township of Penghu County, the main island of the chain also known as the Pescadores.
"It was thunderstorm conditions during the crash," said Hsi Wen-guang, a spokesman for the Penghu County Government Fire Bureau.
"From the crash site we sent 11 people to hospital with injuries. A few empty apartment buildings adjacent to the runway caught fire, but no one was inside at the time and the fire was extinguished."
About 100 firefighters were sent to the scene, as well as 152 military personnel and 255 police, he added.
According to an official at the Civil Aeronautics Administration, air traffic control reported that the inclement weather at the time of the crash did not exceed international regulations for landing.
Visibility was 1,600 meters and the cloud cover was as low as 600 meters, added the official, who declined to be identified.
Television networks aired footage of TransAsia's president, Chooi Yee-choong, bowing in apology.
"We express our deepest apologies to everyone for this unfortunate event."
Typhoon Matmo hit Taiwan on Wednesday, bringing heavy rain and strong winds, shutting financial markets and schools. It passed the island and headed into China, downgraded from typhoon to tropical storm.
TransAsia Airways is a Taiwan-based airline with a fleet of around 23 Airbus and ATR aircraft, operating chiefly short-haul flights on domestic routes as well as to mainland China, Japan, Thailand and Cambodia, among its Asian destinations.