An estimated 1,031,000 babies were born in 2013, down about 6,000 from a year earlier, the ministry said.
On the other hand, around 1,275,000 people died up about 19,000 from the previous year, the highest annual rise since World War 2.
As a result, the natural population decline came to a record 244,000, the ministry said, beating the previous highest fall of 212,000 in 2012.
Japan's population totalled 126,393,679 as of March 31, down 0.21% from a year earlier, according to a government figure. It has continually declined since 2007 by natural attrition deaths minus births.
Japan is rapidly greying, with more than 20% of the population aged 65 or over one of the highest proportions of elderly people in the world. The country has very little immigration and any suggestion of opening its borders to young workers who could help plug the population gap provokes strong reactions among the public.
The proportion of people aged 65 or over will reach nearly 40% of the population in 2060, according to a 2012 government report.