Abbott made the comment while announcing Australia would likely join humanitarian airdrops and even airlifts of endangered civilians threatened by Sunni extremist IS militants in a barren northern Iraqi mountain range, possibly later this week
The picture, taken in the northern Syrian city of Raqa, was posted on the Twitter account of Khaled Sharrouf, an Australian man who fled to Syria last year and is now an IS fighter, The Australian newspaper said.
It reportedly shows Sharrouf's seven-year-old son, dressed like any other young boy in blue checked trousers, a blue shirt and baseball cap, struggling to hold up the severed head of a slain Syrian soldier by his hair.
Underneath it were the words "That's my boy".
Another photo published by the newspaper shows Sharrouf dressed in camouflage fatigues posing with three young boys it said security agencies believe are his sons.
All are holding guns in front of the flag of the Islamic State.
Abbott, speaking to ABC radio from the Netherlands, said the pictures showed the barbaric nature of the terrorist group.
"What we've got to appreciate is that Islamic State -- as they're now calling themselves -- is not just a terrorist group, it's a terrorist army and they're seeking not just a terrorist enclave but effectively a terrorist state, a terrorist nation," he said.
"And this does pose extraordinary problems... not just for the people of the Middle East but for the wider world.
"And we see more and more evidence of just how barbaric this particular entity is."
Australia has an arrest warrant out for Sharrouf, who has also been pictured posing with severed heads. Officials have said up to 150 Australians are fighting alongside militants overseas, mostly in Iraq and Syria.
Abbott, meanwhile, said Australia was ready to participate in United States airdrops of food and water to civilians threatened in Iraq, and could also deploy two aircraft for any airlift mission.
"Australia will gladly join the humanitarian airlift to the people stranded on Mount Sinjar, this is a potential humanitarian catastrophe, President Obama has said it's a potential genocide," he said.