Each test is a single sheet of paper. The reading sheetheres an example in Hindi and heres one in English, and there are 15 other languages as wellhas four sections. The volunteers ask children to read letters, words, a short paragraph and a longer story. The maths sheet has single-digit and double-digit number recognition, two-digit subtraction with borrowing, and division on the level of, for example, 673 divided by 8.
The volunteers record the highest level in reading and maths the child can manage comfortably. Then they to go another house: 20 chosen at random from various parts of the village.
During October and November, volunteers will test between 600,000 and 700,000 children, including some in every rural district in India.
This years Nobel Peace Prize, given to Kailash Satyarthi, an Indian campaigner against child labour, and Malala Yousafzai, who needs no introduction, is the most recent global endorsement of the idea that children belong in school, not at work.
School attendance is rising nearly everywhere. In India, for example, 96% of school-age children are enrolledin part due to a 2009 law making education free and compulsory for children ages 6 to 14. India is winning the battle to get children into school.
But it is losing the war: Only some of these children are getting an education.
We know this mainly because of the tests done by the volunteers. Their report is called ASER, the Annual Status of Education Report (aser also means impact in Hindi), which is now in its 10th year. The project is organised not by the government, but by a particularly successful and rigorous non-governmental group called Pratham.
ASER is more than a survey. By making childrens learning visible to parents, teachers and policymakers, it has become a mobilising force for better-quality education. It has helped to turn the governments focus beyond enrolment, towards learning. And Pratham is not just diagnosing the problem. It is also introducing simple methods that teach children basic skills. Because of ASER, communities and now states are adopting these methods in learning camps and during the school day.
Pakistan, Kenya, Uganda, Tanzania, Mali and Senegal all now use a form of ASER, and Nigeria and Mexico are starting. The survey is attractive because it is organised and carried out by citizens, and because it can be done on an enormous scale at a tiny cost. Last year Indias nationwide ASER cost $1.3 million. (Not a typo.) The survey, moreover, can be done and tallied quicklyPratham publishes the results each year about 100 days after the survey begins.
Those results are heartbreaking. Last year, only 40% of third graders could read a first-grade-level paragraph and more than one-third couldnt even read words. Of fifth graders, less than half could read a second-grade storyand 5% couldnt even recognise letters.
In maths, 30% of third graders tested recognised two-digit numbers. Only one-quarter of fifth graders could do the three digit by one digit division problemsa skill normally taught in third or fourth grade. The ASER also shows that learning levels are droppingresults for 2013 are the worst yet.
The raw results show that private-school students do much better than government-school students. But much of the difference can be explained by their family backgrounds and incomes (which in India, as elsewhere, correlate with school achievement). If you control for that, private-school students do only a tiny bit better.
Why do students learn so little There are the usual problems: There are teachers who dont show up and students who dont show upin some states, only half of enrolled children are in school on any given day. (Poor attendance is both cause and effect of poor achievementwhy should a student who isnt learning go to school when she can be helping her family on the farm) Classes are enormous, there are few or no books and children learn through rote repetition.
Indias government is trying to solve these problems. It has substantially increased spending to add books, reduce class size, increase teacher salaries and improve infrastructure. Yet these have not led to more learning. Rukmini Banerji, who leads the ASER, says Prathams evidence shows that the most important reason is something else: By law in India, the teacher must cover the entire years formal curriculum. When the fourth-grade teacher uses the fourth-grade textbook, youre eliminating 80% of the class, Banerji said. Someone sitting in a fourth-grade classroom who cant read a simple sentence will be lost on the first dayand never catch up. The learning curve is flat.
The ASER survey is not Indias only nationwide test of school achievement. The official test is the National Achievement Survey, a written exam given in school in third and eighth grades. That survey shows learning is improving, and students are doing somewhat better than the ASER test would suggest. Government officials have dismissed the ASER as amateurish and cursory.
Only in India could a survey covering 500,000 children be dismissed as cursory, said Lant Pritchett, a professor of international development at Harvards Kennedy School who observed Pratham firsthand when he worked for the World Bank in Indiaand has helped to raise funds for the organisation. Its not supposed to be a comprehensive assessment about what children in India know, he said. Its meant to be simple enough so an illiterate mother or father can understand what a child can and cannot dosimplicity is one of its virtues. I think the government deliberately conflatesas if cursory means its inaccurate at what it measures. I think its super accurate at what it measures.
The ASER is the largest non-governmental measure of learning, but there are at least five othersand they all come out closer to the ASER than to the governments school-based test. One reason for the discrepancy might be that children take the ASER at home, so it catches those who are absent. Pritchett believes another reason is that the government test puts the questions in exactly the same form in which students study themeasier to answer, but not the best test of learning.
Karthik Muralidharan, an associate professor of economics at the University of California at San Diego, who has studied primary education in India extensively, thinks both the government test and ASER could be accurate. The government test does better at capturing changes in learning for high-achieving studentsthose scores could be improving while scores for low-achieving students are falling at the same time.
Testing children at home not only catches a more representative sample, it creates ASERs impact. When children dont go to school, its visible. When they go to school but dont learn, its invisible. That children are in school but not learning is a new realisation for parents as well as policymakers, Banerji said. Parents dont know about thiseven those who can read themselves. They assume that in school means OK.
The ASER makes a lack of learning visible. School is a foreign country to many parents, especially illiterate ones. This might be the first time parents really observe what their kids can and cant do, said Ruth Levine, director of global development and population programmes at the Hewlett Foundation (a major funder of Pratham), and a former chief of evaluation policy for USAID. Its the quintessential teachable moment. Parents are observing their children unable to correctly read a simple sentence. Volunteers are trained to start a conversation about what can be done. What can be done Changing Indias educational system is a challengefew bureaucracies are more rigid.
Prathams solution is a programme called Read India. The idea is simple: Teachers group children by ability instead of grade level. (These are fluid groups; kids who master letters then move to the words group.) And children abandon the official textbook for a few hours to concentrate on the basicstaught not in the usual rote-memorisation mode, but through games. Read India classes are usually held outdoors, where parents can watch.
Volunteers run week-long Read India learning camps in thousands of villages each year. They test every child in the village, then share the results at a village meeting. At camp, third-, fourth- and fifth-grade children who are far behind in reading or maths spend three or four hours a day using activities, games and colourful materials to work on the basics. Children almost always move up at least one level during the course of the week. Camp comes back to the village two months later.
Read India also works in the classroom. In parts of Haryana, Bihar and Uttarakhand, teachers set aside the last hour or 90 minutes of the day to use Prathams methods. The same teachers who were getting zero results with their normal methods saw big gains when they grouped children by level and worked on basic skills. (In Bihar and Uttarakhand, gains came when schools added community volunteers to work after school with the weakest children.)
Education in India is a paradox. Indias scientists and engineers are dominant in global technology, medicine and other fields. Yet 40% of its third graders cant read words. The Indian education system has always been good at the top of the distributionwhich is where the elites are drawn from, Muralidharan said. The design of education systems in developing countries has historically focused on screening for high-performing students as opposed to adding value to all students.
That is changing in IndiaPratham is a big reason whybut slowly. Now in India you dont need to explain to everyone that kids need to go to school, Banerji said. But that children need to learn and understandthat has another 10 years to go.