20141201Measuring results in education through psychometrics ContentFirst, what is psychometrics?

Psychometrics is the science of measurement, and the field has been in existence since Charles Darwin’s groundbreaking work on individual differences in the 19th century.

It has evolved into a sophisticated area of study about the measurement of people’s knowledge, skills, and traits. Psychometricians are trained in areas such as evaluation, instrument development, sampling, data collection, statistical analysis and reporting.

Why is psychometrics important in educational development?

At Coffey, we apply psychometrics to our education development projects. Clients and governments who fund new textbooks or teacher training programs want to know whether pupils are improving in areas of reading, mathematics and science. So we develop tools (tests and surveys), set a baseline (starting point), and measure progress (evaluation) over the life of a project. This information is used to find out whether education interventions are having a desired effect on pupils.

In a best-case scenario, we also build capacity of local governments and non-governmental organisations to use psychometrics in conducting their own education assessments of student learning and teacher knowledge. The goal is to carry out this work with a high degree of validity (accuracy) and reliability (consistency). Tests must be high quality and carried out using standardised procedures. The results must be analysed accurately and in a timely manner.

Using psychometrics in the largest reading study ever in Pakistan

Starting in early 2014, we were awarded a contract by the US Agency for International Development to conduct a baseline study of children’s reading levels in Pakistan. It took a year to complete, and it turned out to be the largest such study ever conducted in a developing country. We reached a sample of 31,500 pupils in 1120 schools in all eight of Pakistan’s provinces and territories under extremely challenging conditions

Conducting the study initially involved adapting an instrument called the Early Grade Reading Assessment (EGRA), which has been used in various forms with children in more than 50 countries. It is a test that is individually administered to young children using a series of reading tasks – recognising letters, reading words, answering comprehension questions and more.

In Pakistan, we used psychometrics throughout the process by:

  • adapting the EGRA tool in three languages (English, Sindhi, and Urdu)
  • field testing the tools
  • training enumerators to standardise the data collection
  • sampling the schools
  • administering the tools to the target sample of pupils
  • analysing the data
  • generalising the baseline results to the target population.

The data are being used for two purposes in Pakistan. First, to plan modifications for current USAID-funded reading projects based off of the findings and second, to establish a starting point for evaluating children’s progress over the life of the projects. The EGRA tools will be administered at two other points (midline and endline) during the five-year implementation of the projects.

How Coffey can help

Having psychometrics expertise at Coffey is a valuable resource for our clients. Our team is able to collaborate with local governments, undertake assessments, produce reports, make presentations and build institutional capacity. We also conduct policy analysis to use the results for improving education systems in developing countries – which ultimately benefits children around the world.

If you would like to know more, please contact Jeff Davis at jdavis@msi-inc.com or +1 703 979 7100 extension 635.

To view a PDF version of this insight, please click here