Measuring levels of financial capability and the effectiveness of financial education

A major three-year program of research into financial capability has just completed. Between 2010 and 2013, a Trust Fund supported by the Russian Federation provided substantial resources to the World Bank and the Organisation for Economic Co-operation and Development (OECD) to undertake two distinct but coordinated programs of work.  This concluded at the end of June 2013 and the results have just been published.

The World Bank focused its efforts on developing diagnostic and measurement tools and undertaking research on the results achieved by a wide range of programs designed to improve financial capability.  This work is ground-breaking in its content and involved:

• The development of new survey instruments to measure and monitor levels of financial capability of individuals. This involved working with research teams in 12 low- and middle-income countries. Questionnaire content was developed through 70 focus groups, followed by 228 cognitive interviews. And, using data from the seven surveys that were completed on time, methods were developed for creating scores and providing an overview of levels of financial capability. These included a combination of factor analysis, cluster analysis and regression analysis. A full report giving details of the development of the survey instruments and methods of analysis and also the survey findings has been published (PDF) (together with a summary of the report (PDF)). This is accompanied by a 138 handbook containing all the questionnaires (PDF) used as well as full details of how to replicate the survey, which are as relevant to high-income countries as they are to low- and middle-income ones.
• 17 model impact evaluations of a broad range of financial capability interventions, including ones using more innovative methods of developing financial capability and influencing behaviour as well as more traditional programs based on workshops and seminars. There is, generally, a dearth of rigorous impact evaluations of financial capability programs and these were financed to fill that gap. Taken together they demonstrate that the more innovative approaches are more likely to raise levels of financial capability and to have a more sustained effect. But effects of interventions do, generally, fade over time. Both the quality of the content of a program and its timing affect outcomes and the more relevant they are to an audience the greater the impact (see the evaluation summary report (PDF) for further details).
• An evaluation toolkit (PDF), which at 256 pages, covers all that anyone would need to know if they are considering whether to evaluate a  financial capability program and, if so, how best to do so. It is helpfully structured so that it meets the needs of both policy makers, while also catering for the technical needs of experienced researchers. Part 1 covers understanding the Monitoring & Evaluation (M&E) process and concepts and is intended to provide an overview of key concepts as well as how one decides whether or not to evaluate a program. Part 2 on conducting M&E for financial capability programs explains in some detail what is involved in monitoring, process evaluation and impact evaluation and how these approaches can be combined. While part 3, collecting and analysing M&E data for financial capability programs, is more technical and describes in some detail the range of research methods an evaluator might want to use. The final part covers a range of practical issues including implementing the evaluation, ethical considerations and documenting and communicating results. Its strength lies in the evidence it gathered from experiences of conducting real life evaluations, including the 17 model evaluations described above.

In addition there is an overview report (PDF) that not only brings together these three strands of work but also sets it in a very helpful conceptual context, which draws a distinction between traditional approaches to measuring financial literacy (which tend to be normative and cognitive)  and the more empirical positive/agnostic approach to measuring financial capability that was adopted for the World Bank program of work.

Professor Elaine Kempson was the lead consultant on the World Bank program, and she is first named author on the two publications relating to the surveys. She was also a contributor to the tool kit and overview report. PFRC’s Director, Sharon Collard, was a consultant on the survey work.

In parallel, the OECD, in conjunction with the International Network for Financial Education (INFE), focused its efforts on the review and stocktaking of policy development and national strategies to facilitate the sharing of experience and formulate principles, guidelines and best practices. It also developed a short survey instrument to measure and monitor the knowledge, attitudes, skills, and behaviour of individuals.

Professor Elaine Kempson was also consultant to the OECD developmental work on the survey, and author of a report informing the work of the OECD in this area.

---

Links

Final outputs of the Trust Fund [External link to Financial Literacy and Education website, which contains links to all of the outputs referred to above.]

---

(Back to top)