Multilevel Modeling with HLM: Taking a Second Look at PISA

Abstract

The purpose of this book is to provide a synthesis of thought and practice in research in literacy and science education intended to lead to evidence-based results and generalizations that will serve as a foundation for public policy and informed curriculum, teaching, and assessment practices in education. The Gold Standard of educational research funding in the United States can be viewed as a response to the general dissatisfaction with the utility of educational research; this federal mandate fosters a shift of educational research toward positivist empirical research approaches, such as random controlled trials (RCTs). There is an expectation of greater generalization and policy relevance as Gold Standard research is conducted and reported. It should be noted this dissatisfaction is not confined to the United States. An international response to this dissatisfaction with educational research-systematic reviews of educational research such as the Campbell Collaboration (Campbell Collaboration, n.d.), the What Works Clearinghouse of the US Office of Education (US Institute of Education Sciences, n.d.), and the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) in the United Kingdom (EPPI-Centre, n.d.)- has also identified the general dearth of rigorous empirical research that can support meaningful generalization of research findings. The deliberations about qualitative-quantitative approaches to educational research over the last 25 years have established parallels for quantitative data considerations (reliability, validity, significance, objectivity) that consider dependability, credibility, believability, and confirmability of information (Howe & Eisenhart, 1990; Husen, 1988; Lather, 1992; Phillips, 2005; Pring, 2000). The enactment of these considerations has produced a diverse collection of qualitative, quantitative, and mixed-methods research in literacy and science education. Inspection of these research studies during the 2nd Island Conference revealed concerns about the clarity of constructs involved in the research, measurements of these constructs, interpretative frameworks and scoring rubrics, data scales (nominal, ordinal, internal, ratio), and statistical analyses of these data. Procedural rigor, clarity, mining and secondary analysis of existing databases and information resources, generalization of research results, research ethics, and mixed-methods and other innovative approaches were identified as issues to enhance research quality (see Yore & Boscolo, Chap. 2).