2/20/2020

PISA Scores Mean Something, But What?

Released in December 2019, the scores from the PISA exam administered in 2018 do not appear to bode well for students in the United States. Before we delve into the latest findings, let’s define the PISA itself.

What is the PISA?

The Programme for International Student Assessment is a worldwide examination of students’ skills in core academic subjects that is overseen by the Organisation for Economic Co-operation and Development (OECD)—an international, multi-governmental agency focused on promoting policies and programs intended to improve people’s lives around the world. According to an informational video on the OECD website, the triennial exam has been in existence since 2000, with each iteration developed by education experts from the various countries that administer it.

Intended to measure whether students can “apply what they’ve learned in school to real-life situations,” the PISA assesses randomly selected 15-year-olds on their skills in math, reading, science, and related topics (e.g., financial literacy) and is given in the language used at each school administering it.

Notably, despite being randomly selected from pre-approved schools in participating countries, students who take the PISA must be demographically representative of their country or region. In other words, areas cannot simply put forward their most advanced or privileged students to take the test.

What were the 2018 PISA results?

The PISA highlights one subject (math, reading, or science) at a time, with literacy skills in the spotlight for the most recent iteration. As New York Times education writer Dana Goldstein noted in a recap of the 2018 test, “about half of the questions were devoted to reading,” and U.S. students' results elicited widespread dismay. Per Goldstein:

“The performance of American teenagers in reading and math has been stagnant since 2000 … despite a decades-long effort to raise standards and help students compete with peers across the globe.”

Goldstein went on to note that the “achievement gap in reading between high and low performers is widening”—in other words, the country's highest-performing students did better than their cohorts in previous years, while the lowest performers fell farther behind. Although U.S. students scored “slightly above students from peer nations in reading,” close to 20% of American 15-year-olds who took the 2018 PISA appeared to have “not mastered reading skills expected of a 10-year-old.” 

How to interpret the 2018 PISA results

Because the PISA assesses students’ future-readiness, U.S. students' poor test performance may indicate diminished prospects. This outlook is particularly troubling in light of what Goldstein characterized as “decades of bipartisan education reform efforts, costing many billions of dollars .... but [yielding] uneven results.”

As reported by Lauren Camera of the U.S. News & World Report, U.S. Secretary of Education Betsy DeVos claimed the 2018 PISA results point to a consistent lack of improvement within the nation's school system as a whole. But while DeVos focused on American students appearing to be “behind” those from countries such as Estonia and Finland, many observers honed in on the socioeconomic gaps reflected within the U.S. data. According to Camera, fewer than 5% of the top PISA scorers came from economically disadvantaged families, while close to 30% were considered “wealthier”—a disparity that points to a larger and more complex problem than a simple difference in reading skills. 

Indeed, the issue merited a 2019 special investigation by the National Public Radio show “On Point” that delved into the causes of the achievement gap as well as the “50-year fight” to fix it. In lieu of offering easy answers, the show's guests attempted to reframe the gap as a product of persistent structural and racial inequality in the U.S.

In a piece about the latest PISA results published by the education news source Chalkbeat, policy and research reporter Matt Barnum noted that “poorer students did worse on average than their more affluent peers” in many countries, not just in the U.S. While this insight is hardly a cause for celebration, it seems to reinforce the idea that test scores are closely connected to socioeconomic status, just as research has shown. 

Barnum went on to further contextualize the results by proposing another potential differentiator: The PISA, which is widely regarded as a “low-stakes assessment” in the U.S., is given at a time when high-schoolers may be more focused on higher-stakes tests, such as the ACT.

While Barnum was careful to advise against brushing off PISA results altogether, he acknowledged that their true meaning may not be as straightforward as previously believed. 

“It’s extremely difficult to say what policies explain better or worse scores between different countries,” he wrote. “And even if we knew which policies worked in one country, it’s not clear they could be exported elsewhere.”

For example, the U.S. does not have a central approach to teaching or the curriculum—as indicated by the challenges faced by policymakers trying to broadly implement the Common Core State Standards. 

All hands on deck

So, in the wake of the 2018 PISA results, where can we go from here? 

According to the PISA website, countries such as Germany and Brazil used their students’ previously disappointing scores as a jumping-off point from which to enact reforms around teaching and learning. For these countries, strategies such as investing in students, acknowledging income-driven gaps, and setting clearly articulated goals seem to be paying off in terms of yielding more encouraging scores on subsequent PISA tests. For the U.S., the path to improvement may lie in adopting a similarly comprehensive approach.