They used to say that everyone had a good time when the circus came to town. Well that may be true, but I’m not so sure that this is happening with the circus that has surrounded the release of the 2015 PISA results this week. While the PISA process itself is quite straightforward, it’s the political and media storm that surrounds its release that creates the circus, and this release has been no exception.
Every 3 years, PISA, an arm of the OECD, oversees more than half a million statistically sampled 15 year old students across 72 countries, complete assessments in reading, mathematics and science together with a range of questions focused on the student’s background, demographic and access to resources.
Whether or not you think triennial international education benchmarking is a good thing or not, it looks like it’s here to stay, and the impact it has politically is now even more significant than ever. Many countries have made major education policy shifts in response to previous PISA releases, again for good or bad reasons. But one thing for certain: like the circus, it’s hard to ignore.
Like any tests, there are winners and losers, and while some results are predictable, there are often a few wildcards. Finland has probably been the most notable, pulled out of educational obscurity when it was ranked very ‘highly’ in earlier PISA reports, while the middle ranking performance of countries such as the US and the UK has been a source of much discussion, particularly when ranked against relative funding per student.
At a superficial level, it is seen by most media commentators as a scoreboard, and therefore there are winners and losers. And with that a corresponding response across various countries’ media. Australia’s results have ‘fallen’ in this round of PISA, while in last week’s TIMM’s results, Australia was ‘beaten’ by Kazakhstan and our media was apoplectic; even our national broadcaster was shaken. Of course all the Borat references didn’t help. Showing some genuine respect for the investment that some eastern block countries are now making in the quality of their education systems would have been a more appropriate response. My recent work in Estonia, Georgia and Azerbaijan certainly confirms that.
Interestingly the US, media seemed less concerned about it, which could have been a response to how its students performed, or that frankly doesn’t care much for what the rest of the world thinks, and maybe even more so with a change of President. Additionally there were a number of articles that simply attacked the validity of the test process itself, while in the UK there has been a lot of commentary around the political implications for schools across the British Isles.
As an aside it’s interesting to see that following China’s lead last round in nominating Shanghai as ‘representing China’ rather than sampling across the whole country, the US has ‘seen your Shanghai and raised them Boston’…and so Massachusetts was also sampled additionally to the US, for reasons that should be rather obvious.
But you get the picture. So amidst all the misleading hype and hysteria that accompanies the release of PISA and TIMMS, it has now becoming much harder to sort the wheat from the chaff, to use an Australian colloquial expression. But I do think it warrants deeper analysis.
Looking at the end result and how it is often perceived publicly, I do worry that the education commentariat is too often slim on fact, heavy on opinion. Consequently we then find everyone responding to the commentary that surrounds these sorts of reports, and in doing so we are distracted and miss the deeper ideas and insights that can, after all, be quite valuable.
As much as I try to maintain a somewhat skeptical eye on the assessments themselves, I am rather more intrigued by how the media and the broader community react to them, and most significantly the impact it has in the medium to longer-term on educational policy development for schools across the globe.
My first contact with PISA was however, rather amusing, if not nearly very embarrassing.
In the early 2000’s after speaking at a conference in Frankfurt shortly after the very first PISA report had been released, I was seated next to several EU Education Ministers for dinner, when one turned to me and asked, “And what do you think of PISA?”
The first answer that came to mind was “Well it hasn’t fallen down so that’s a good thing,” but as luck would have it, I paused before blurting that out, and fortunately responded instead to the question that followed asking if I was proud of how well Australia ranked.
As was obvious at that time I, like most of the non-European world had no knowledge of PISA. But in the intervening years it’s certainly been interesting to watch its increasing impact from both an educational and political perspective. During that time, I have attended PISA launches in Paris and Doha, and I have also spoken with the Director Andreas Schleicher on several occasions.
It’s funny to reflect on now, but do you remember the time of BP…Before Pisa…when we all used to mind our own business and we were happy in our own little education worlds believing we were doing a good job? The funny part was of course, that many were, but then who would have known?
Either way, international benchmarking is a reality now, and PISA’s release highlights yet again how important it is for educators to lead the public conversation that such a release inevitably provokes. Failing to take the lead at this time simply hands the public microphone to politicians and poorly informed journalists who have a deadline to meet, or a shrill headline to create about ‘failing educational standards’.
Contrary to popular opinion, I would have to say that I think much of the intent and thinking around PISA is well placed, despite ongoing and quite fervent opposition. As an OECD initiative it carries a European perspective with a very strong emphasis on equity, and above all, equality of educational opportunity. If you can ignore the crude rankings, but focus on the dissections of them, it does highlight how some countries have paid a lot more attention to a broad span of abilities and backgrounds, rather than focus on one group at the expense of others.
They provide a lens into schools across the globe, (well, in 72 countries) which if reviewed carefully can provide useful insights and policy initiatives that have proved effective in many countries. The country profiles, such as this one on Estonia, are particularly enlightening as they move beyond the test results and look deeper into the country’s governance, funding and key policy initiatives.
As an aside, I must admit to being somewhat amused when Pasi Sahlberg from Finland, now based at Harvard, told me that despite the countless US delegations to visit Finland to try and find out “why they did so well on PISA,” he saw little evidence of any learning for US policy makers from the visits. In stark contrast to US schools, Finnish students have little homework, no annual testing, short school days, a shorter school year, and tons of playtime, and they still perform very well on the international benchmarks. Yet few countries, including the US, seem to want to adopt any of their education policies, particularly in regard to teacher quality.
And while we’re on the subject of performing well, how many people actually know what the raw scores mean?
The Australian government is now panicking because we scored 510 in Science and 494 in Math compared to New Zealand’s 513 and 495 respectively; but how significant is all that? Who knows? Who cares? I realize that to lose a game of basketball by one point is not much better than losing by 10, but this is not sport..is it?
Interestingly though, I’ve always had a high regard for both the New Zealand and Canadian education systems; the former because of the openness of its curriculum which accords respect for their teachers as professionals, while Canada on the other hand, has the benefit of not having a national Ministry of Education. In Australia’s case, that has only served to bog schools down with a national curriculum and national league-table ranked testing. Not sure how any of that impacts their schools, but both countries consistently report good outcomes over successive PISA result.
We talk about the need to be more statistically literate, but we certainly have a long way to go. Oh, and while we’re on that topic, the often mentioned work of our colleague Conrad Wolfram gets a nod here, as the first country to take up his computer-based math curriculum around statistics and probability was Estonia, and on this year’s PISA report it is ‘rising with a Red Bullet’!
Out of all of this sadly the media and politicians still seem besotted by country rankings rather than the underlying trends and deeper insights the data highlights.
If you take the time to listen to much of what PISA’s Schleicher is emphasizing, it is not the rankings. He is a statistical researcher and as such he believes his role is to focus on trends and indicators that he can highlight and share as potential educational policy initiatives for countries to consider.
For the record, these were the four main policy recommendations that came from the report:
- Continue investing in good teachers and empower them to teach effectively. The most successful education systems attract and retain the best teachers, offer adequate compensation, encourage continued professional development, and provide ongoing feedback.
And granting schools more autonomy may give teachers more opportunities to adapt their instruction to students’ needs and knowledge.
- Delay the sorting process. At Hogwarts’ School of Witchcraft and Wizardry, Harry Potter and his classmates were sorted into Gryffindor, Ravenclaw, Hufflepuff and Slytherin on their very first day. An early sorting may be appropriate for students of magic, but it doesn’t always work in the real world!
All students, whether disadvantaged or advantaged, immigrant or non‑immigrant, would benefit from a more limited application of policies that sort students into different programme tracks or schools, particularly if such policies are applied in the earliest years of secondary school.
The later students are selected and grouped, and the less they repeat grades, the better they are supported by their teachers and the weaker the association between students’ socioeconomic status and science performance.
- Break down stereotypes.Parents, teachers, policymakers and opinion leaders should actively challenge gender stereotypes about science-related activities and occupations (e.g. ‘‘computer science is for boys’’, ‘‘biology is for girls’’) \
- Increase support to disadvantaged students and schools. Disadvantaged students need more time in regular lessons with better teaching. And in countries and economies where students in advantaged schools spend more time studying after school, including Croatia, Italy, Japan, Korea, Macao (China) and Chinese Taipei, governments may need to provide disadvantaged schools with additional resources for free-of-charge tutoring to prevent the development of a shadow education system ─ and to ensure equity in education opportunities.
Here’s my point: If we could just stop being so fixated and hysterical about country rankings, hidden away inside work like this are some quite worthwhile policy ideas that would benefit all kids. PISA is not TIMMS, and as such looks to explore a much broader profile of not only schools but students, their opinions and their personal circumstances.
On the other hand I could spend quite some time sharing my thoughts on some of PISA’s shortcomings, which include its extremely limited awareness of the impact technology is having within schools, and the challenge it faces in moving the measurement focus forward to those skills that are now universally identified as critical to college, career and life success.
These are of course familiar to you, but I would hope there is now broad agreement that we do need to better understand how we can enable kids to be more collaborative, creative problem solvers who can think computationally and not just be able to do well what they are taught to do, but realize that success in the rapidly changing world of the future depends on being able to do well what they were not taught to do.
Not wishing to add greatly to those concerns, it would be remiss of me to also not highlight one of the biggest challenges PISA faces regarding the unreasonable influence of excessive tutoring and ‘Cram Schools’ in Asian countries such as Korea and Japan in particular, and to an extent in China and Singapore.
Keeping this in mind, it is reasonable to suggest that some of the PISA results are simply not comparing Apples to Apples, especially if a child is receiving 20 or 30 hours additional private tutoring each week, as is reported to be the case in some of those countries. Once the domain of the elite, private tutoring has now become widespread across Asia, according to a report released in July 2012 by the Asian Development Bank and the Comparative Education Research Center at the University of Hong Kong.
That report, quoted in the New York Times suggests that up to 97 percent of all Singaporean students, nearly 90 percent of South Korean primary students and about 85 percent of Hong Kong senior secondary students receive tutoring. The article cites three South Korean celebrity tutors: Kim Ki-hoon earns $4 million a year in South Korea, Woo Hyeong-cheol, who reportedly earns $3.9 million per year offering Web-based math classes to 50,000 students; and Rose Lee, “the Queen of English,” said to earn $6.8 million per year, also through online classes. In 2012, South Korean parents spent more than $17 billion on tutoring services, which was $2billion more than US families spent on video games.
All that aside, a report such as PISA deserves serious attention from the media, but yet again we have examples of how poorly so much of what we do in education is covered by the media. While there are already many examples regarding this latest report, one example by Jennifer Buckingham published in one of Australia’s leading newspapers, the Financial Review, carried the following comment explaining why Australia’s PISA result ’were a catastrophe’.
“These findings suggest Australia’s fixation with discovery or inquiry-based learning approaches to improve achievement with maths and science by developing student “engagement”-often at the expense of learning facts and concepts-is misguided and detrimental.”
Now that’s not even stretching a long bow. That’s a slingshot.
Oh, if only that ‘fixation’ on inquiry-based learning was actually true. Somehow an article around PISA is reduced to a cheap ideological shot which bares little relationship to the reality in Australian schools or the results themselves. If Ms Buckingham read more widely she could have seen that PISA Director Schleicher believes otherwise:
“Perhaps most importantly, technology can support new pedagogies that focus on learners as active participants with tools for inquiry-based pedagogies and collaborative workspaces.”
So therein lies the problem. Too often it’s not so much the educational benchmarks themselves that provoke the accountability challenges for schools, but rather the superficial way their findings are reported; it just highlights the urgent need for educators to be more proactive in the public media space.
Personally, while I’ll never be a big fan of any ranked comparative testing, national or international and while PISA is certainly not perfect, and never will be, I think it does a far better job in trying to identify educational policy areas that will address inequity in our schools and support kids than 95% of the media does in reporting on its work.
