EXAMS ANALYSED-why we have them, what they measure and how to succeed in them

Part 1

In 1963 my life was radically changed because I passed an examination, called the 11 Plus, which meant that I would go to a school, very different from my local one, five miles away. My parents knew nothing about these ‘posh schools’, as we called them, and we had no idea how I had achieved this apparent success.

Five years later, a similar mystery occurred as I somehow passed, with high grades, national exams, called ‘O’ levels, we still had no understanding of how this had happened. Throughout the next six years, I continued to take and ‘pass’ many more exams at 18 (‘A’ Level), 21 (Honours Degree) and 22 (Post Graduate). When I began teaching in the seventies I was keen to discover what exams assess and understand what we need to learn to achieve exam success.

In 1978, I began teaching Chemistry to 17/18 year old students at A level (Advanced) that was to prepare them for University Entrance. I had taught most of these students for about five years, initially Science, and then Chemistry. These students were extremely ‘bright’ (good cognitive skills) who had achieved outstanding ‘O’ Level grades, and I was surprised that all of them were struggling in the practice ‘A’ level questions, so with their help, I intensified my research (we were scientists after all) into the factors for exam success.
This research involved studying:-
• When and why exams were introduced (history of exams)
• How exams in different subjects at various levels were written and structured (style of exams)
• How exams in different subjects and levels were marked and assessed (success criteria)

When and why exams were introduced (history of exams)

1967 History of Education in Great Britain S.J. Curtis
1966 Short History of Educational Ideas S.J. Curtis, M.E.A. Boultwood
2002 Middle Classes: Their Rise and Sprawl Simon Gunn, Rachel Bell
Education in England: a brief history http://www.educationengland.org.uk/history Derek Gillard

My research into ‘when and why exams were introduced (history of exams)’ began in the early eighties when I had access to books in the college (university) library for my post graduate study (part-time) into Personal and Social Education. Discovering the history of education and examinations (in England mainly) and was invaluable in helping to understand how exams had become the dominant measurement in education throughout the 21st century.

I had been teaching for over 10 years and had (finally!) become enthused with reading, studying and researching from books but I was surprised at how few colleagues seem to be aware of the history of exams or why it had become so central to education in England and the developed world.

When I began providing sessions to staff and parents from the late nineties I tried to include a section on  ‘What do exams actually measure and why have they become so important?’  Consequently I’ve now had the opportunity to discover that remarkably few people seem to be able to answer these questions, despite it having a huge influence on most. The blog on ‘Good Schools’ will also attempt to explain how exams have become extremely central to schools in England and the influence of the traditional Independent schools (Eton, Harrow, Rugby, Winchester, Westminster etc.).

In 1858, The first regular examinations under examination boards took place for schools to assess boys (girls began in 1867) approaching entrance to Oxford and Cambridge universities. The Junior exam for pupils under 16 years old and the Senior exam for under 18 years old, establishing the academic subjects of Arithmetic, Chemistry, Drawing, English Language, English Literature, French, Geography, Geology, German, Greek, History, Latin, Law, Mathematics, Music, Physical Sciences, Political Economy, Religious Knowledge and Zoology.

They immediately established very strict procedures, them being monitored by presiding examiners travelling from Cambridge carrying the exam papers in a locked box, with exam timetables of about seven consecutive days in blocks throughout the day and evenings. They mainly stressed recalling information but gradually introduced more analysis and understanding as well as simple knowledge. The number of exam boards increased and these boards also became a system of school inspection.

In 1906 the French psychologist, Alfred Binet, invented the first practical intelligence test to identify by the French government which students were mostly likely to experience difficulty in schools. Although Binet himself did not believe that his test measured a permanent or inborn degree of intelligence, these tests rapidly became very popular internationally (especially the USA), helping to support the belief that intelligence could be assessed by written tests and exams.

In 1933, in England, the educational psychologist Cyril Burt, stated in his book How the mind works, his definition of ‘human intelligence’ as

“inborn, all-round intellectual ability. It is inherited, or at least innate, not due to teaching or training; it is intellectual, not emotional or moral, and remains uninfluenced by industry or zeal; it is general, not specific, ie it is not limited to any particular kind of work, but enters into all we do or say or think. Of all our mental qualities, it is the most far-reaching. Fortunately, it can be measured with accuracy and ease.”             (Burt 1933:28-9, quoted in Chitty 2004:26).

Although after Burt died (1971), his studies of inheritance and intelligence came into disrepute after evidence emerged indicating he had falsified research data, it had a huge influence on the establishment of the 11-plus or Eleven plus examination, introduced in 1944, used throughout England and Wales to test a student’s ability (the intention was that it should be a general test for intelligence), to determine which type of school the student should attend after primary education: a grammar school, a secondary modern school, or a technical school.

The term ‘passing or failing the 11-plus’ helped to establish the perceived superiority of the academic studies at Grammar schools and in 1951, the General Certificate of Education (GCE) began, divided into Ordinary Level for 16 year olds and Advanced Level for 18 year olds.

The GCE at O and A levels, meant the universities could retained their influence and the predominance of the academic elite, despite the introduction of comprehensive education and General Certificate of Secondary Education (GCSE) replacing O level in 1986.

The 1992 Education (Schools) Act introduced Ofsted, to inspect schools and with the introduction of School Performance Tables for Secondary (1994) and Primary (1996) school exam results became central to the assessment of schools and a focus for the media.

Since the start of the 21st century the Organisation for Economic Co-operation and Development (OECD) has claimed to be comparing the education standards across the developed world using the results of 15 year old students in particular exams often called the “Pisa tests”. So why have exams gained a central place in education and societies throughout the developed world in over 150 years and what do they actually measure?
The answer to the first part can be obtained from “Education in England: a brief history – Derek Gillard –

Chapter 3 : 1860-1900 Class divisions” with this title and the subtitle that follows ‘Three classes – three commissions’ providing the key clue, in that England in the 19th century was hugely determined by the class structure and with the British Empire so influential, it’s values had a huge effect throughout the world.
This chapter begins –“Unlike the United States, which by the 1830s was establishing a public school system based on a common education for all its citizens, England, as we have seen, had allowed a divided school system to develop in line with its class structure.”

Chapter 6 of the book Middle Classes: Their Rise and Sprawl – Simon Gunn and Rachel Bell is entitled ‘Lesson In Class’ and provides a very clear understanding of how exams became central to education in England and much of the world.

“Education is absolutely central to the English middle classes: they are the people who pass exams.” (Page 147).
There was a rapid expansion of the number of ‘public schools’ of England on the seven established ones- Eton, Westminster, Winchester, Rugby, Harrow, Charterhouse and Shrewsbury and several education reforms, clearly reflect on Page 154 – “The result of the educational reforms of the 1860s and early 1870s was the creation of a system of education more rigidly defined by class and status than ever before.”

The desire to emulate the aristocracy in Victorian England has had extremely long term consequences.

“The public schools model was so strong that when state-funded secondary schools were finally established in 1902 they too furnished themselves with a simulacrum of its traditions.” (Page 157). “The 1902 Education Act firmly enshrined social division in education through the exam process.”(Page 160)

The importance of examinations cannot be understated in the 19th century, since it created and established academia, literacy and numeracy as being central to education and virtually all schools, which has remained unchanged into the 21st century.
“Making examinations the test of ability reflected the nature of the majority of middle-class occupations, based as they were on literacy and numeracy.”
(Page 161).

The UK government in 1992,(Education Schools Act) decided to introduce a national scheme of inspections of schools, which became known as the Office for Standards in Education (Ofsted) to supervise the inspection (HMI) of each state-funded school in the country, and would publish its reports. Following this came the School Performance Tables for secondary (1994) and primary (1996) schools based on results in the national examinations. Consequently examination results were thoroughly established as the key criteria for the grading of schools, with the huge rise in media coverage.

The dominance of examination results in schools was further established internationally with the introduction of the PISA (Programme for International Student Assessment) tests by the Organisation for Economic Co-operation and Development at the start of the 21st century.

(How Pisa became the world’s most important exam – BBC 27 November 2013)

‘the state of today’s education standards across the developed world will be revealed with the publication of the results of the OECD’s PISA tests’ reflects this belief.

Clearly the establishment of exams in Victorian England over 150 years ago has had a massive effect on education but basically it only assesses 3 of the 8 skills the extensive evidence shows are needed to be have good health, well-being, success, and life chances in the 21st century.

SCHOOLS_UNCHANGED

Part 2 focuses on

  • How exams in different subjects at various levels were written, structured (style of exams), marked and assessed (success criteria)
  • How S.U.P.E.R.learning can provide exam success and help develop the 8 skills

Slide14

25 years ago I discovered while studying for my M.Ed. that my background had contributed to my fascination and research into exams. I had been carrying out teacher-led research for 16 years, and had already discovered a great deal about exam success. It transpired that there were several key factors that were necessary to achieve this.
• Interested/skilled in Research & Statistical Analysis
• Keen to develop & measure effective learning & teaching
• Access to a Large cohort of 11-18 year old students
• Taught P.E. and Chemistry & Science at O & A Level
• Knew the school & environment as student and teacher
• Able to empathise from the student perspective

I explained in Part 1 that my fascination and research into exams was initiated by my mysterious success in them. Consequently when I began teaching in 1974 I immediately began experimenting and researching with my students to discover what exams actually measure and how they could be helped to succeed in them. Having numerous students over many years able to assist in this research proved invaluable, as was having access to exam papers and marking schemes in many different subjects and levels. I was fortunate in having many colleagues who were willing to provide me with exam papers and mark schemes. I also worked for several exam boards as an examiner to research how exam papers were marked, discovering how rigid and strict the marking scheme was adhered to, claiming that it is essential if there was to be consistency.

The 6 Exam Skills

By the start of the eighties, the findings of these experiments and research had uncovered 6 key skills (or activities) that were central to all written exams.

_6_EXAM_ACTIVITIES

This was an experience that even in the seventies was quite rare for most students, in the decades since then it is has become extremely rare. The numerous discussions and surveys that I was able to carry out in school revealed that few students spent any time doing it home (or in school) and clearly most found it difficult and struggled with it. Most students said that ‘silence’ was very rare in their lives and found it extremely uncomfortable.
To highlight the need to practise being ‘UNDER EXAM CONDITIONS’, I created posters similar to these and shared this message to students, staff and parents.

EXAM_CONDITIONS

Assessment for Learning

Discovering these 6 exam skills (or activities) immediately had a huge effect on the older (O and A Level) students as they were simply (and accurately) able to identify why they were ‘losing marks’ and which exam skills to practice. Crucially this also meant they were able to ‘take control of their exam preparation’, and we stopped calling it ‘revision’ as that term now seemed inappropriate.
Furthermore, these 6 exam skills helped to me continue to focus on developing the student’s skills, using the exam papers and mark schemes as opportunities and resources to do so. In fact, the feedback from the students using these 6 exam skills helped to gradually develop five key questions that we realised could be used to analyse ALL WRITTEN EXAMS.

_5_REASONS_for_LOST_EXAM_MARKS

With so many students using this analysis I had a great deal of feedback and evidence on what exams were actually assessing and how I could more effectively help students (anyone actually) identify the difficulties that really needed to be overcome to achieve exam success.
In 1998, the booklet, “Inside the black box”, written by Paul Black and Dylan Wiliam, was published that highlighted that students who learn using assessment for learning achieve significantly better than matched control groups receiving normal teaching, prompting a huge emphasis on ‘Assessment for Learning’ in UK schools.
Until this point I had not realised that I had been using ‘Assessment for Learning’ throughout my teaching career. Basically:-
“Assessment for Learning’ (AfL) refers to activities undertaken by teachers, and by their students in assessing themselves, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged.”
The UK Assessment Reform Group (1999) identified ‘The big 5 principles of assessment for learning
1. The provision of effective feedback to students.
2. The active involvement of students in their own learning.
3. Adjusting teaching to take account of the results of assessment.
4. Recognition of the profound influence assessment has on the motivation and self-esteem of pupils, both of which are critical influences on learning.
5. The need for students to be able to assess themselves and understand how to improve.
Since I had struggled to cope as a student with most teachers in school, I had learnt most effectively outside lessons or in sport, using my own analysis, assessment and practice. it appears that ‘Assessment for Learning’ was central to my approach to teaching right from the start in science and sport simply because my own experience, thinking and research had discovered its effectiveness.

The Five Learning Requirements

My experiments in the seventies when teaching science, chemistry and sport provided me with a vast amount of invaluable feedback and learning, transforming my self-awareness and cognition on the skills needed to succeed and teacher effectiveness. I was able to frequently study and explore (experiment) the key factors involved in the learning of my students (and myself) in lessons and in sport, eventually I postulated that there were 5 key factors central to effective learning which I called ‘The 5 Learning Requirements’.

_5_LEARNING_REQUIREMENTS

By the early eighties, following my incessant experimenting and scrutiny, ‘The 5 Learning Requirements’ became central to all my teaching and learning, with this poster being displayed and shared with students, staff and parents. I was determined to analyse the key factors in learning so that I could improve the learning of the students and I integrated the 6 exam skills with the 5 learning requirement to construct a questionnaire to explore the most important learning difficulties encountered by students.

SUPERlearning

The evidence from hundreds of students, together with many discussions and interviews was invaluable and extremely revealing, providing a clear conclusion:

‘Having the motivation (self-discipline) to prioritise their study to overcome their difficulties’, was easily the greatest difficulty.

In recent years these factors have been linked with Mindsets (Carol Dweck), Grit (Angela Duckworth), non-cognitive skills (James Heckman), self-regulation and self-control (Walter Mischel).

Measuring Exam Success

I began this blog by pointing out that my success in exams was a huge mystery to me but radically changed my life. I considered passing the 11+ to be a success because my family knew of nobody that had achieved this before. When I got my O level results I almost feinted with shock (fortunately I was in our kitchen with my family) because I’d passed so many more than anyone expected. However, these apparent successes were very personal and related to my background and environment, but had little statistical viability. In the seventies, when I began teaching there seemed to be little collation and comparison of exam results. I felt that if I was to discover how to help my students achieve exam success I needed to discover a way of measuring exam success.

Year on Year

My Head of department in science when I began teaching in the seventies had used a very simple measure for many years, by simply comparing the number of students who got O Level passes (grades 1-6 in those days) and A level grades, A to E. The following table shows that this approach has been the traditional one since O and A levels were first introduced. In essence, it relies on the cohorts being much the same each year and in the Grammar schools they probably were. However, in the seventies, comprehensive schools had arrived and the cohorts were varying a great deal. Using this simple approach there was a little change in numbers between 1974 and 1979.

SUPERlearning

Inter-Subject

When I became Head of Chemistry in 1978 I decided to introduce another measure of exam success by comparing the student’s grades in their each of the different subjects. I felt this may allow for the variation in cohorts.
O LEVEL EXAM RESULTS

SUPERlearning

This simple extract shows how this analysis compared O level results between departments, which did help me to illustrate how students in my department were achieving better exam success than others. However, it does not really provide an analysis of the relative exam success of each student.

Student-Centred

By the time I began my Dip. Ed. In 1984, the 5 Learning Requirements, 6 Exam Skills and the Marks Lost In Exams Analysis were all well established and it had become clear that exam success was more due to the skills (or qualities as some preferred to refer to them) of the student. Therefore I was determined to create an analysis that would focus on measuring the exam success of each student.
I had measured my own exam success by measuring my ‘passes’ with the people around me. So when I passed the 11+ and we knew of nobody that had achieved that, I considered myself to be successful. When I passed 8 out of 9 subjects at O level (English Language was my only ‘failure’ that I passed in the resit), it was more than almost anyone else in our school year, so again I felt I was successful. From numerous discussions with students, parents and staff it appeared that virtually everyone used this simple (crude) measure, so I decided that I should begin collecting and collating these ‘student-centred’ measures of exam success.
As shown by the table of Examination Results, England and Wales, since the introduction of GCE O levels the benchmark of 5 O level passes was used (the grades varied over years and between exam boards). When GCSEs replaced, O levels and CSEs in the late eighties, this benchmark was continued with Grades C and above being referred to as ‘passes’ despite the term ‘FAIL’ not appearing anywhere. By the time the Government introduced ‘Exam League Tables’ in 1991, I had spent over decade analysing exam results, the availability and emphasis on exam results helped me greatly in my data collection and analysis.
I completed my M.Ed. in Research and Evaluation in 1992, by which time, I had sufficient evidence to illustrate that in most schools:-

VERY FEW STUDENTS ACTUALLY GET ABOUT 5 EXAM PASSESS (A*-C)

MOST STUDENTS WILL ATTAIN ALMOST THE MAXIMUM OR THE MINIMUM NUMBER OF PASSES (A*-C)

Throughout the nineties I was able to collect, analyse, compare and predict results for not only my school but also others.
SUPERlearning_STUDENT_CENTRED

It became very obvious that statistics and data analysis was not a strength for many people (including teachers) and with the increasing data and analyses I experimented in a variety of ways of displaying it, trying to simplify it.
SUPERlearning
I created activities, quizzes, discussions and INSET using the data and evidence from it. My Maths, Science and PSHE students regularly took part in these activities which prompted requests from parents to provide presentations for them. Here is an example of one of these activities

QUESTION

If a typical tutor group contains 31 students, attempt to estimate (guess) the number of students that are likely (an average) to attain:-

SUPERlearning

Gradually through the nineties schools Initially secondaries, then primaries, began trying to analyse exam results, all either used the Year on Year measures or inter-departmental (or inter-school). Whenever, I was given an opportunity to explain the student-centred analysis, the recipient was impressed.

Measuring Student Improvement

If you’ve read my blog https://succesfeelosophy.wordpress.com/now-i-understand-how-to-close-the-disadvantage-gap/ you will be aware that in the late seventies I introduced a phrase (now frequently used) “It’s a marathon not a sprint” to explain why short term measures are very limited in use and it is important to measure frequently only the things that really matter in the long term, William Edwards Deming. ‘The father of quality’, emphasises a similar approach. Therefore, once my research had uncovered the Five Learning Requirements and Six Exam Skills at the start of the eighties, I began researching how to measure the skills that matter as regular, simply and effectively as possible.
As a P.E. teacher and scientist, I had spent many years analysing, measuring and developing the key skills needed to succeed in various sports, therefore, I merely applied this same approach to all my lessons and students. Consequently, I experimented repeatedly with activities, tests, quizzes, and studied students very carefully throughout the eighties to research how to measure the progress of the students, the following report being an example.

SUPERlearning
In 1987, I was able to extend this research a great deal when I became Head of Year/Director of Learning for 250+ students. This meant I had a large cohort for 5 years on which I could experiment and research. Since in the eighties there were no SATs (Key Stage 2 SATs introduced 1994) I used cognitive abilities tests (CAT) scores as a (crude) measure of exam skills alongside their reading age at 11. I had access to all the 250+ students’ end of year exam scores in all the subjects for each year through to their Mock GCSEs and final GCSE grades. I used all this data to compile and analyse my students’ progress.

Progress 8 –“measuring exam success”

“Progress 8 will be introduced for all schools in 2016 (based on 2016 exam results, with the Progress 8 score showing in performance tables published in late 2016/early 2017).
The Progress 8 measure is designed to encourage schools to offer a broad and balanced curriculum at KS4, and reward schools for the teaching of all their pupils. The new measure will be based on students’ progress measured across eight subjects:
From 2016, the floor standard will be based on schools’ results on the Progress 8 measure.” (DofE)

I think the similarity between Progress 8 and my attempt to measure progress of my students in the nineties is obvious. However, I did not have the measurements of academia and exam achievement that is now available, but our objectives were the same. It is important to appreciate that these are only measurements in progress in exam success. With such comprehensive longitudinal data on so many students (I repeated it for my next cohort, another 250+ students) I had clear evidence from this extensive statistical analysis and research. It provided me with some clear conclusions that led to the creation of S.U.P.E.R.learning for success which is explained in part 3 (focuses on effective development of their exam skills).

EXAMS MEASURE the students ability to
• CONCENTRATE on exam papers on their own, in complete silence, without a break
• READ exam papers
• UNDERSTAND the information and the questions in exam papers
• RECALL the information needed in exams
• Realise which information is needed to MATCH the question
• WRITES the answers clear enough to match the mark scheme

Development and application of these SIX EXAM SKILLS depends on
• Organises and plans exam preparation
• Environment at home to study without distractions.
• Friends that have positive attitude to study.
• Has the motivation (self-discipline) to prioritise study to overcome the difficulties

Part3 WILL EXPLAIN

_2_SUPERlearning for EXAM success