The “Math” riddle: Uncovering the true reasons behind the debated O/L mathematics paper

With the heat of Ordinary Level (OL) examinations, various opinions on the recently held mathematics paper are expressed in the media, and yet again our education system as a whole is in the spotlight.

“The assessment is too difficult” seems to be the common public outcry and “A justice will be done this time” seems to be the expected solution. We should ask ourselves “Are the true facts and reasons being addressed in such popular discussions?”, “Do the individuals and entities responsible to the public behave responsibly?” and “Does mass media report responsibly and appropriately?”.

This is not the first time we are hearing such dialogue and before the heat dissipates and the discussions are buried in the past, we should direct our intellectual capacity at its maximum strength to understand why such rhetoric is common and what actually lies beneath it.  A deeper look in to the crux of the issue is timely given its national importance. A broader public discussion on the identified issues and seeking expert know how to remedy the situation with the participation of all stakeholders without prejudice should be the eventual common goal of our nation.

   The conclusion that the authors could reach based on the reasoning in this article are:

1. The true facts and reasons are far from being addressed in the popular discussion.
2. The reactions to public outcry could have been based on real facts and figures.
3. With mass media support reporting without evidence, highlighting unproved claims may only address the issue superficially while overseeing the real problems.
4. Examinations (Assessments) and how it is reported have only promoted instrumental learning and increased competition.
5. The expectation of improving results through changes in assessment and the way outcomes of assessment is reported (signifying insignificant figures), without a serious intervention in the teaching learning process is the root cause.
6. Shortcomings in the teaching learning process is a serious concern.  Inadequate human resource and the lack of interest in human resource development to elicit expected outcomes of education is the major bottleneck.
7. Curriculum revisions in mathematics should encourage higher levels of cognitive development of the students, given that mathematics involves manipulatives mainly in the cognitive domain.
8. Unequal opportunities and emphasis of faulty quality measures in education is the ghostly figure lurking in the background.

Though the discussion is mainly centered about mathematics education in the schools, most of the issues we discuss remains relevant to the overall education of the country. We believe that the main arguments in this article are, at least, applicable to subjects related to Science, Technology, Engineering and Mathematics (STEM). As a national level dialog on improving STEM education is taking place at present, we invite the stakeholders to closely consider what is being discussed here.

A journey of random actions

The quality of an education system depends on to what extent its students express attributes (graduate attributes) such as scholarship, global citizenship and lifelong learning.  Despite numerous changes that our education system was subjected to during the past few decades, whether these attributes have improved in students is a genuine concern. An inquiring mind may even suspect a negative correlation between reforms and the quality of our students.

Above quality, most reforms were expected to address an ever increasing competition, never improving student attributes and diverse opinions on the impact of education on socioeconomic factors. However, giving due diligence to the process of education, the needs in a national education policy and the consideration of mutual dependency of education and socioeconomic factors seems lacking in the reasoning that supports many of these reforms. In fact, aspects of under budgeting the education hidden in these reforms are sometimes more prominent. As a result, the competition has only increased, while the quality of students does not seem to have significantly increased and the contribution from our education system to our economy and society appears to be minimal.

Lapses in improvements suggest that these changes could possibly be based only on poor judgments and improper and insufficient reasoning that does not support the alignment of intended outcomes (such as Scholarship, Global citizenship and Lifelong learning), teaching learning process and assessment. We do not intend to say that all changes have had a negative impact; however, the resultant of all these changes seems to have left only an insignificant or a negative impact and the journey we have come is no better than a random walk.

As the problems in education also lies at the root to most issues we face as a nation, we should thoroughly rethink why these problems persists and how we can overcome them. A dialogue at the national level is needed before bringing in yet more reforms. This article explores only the tip of this “iceberg” that recently made an appearance in the context of assessment.

The status of Mathematics education

Being a compulsory subject for Ordinary Level students, high failure rates in mathematics is the most highlighted concern. Most syllabus revisions and restructurings of the examination are seemingly to address this issue. The fundamental question in this regard is in what proportions teaching, learning and assessment are responsible for this issue. At least the most recent revisions, seems to be addressing higher failure rates by bringing in changes to the assessment process than in the overall quality of teaching learning process.  

For example, Part I of the mathematics paper in the new syllabus (since 2016) is now based on the so called “Essential Learning Concepts” (ELCs) which have been laid down too specifically, even to the extent of giving specific numbers that can appear in certain contexts or giving limits to numerical values. A close investigation of these ELCs suggests that these are more towards making Part 1 more explicit to give more opportunity for students to pass the exam than addressing genuine essential learning concepts in mathematics of day-to-day life. For example, radii of circles that we encounter in day-to-day life are most of the time not multiples of seven. The document on ELCs available online at the National Institute of Education website says the following.

“[Hence,] if the pass mark of the Ordinary Level mathematics paper is considered 35%, any student scoring 70% (This is adjudged the basic achievement level of ELCs) for the first paper can have a pass in mathematics even without scoring any mark for the second paper.”

(http://nie.lk/pdffiles/other/eGr10OM%20ELC.pdf, Page 3. Accessed on December 28, 2017).

The biggest concern with this approach is that how can one expect a substantial improvement in students’ learning without a significant intervention in the teaching learning process. Isn’t this simply saying that “…to pass more students, we have to give an easy exam”?

In extreme circumstances, one may argue results should be improved through a reform that allows number crunching. If so, such reforms should be backed by serious investigations indicating that the problem is in the evaluation system rather than in the teaching learning process. However, in our case we argue that there is more evidence for the problem to be in the teaching learning process than in the evaluation system. To handle minor variations in the examinations, exams like SAT report a standardized score, which is less dramatic than redefining the assessment. Department of Examinations (DOE) and other scholars who have the knowledge and ability should study the OL mathematics paper, analyzing the difficulty level and how it has evolved over time, to back such reforms. Such studies will be of indispensable value to understand why such changes were introduced and should such changes be done in the first place.

In the case that this is indeed the required change and students needed more opportunity to pass mathematics, it should be done with great care, with proper research and planning, so that the STEM education as a whole does not get compromised. More specifically, in our case, if Paper 1 should be sufficient to pass OL mathematics, Paper 2 should be able to assess higher order skills enabling to differentiate talented students. Moreover, it could be the case that maintaining a minimum pass rate is important. However, it was not clear to the authors even whether there is a proper understanding of what that acceptable failure rate should be. Low pass rates and under performance in mathematics is common to many countries. Therefore, it is worth looking at the changes other countries have adopted, which can achieve both acceptable pass rates while promoting STEM education. We believe most of these reforms are in the form of improvements in teaching and learning process.

With the attempt to find solutions to problems in mathematics education in assessment, many curriculum revisions seems to undermine the cognitive domain of the mathematics education. A (school) curriculum should cover all domains covering cognitive skills, fine motor skills and attitudes. In school curriculum, mathematics plays a major role in the cognitive domain. However, over the decades more and more aspects that challenge students’ cognitive domain, such as geometry, has been undermined in mathematics curriculum. Even the syllabus and guidelines, in some instances, seems to be focusing on how to “restrict” assessment, rather than being more specific on outcomes expected, which discourage cognitive development and relational learning. For example OL mathematics paper is supposed to test topics from grade 10 and 11 only. It would have been much better, if the expected learning outcomes were mentioned, than giving such narrow specifications. Questions and concerns raised by such reforms are serious, though not discussed in public. Thus, we intend to study these concerns in a separate article.

Is the problem in assessment? Or is it in the teaching learning process? Lots of useful information in this regard was shared in the “Symposium on Mathematics Education in the Secondary Education: Past, Present and Future”, organized by the Ministry of Education in March 2017. During the symposium, many scholars indicated the need and ways of improving the teaching learning process and have identified equipping teachers with proper understanding and methodologies through teacher training as an important factor. “Pedagogical Content Knowledge in Mathematics Teacher Education” by G. M. Wadanambi, “Does one size fits all? Authentic Instruction and Assessment in Mathematics” by Dr. U. Sedere and “Multilevel Teaching as one model of Child Centered, Child friendly methodology in primary education” by Mrs. A. Pandithasekara are only a few to mention. We should ask ourselves to what extent we have adopted these into our education system.

Some important statistics were reported in “GCE (OL) Examination 2015” by G. Abeygunasekera, at the same symposium. It indicated that for the 2015 OL mathematics paper, more than 20% of the students have taken marks in the range 0 to 10 and 19% is in the range 11 to 20 marks. Thus, more than 39% of the students have only 0 to 20 marks. These numbers direct us to the root cause of the problem: The quality of the teaching learning process of mathematics education. The authors of this article believe that the fact that over 19% of the students get less than 11 marks is a strong indicator of the serious concerns in the teaching learning process. The statistics for the other years are also comparable. For example the percentage of students with 0 to 10 marks in 2013 was about 14% and in 2012 it was 18%.

Evaluation reports published by the DOE contains a commentary on a problem to problem basis. These sources provide valuable information on the status of mathematics education in our country and every individual interested and responsible for mathematics education should have referred these. Numbers that appear in these reports often can only be described as shocking and raises serious concerns about students’ learning.  For example in 2011, 44% percent of the candidates could not calculate  and in 2013, 35% percent of the students were not able to correctly express 1 ½ meters in centimeters!

The above numbers suggests that by simply improving the knowledge in basics and fundamental concepts, a significant improvement in the pass rate could be achieved. DOE could shed light by studying the percentage improvement that could be expected if the failing population had known basics. This number is essential to continue a meaningful dialog in the future.

What about teachers? There is growing evidence that the lack of proper teacher training is a major bottleneck in improving students learning. We also invite the responsible authorities that have more information on this regard to share their findings with the nation, to facilitate a healthy dialogue. Not only mathematical knowledge, but teaching methodologies and understanding of pedagogical aspects are also a serious concerns. The authors, through numerous teacher training workshops and by other interactions, have also learned that teacher, as well as many individuals responsible for mathematics education in the system, lack proper understanding of various mathematical and pedagogical concepts. However, when the opportunities are given in the form of training and dialogue, significant improvements have been achieved. No one is perfect and we as a nation should seriously consider human resource development, by providing ample opportunities, and empowering our valuable individuals to do an even better service. However, this aspect has been widely neglected not only in most educational reforms, but also in day to day practice.

We believe that we have given enough evidence to indicate that the real problems lies in the teaching learning process. However, the story does not stop here. There is another twist to this story. For that we need to look at some more data!

Table 1: Distribution of O/L Mathematics Grades for the years 2010-2016. 2016 is a new syllabus.                                                 Source: http://www.doenets.lk/exam/downloads.jsf

The celebrity status of the mathematics paper is revealed through this table. Note that in all the years number of “A”s is significantly larger than the number of “B”s. In 2010, 2011, 2013 and 2016 the number of “A”s is almost twice the number of “B”s. We believe that this disproportionate number of “A”s, caused by the way grades are reported, have given many students an unrealistic expectation on their mathematical ability. This could be a major reason for the exaggerated competition to get an “A”. There also seems to be a correlation with the public opinion of the difficulty level of the paper and the ratio between “A” and “B”. The years where the number of “A”s go down compared to number of “B”s are the years the mathematics paper was criticized. Thus, despite persisting failure rates, the opinions that are addressed in the public media actually do not concern the failing students. These opinions actually reflects the unrealistic student expectations due to the competition to get an “A”. Also, in many cases the voice is raised mostly on behalf of those who fail to achieve unrealistic goals.

In each year, at least in the way grades are reported, suggests that our student population at large is divided in to two camps (bi-modal). It is worth asking whether there is a privileged group, most of whom that obtain “A” passes, and an underprivileged group, most of whom simply fails. We should pay more attention to how the socioeconomic factors like income and demography plays a role in the education and if so what are the determining factors. For example, the “A” percentages of OL mathematics paper in 2015 for some demographic areas (educational zones) are as follows: Colombo, 28.6%; Homagama, 7.34%; Kandy, 22.54%; Gampola, 4.38%.  Fail (reported as “W”) rates also indicate the same story: Colombo, 29.15%; Homagama, 50.06%; Kandy, 28.39%; Gampola 57.39%. We invite the interested reader to refer evaluation reports published by DOE.

It is not a secret that equal opportunity for education is being abused in public, and is at the root of various public outcries. It forces a greater urge in individuals and resources to fight for resources. At a time free school education caters only a privileged community, genuine reforms should be brought in to guarantee equal education. It is of great dismay to see that we are yet to address this issue.

As a result of unequal opportunities, disproportions in reporting grades and negative influence of assessment, a significant portion of our students are caught in a rat race, while another portion idling without access to a quality education. As a result, the entire students’ community, on average, seems to achieve less in the long run. For example, despite high “A” rates in OLs, the combined mathematics results seems to be in decline, as more and more students with exaggerated high grades are pumped in to the stream.

Disparities in opportunities in education is a serious concern and each year the problem seems to be only getting worse. The restoration of equal opportunities in education, through human resource development, should be a key strategy. Increased higher and vocational education opportunities will lessen similar issues in the Advance Level.

Though it is too early to say much on the new syllabus based only on the figures for 2016, we invite the readers to think about what the numbers could mean, particularly in the context of this discussion.

Let us summarize what we have observed.

1. Syllabus revisions or reforms in the assessment process will not provide solutions to the existing problems in mathematics education at the moment. The drawback is in the teaching learning process and disparities in educational opportunities are the driving forces of the present day issues.

2. Human resource development in the form of recruiting qualified mathematics teachers and training those who are already in the system is the key to improve the teaching learning process. Frequent opportunities for the teachers to involve in quality dialogue are a must for them to be up-to-date.

3. Inequality in educational opportunities is the next driving force. Human resource development is an important part in addressing this issue.

We hope there is ample reasoning to support these propositions. Though these are not new findings, the public seems to be unaware of the genuine reasons behind issues that they and their children are facing.

The role of assessment

Education has three major components: Teaching, Learning and Assessment. The previous discussion (Last week) mainly revolved around teaching and learning. What about Assessment? The most sensitive topic to us!

The fundamental aspects of the assessments are at least three fold. See http://www.edu.gov.mb.ca/k12/assess/role.html for more information.

1. Assessment for learning.

Assessment helps gain insight into what students understand in order to plan and guide instruction, and provide helpful feedback to students.

2. Assessment as learning.

Students develop an awareness of how they learn and use that awareness to adjust and advance their learning, taking an increased responsibility for their learning.

3. Assessment of learning:

Assessment informs the entire community the success, the required interventions and support that needs to be continued.

Assessment in fact governs the entire teaching learning process; determining how students are motivated and learn and how teachers teach. Just as we animals adopt (evolve) to face challenges presented by nature, which is the assessment of toughness, the teaching learning process adopts to survive the assessment process. Thus, assessment must be planned with all of its purposes in mind. All of these three aspects have a role to play in supporting and improving the teaching learning process, and must be appropriately balanced.  An important part of assessment is the interpretation and use of the information for its intended purpose.

The tuition culture (or should we call it “Assessment Crisis”): A case study

First of all, what we mean by the “Tuition Culture” is the promotion of instrumental learning, the habit of cramming to pass exams, irrespective to whether it is done in a tuition class or in the classroom of a school.  We apologize to the few really good teachers, who are operating in their private capacity to fill in the blanks in the public education system, for using this term.

The wide spread tuition culture is a real burden to our education system and all the individuals related to the education are affected by it. It is often an important topic in the discussions related to education reforms and we expect that through new reforms this problem can be solved. This was done in the past and we even hear it in the present, with various responsible individuals stating how they think the proposed reforms will solve this issue. The irony is that we have not understood that what promotes the tuition culture is nothing but our exams, the assessment. Let us get this straight. Keep in mind the roles assessment plays in the teaching learning process, which we mentioned above.

The teaching learning process adopts to the evaluation system. When a predictable poor set of assessments, which for example promote instrumental learning, are presented to the community for a prolonged time period, the teaching learning process learns that what is being evaluated is cramming and adjust to it. This is exactly what has happened to our education system. Our society, which is at large in a rat race, really like this and it is exactly the change that they do not like.

We believe that our exams are in this regard responsible at large for the tuition culture. Had the assessment been done correctly with proper understanding, even private education could have been guided towards the proper direction. It is an unrealistic exercise to try to stop the tuition culture and related stresses by changing the syllabus, or by reducing the number of subjects or by any other means than assessment.

It should be mentioned here that some important changes, revisions and inclusions in the assessment are included in the new mathematics syllabus. However, it is really unfortunate that the true advantages of these changes are hindered due to the issues in the teaching learning process and diverse opinions on assessment.

The events that followed the recent OL mathematics paper is the best example. One major criticism is centered on one particular problem, where the public opinion went to the extent of saying “it was even hard for the teachers”. Careful analysis and discussions with teachers shows that the real reason behind this is that for a prolonged period of time, some basic concepts were not promoted in the community in a proper way and even teachers have forgotten grade 6 to 9 basic arithmetic. What followed also indicates the real stress in the public related to the assessment process.

Due to the stresses in the education system, such as unequal opportunities, and their prolonged existence, we as a nation have completely misunderstood and lost the ability to understand the true meaning of assessment. To us, assessment has unfortunately boiled down to the judgment of whether “we live or not”, and the society always is suspicious of any change, good or bad.

The issues in assessment, its subjective nature rather than the objective nature, is one of the biggest problems we are facing in our education system. There are several ways to improve the issues in our assessments.

1. Proper understanding of the role of assessment.

An exam is for a generation, it determines the future learning process and thus cannot be restricted to overcoming only problems in this year.

2. Use numbers and statistics in the making and decision making process of assessments, instead of opinions.

3. Use the numbers as a feedback to clearly understand how assessment should change.

For example, if a particular problem or a section turns out to be too difficult for the students but is an important learning outcome, then we cannot simply say this is too hard for the students. Instead we should promote learning it by emphasizing that aspect in the assessment.

Towards a progressive mathematics education

At a time where various opinions are expressed and dialog is on, we wish to emphasize that we should be thinking of addressing some of the major issues that we have highlighted here first, as one should fix the bottom that is falling off from a bucket before fixing the handle. We need to work towards a national policy covering the aspects of curriculum, teaching, learning and assessment. Decision making should be based on data and research, not on the thinking of how to survive this time!

Responsible parties should seriously think of teacher training, recruiting quality individuals and retaining them in the system.  Once this is in process, disparity in educational opportunities, at least at the school level, can be reduced. Any reform that is brought in without properly addressing these could be another disaster.

Finally, this will be incomplete unless we relate our discussion to a recent dialog on Finland and its education system. Of course, there are so many lessons to learn from Finland and it would be great to have such a system in our country as well. However, we seem to be learning it all in the opposite direction: the last lesson first.

Finland has come to this status through decades of proper investments in their teachers. The educators in Finland are now highly qualified and well-paid. As a result, they at present have come to a position which enables them to introduce more radical changes to the teaching learning process in their country. We should not forget that they have become an example only because they have invested early on their educators and they are now rich in human resources. We wish to cite the following.

“This recognition led to a sweeping set of reforms that significantly raised the bar for aspiring teachers by moving teacher preparation from the seminarium (the Finnish equivalent of teacher college) into the university, and ultimately requiring all teachers, primary through upper secondary, to obtain a masters degree as a condition of employment. … Finland also has a long tradition of in-service teacher training that developed over the years as national curricular changes have been implemented. During the intensive adaptation to the new educational structure from 1972 to 1977, Finland instituted a special, comprehensive, compulsory in-service training programme for all teachers in all municipalities.”

(https://www.oecd.org/pisa/pisaproducts/46581035.pdf, Page 5. Accessed on December 28, 2017)

Now imagine what will happen if we adopt their most recent change before doing what they did first! Especially when the OL mathematics paper is even a challenge to a teacher!

Dr. Jayampathy Ratnayake (PhD, Indiana University, Bloomington, USA). Senior Lecturer, Department of Mathematics, University of Colombo.

Dr. Siyath Gunawardena (PhD, University of Maine, Maine, USA). Senior Lecturer, Department of Physics, University of Colombo.

Dr. Jayantha Senadheera (PhD, North Texas University, Texas, USA). Senior Lecturer, Department of Mathematics, Open University of Sri Lanka.