Uncategorized

Technology causes deflation

Technology enables us to do more in a period of time.  A classic example is the washing-machine that requires you to do little more than load your dirty clothes and switch it on rather than laboriously wash, scrub and rinse each item repeatedly.  It costs less time to do the same thing and so we experience time-deflation.  It’s the same as with money: if you can buy two hamburgers today for the price of one yesterday then there has been some deflation.  In these circumstances, it becomes less important to have a large income because the necessities of life have reduced in price, and so you could work less hard, start saving more (but for what?) or buy some of life’s luxuries.  However, the analogy between time and money breaks down at this point, because you can’t reduce your supply of time or save it, you have to spend it.  But advancing technology means nearly everything costs less time and so it gets harder and harder to spend your alloted time.  Many of us react by trying to do more and more diverse activities, and often simultaneously, with the result that we over-compensate for time-deflation and become bankrupt, or burnt out wrecks.

We can cheat technology’s deflating effect by pursuing activities that involve no time-saving technology such as walking, reading, thinking and spending time with our loved ones.  In the last case, the clue is in the phraseology!

BTW – I will be on deep vacation by the time you read this post. Amongst other things, I will be curing my tsundoko by reading the books I bought in Camden Lock Books earlier in the summer [see my post entitled ‘Tsundoko‘ on May 24th 2017].

Feedback on feedback

Feedback on students’ assignments is a challenge for many in higher education.  Students appear to be increasingly dissatisfied with it and academics are frustrated by its apparent ineffectiveness, especially when set against the effort required for its provision.  In the UK, the National Student Survey results show that satisfaction with assessment and feedback is increasing but it remains the lowest ranked category in the survey [1].  My own recent experience has been of the students’ insatiable hunger for feedback on a continuing professional development (CPD) programme, despite receiving detailed written feedback and one-to-one oral discussion of their assignments.

So, what is going wrong?  I am aware that many of my academic colleagues in engineering do not invest much time in reading the education research literature; perhaps because, like the engineering research literature, much of it is written in a language that is readily appreciated only by those immersed in the subject.  So, here is an accessible digest of research on effective feedback that meets students’ expectations and realises the potential improvement in their performance.

It is widely accepted that feedback is an essential component [2] in the learning cycle and there is evidence that feedback is the single most powerful influence on student achievement [3, 4].  However, we often fail to realise this potential because our feedback is too generic or vague, not sufficiently timely [5], and transmission-focussed rather than student-centered or participatory [6].  In addition, our students tend not to be ‘assessment literate’, meaning they are unfamiliar with assessment and feedback approaches and they do not interpret assessment expectations in the same way as their tutors [5, 7].  Student reaction to feedback is strongly related to their emotional maturity, self-efficacy and motivation [1]; so that for a student with low self-esteem, negative feedback can be annihilating [8].  Emotional immaturity and assessment illiteracy, such as is typically found amongst first year students, is a toxic mix that in the absence of a supportive tutorial system leads to student dissatisfaction with the feedback process [1].

So, how should we provide feedback?  I provide copious detailed comments on students’ written work following the example of my own university tutor, who I suspect was following example of his tutor, and so on.  I found these comments helpful but at times overwhelming.  I also remember a college tutor who made, what seemed to me, devastatingly negative comments about my writing skills, which destroyed my confidence in my writing ability for decades.  It was only restored by a Professor of English who recently complimented me on my writing; although I still harbour a suspicion that she was just being kind to me.  So, neither of my tutors got it right; although one was clearly worse than the other.  Students tend to find negative feedback unfair and unhelpful, even when it is carefully and politely worded [8].

Students like clear, unambiguous, instructional and direction feedback [8].  Feedback should provide a statement of student performance and suggestions for improvement [9], i.e. identify the gap between actual and expected performance and provide instructive advice on closing the gap.  This implies that specific assessment criteria are required that explicitly define the expectation [2].  The table below lists some of the positive and negative attributes of feedback based on the literature [1,2].  However, deploying the appropriate attributes does not guarantee that students will engage with feedback; sometimes students fail to recognise that feedback is being provided, for example in informal discussion and dialogic teaching; and hence, it is important to identify the nature and purpose of feedback every time it is provided.  We should reduce our over-emphasis on written feedback and make more use of oral feedback and one-to-one, or small group, discussion.  We need to take care that the receipt of grades or marks does not obscure the feedback, perhaps by delaying the release of marks.  You could ask students about the mark they would expect in the light of the feedback; and, you could require students to show in future work how they have used the feedback – both of these actions are likely to improve the effectiveness of feedback [5].

In summary, feedback that is content rather than process-driven is unlikely to engage students [10].  We need to strike a better balance between positive and negative comments, which includes a focus on appropriate guidance and motivation rather than justifying marks and diagnosing short-comings [2].  For most of us, this means learning a new way of providing feedback, which is difficult and potentially arduous; however, the likely rewards are more engaged, higher achieving students who might appreciate their tutors more.

References

[1] Pitt E & Norton L, ‘Now that’s the feedback that I want!’ Students reactions to feedback on graded work and what they do with it. Assessment & Evaluation in HE, 42(4):499-516, 2017.

[2] Weaver MR, Do students value feedback? Student perceptions of tutors’ written responses.  Assessment & Evaluation in HE, 31(3):379-394, 2006.

[3] Hattie JA, Identifying the salient facets of a model of student learning: a synthesis of meta-analyses.  IJ Educational Research, 11(2):187-212, 1987.

[4] Black P & Wiliam D, Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1):7-74, 1998.

[5] O’Donovan B, Rust C & Price M, A scholarly approach to solving the feedback dilemma in practice. Assessment & Evaluation in HE, 41(6):938-949, 2016.

[6] Nicol D & MacFarlane-Dick D, Formative assessment and self-regulatory learning: a model and seven principles of good feedback practice. Studies in HE, 31(2):199-218, 2006.

[7] Price M, Rust C, O’Donovan B, Handley K & Bryant R, Assessment literacy: the foundation for improving student learning. Oxford: Oxford Centre for Staff and Learning Development, 2012.

[8] Sellbjer S, “Have you read my comment? It is not noticeable. Change!” An analysis of feedback given to students who have failed examinations.  Assessment & Evaluation in HE, DOI: 10.1080/02602938.2017.1310801, 2017.

[9] Saddler R, Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in HE, 35(5):535-550, 2010.

[10] Hounsell D, Essay writing and the quality of feedback. In J Richardson, M. Eysenck & D. Piper (eds) Student learning: research in education and cognitive psychology. Milton Keynes: Open University Press, 1987.

Uncertainty about Bayesian methods

I have written before about why people find thermodynamics so hard [see my post entitled ‘Why is thermodynamics so hard?’ on February 11th, 2015] so I think it is time to mention another subject that causes difficulty: statistics.  I am worried that just mentioning the word ‘statistics’ will cause people to stop reading, such is its reputation.  Statistics is used to describe phenomena that do not have single values, like the height or weight of my readers.  I would expect the weights of my readers to be a normal distribution, that is they form a bell-shaped graph when the number of readers at each value of weight is plotted as a vertical bar from a horizontal axis representing weight.  In other words, plotting weight along the x-axis and frequency on the y-axis as in the diagram.

The normal distribution has dominated statistical practice and theory since its equation was first published by De Moivre in 1733.  The mean or average value corresponds to the peak in the bell-shaped curve and the standard deviation describes the shape of the bell, basically how fat the bell is.  That’s why we learn to calculate the mean and standard deviation in elementary statistics classes, although often no one tells us this or we quickly forget it.

If all of you told me your weight then I could plot the frequency distribution described above.  And, if I divided the y-axis, the frequency values, by the total number of readers who sent me weight information then the graph would become a probability density distribution [see my post entitled ‘Wind power‘ on August 7th, 2013].  It would tell me the probability that the reader I met last week had a weight of 70.2kg – the probability would be the height of the bell-shaped curve at 70.2kg.  The most likely weight would correspond to the peak value in the curve.

However, I don’t need any of you to send me your weights to be reasonably confident that the weight of the reader I talked to last week was 70.2kg!  I cannot be certain about it but the probability is high.  The reader was female and lived in the UK and according to the Office of National Statistics (ONS) the average weight of women in the UK is 70.2kg – so it is reasonable to assume that the peak in the bell-shaped curve for my female UK readers will coincide with the national distribution, which makes 70.2kg the most probable weight of the reader I met last week.

However, guessing the weight of a reader becomes more difficult if I don’t know where they live or I can’t access national statistics.  The Reverend Thomas Baye (1701-1761) comes to the rescue with the rule named after him.  In Bayesian statistics, I can assume that the probability density distribution of readers’ weight is the same as for the UK population and when I receive some information about your weights then I can update this probability distribution to better describe the true distribution.  I can update as often as I like and use information about the quality of the new data to control its influence on the updated distribution.  If you have got this far then we have both done well; and, I am not going lose you now by expressing Baye’s law in terms of probability, or talking about prior (that’s my initial guess using national statistics) or posterior (that’s the updated one) distributions; because I think the opaque language is one of the reasons that the use of Bayesian statistics has not become widespread.

By the way, I can never be certain about your weight; even if you tell me directly, because I don’t know whether your scales are accurate and whether you are telling the truth!  But that’s a whole different issue!

Listening with your eyes shut

I am in the London Underground onboard a train on my way to a conference on ‘New Approaches to Higher Education’ organised by the Institution of Engineering and Technology and the Engineering Professors’ Council.  The lady opposite has her eyes closed but she is not asleep because she opens them periodically as we come into stations to check whether it’s her stop.  I wonder if she is trying to reproduce John Hull’s experience of the depth of sounds as a blind person [see my post entitled ‘Rain brings out the contours in everything‘ on February 22, 2017].  For the second time in recent weeks, I close my eyes and try it for myself.  It is surprising how in a crowded train, I can’t hear anyone, just the noise made by the train.  It’s like a wobble board that’s joined by a whole percussion section of an orchestra when we go around a bend or over points.  The first time I closed my eyes was at a concert at the Philharmonic Hall in Liverpool.  My view of the orchestra was obstructed by the person in front of me so, rather than stare at the back of their head, I closed my eyes and allowed the music to dominate my mind.  Switching off the stream of images seemed to release more of my brain cells to register the depth and richness of Bach’s Harpsichord Concerto No. 5.  I was classified as tone deaf at school when I was kicked out of the choir and I learned no musical instruments, so the additional texture and dimensionality in the music was a revelation to me.

Back to the London Underground – many of my fellow passengers were plugged into their phones or tablets via their ears and eyes.  I wondered if any were following the MOOC on Understanding Super Structures that we launched recently.  Unlikely I know, but it’s a bit different, because it is mainly audio clips and not videos.  We’re trying to tap into some of the time many people spend with earbuds plugged into their ears but also make the MOOC more accessible in countries where internet access is mainly via mobile phones.  My recent experiences of listening with my eyes closed, make me realize that perhaps we should ask people to close their eyes when listening to our audio clips so that they can fully appreciate them.  If they are sitting on the train then that’s fine but not recommended if you are walking across campus or in town!

Tsundoku

I used to suffer from tsundoku but now I am almost cured…  Tsundoku is a Japanese word meaning ‘the constant act of buying books but never reading them’.  I still find it hard to walk into a good bookshop and leave without buying a small pile of books.  I did it early this month in the Camden Lock Books and left with ‘The New Leaders‘ by Daniel Goleman, ‘What we talk about when we talk about love‘ by Raymond Carver and ‘The Fires of Autumn‘ by Irène Némirowsky.  I will probably read all of these three books over the coming months so it was not really an act of tsundoku.  But, it’s perhaps only because there are so few really good bookshops left that I don’t  buy more in a year than I can read.  Although this is not quite true in my professional life, because I have started buying books on-line and the pile of unread books in my office is growing; so I am not completely cured of tsundoku.  Actually, all researchers are probably suffering from it because we collect piles of research papers that we never read – in part because we can’t keep up with the 2.5 million papers published every year.  And, it’s growing by about 5% per annum, according to Sarah Boon; perhaps, because there are more than 28,000 scholarly journals publishing peer-reviewed research.  Of course, that’s what happens if you measure research productivity in terms of papers published – it’s a form of Goodhart’s law [see my post entitled ‘Goodhart’s Law‘ on August 6th, 2014].