Deconstructing “Authentic” Assessment⤴
from Blue Collar
view: full | short | titles only
from Blue Collar
from Blue Collar
from Blue Collar
from howsheilaseesIT
Undoubtedly LLM (large language models), and in particular ChatGPT, is the hot topic in education right now. David Hopkins has helpfully started and shared a flipgrid where he is sharing articles around generative AI, and I know many others are doing the same. Amongst the hype there are thankfully a growing body of people who are writing informed critiques. In this post I just want to quickly highlight a couple of publications that I think are a must read.
Firstly the UNESCO Quick Start Guide to ChatGPT and Artificial Intelligence. This is provides a really good overview of issues including a useful flow chart to help decisions around using ChatGPT, applications for education and some of the current issues. I suspect this will become a “go to” resource. It’s something that all educators should read.
And once they’ve done that then I have to recommend 2 longer pieces by Helen Beetham. Firstly, “on language, language models and writing“. In this essay, Helen really gets to grips with a key issue that is missing in many of the articles about LLM and ChatGPT, that is what is the purpose of writing? Why do we do it? It’s not just about structuring of text, personal reading. I think most people (well at least you, dear reader) does now understand that these language models work on prediction, and have no sense of context. So although the text may read well, it will often lack purpose and understanding. As Helen points out ” Writing by human writers is not only about the world, it is of the world and accountable in it.”
She goes on to explore some of the potential benefits of using systems such as ChatGPT. Can they be seen as writing partners? We supply the prompts, they supply the text . . ? I was struck by this.
“The illusion that these are more than tools or interfaces – that they are our partners in language, our interlocutors. We already spend large parts of our lives engaged in vivid graphical and sensory illusions. We should count the costs and benefits before rushing into a life of dialogue with illusory others”
And this
“Students see writing as a diverse, messy, inexact, variously motivated practice they are developing for themselves. Then perhaps they can aspire to be a writer among writers, and not a human version of ChatGPT.
I thank Helen for being the writer she is to have come up with that last turn of phrase. And then she goes on to point out:
But tools are not neutral. Just as language is not ‘simply’ the words we use to express our meanings to other people, tools are not ‘simply’ the means we use for exercising our personal intentions in the world. Tools carry the history of how they were designed and made. They shape practices and contexts and possible futures. . . With so many other tools we can use creatively, we must surely weigh the risks against the creative possibilities.”
In terms of education Helen also raises some really valid points for strategic leadership in universities. It does seem an awful lot of responsibility is being heaped on students, maybe we need to be asking these questions
“While students are held stringently to account for their use of LLMs, how will universities account to students for their own use of these systems? Can they hold out against black-box capabilities being embedded into the platforms they have come to depend on? Who is assessing the risks, and how are those risk assessments and mitigations being shared with the people most affected? These are questions that universities should be attending to with at least as much energy as they are policing students’ use of apps.”
There is also an accompanying piece students assignments in a time of language modelling. Again this is a really thoughtful (and pragmatic) piece about why, how and when to use writing tasks in assessments.
I would thoroughly recommend reading both essays, and engaging with Helen’s writing over on substack.
from NomadWarMachine
Recently I’ve heard two different people say the following about assessment:
I think that both of these statements are obviously wrong, or they ought to be.
I can, sadly, understand how assessment becomes the motivating factor for many students because the stresses of being a student and the pressure to always get the best grade override any joy that there might be in learning for the sake of learning, so I can appreciate (1) being said, at least as an observation about the current state of affairs in HE.
The last semester that I taught undergraduate philosophy was particularly fraught, with many tutorials being cancelled due to union strikes and others snowed off, but one event stands out, even now. A particularly bright student went off at a tangent, and I suggested some reading that might interest him, probably some Wittgenstein. After the tutorial, a group complained to the course convenor that I was wasting their time by talking about materials that would not be needed for the final exam. This wasn’t the only reason that I handed in my notice and walked away from that post, but it was a contributing factor.
However, (2) just strikes me as bizarre. Putting to one side my issues with the term ‘active learning’ (I don’t believe that learning can be passive – for learning to be happening, there must be something active ‘in the learner’s head’), then I think that we could reasonably assert that a student who is working on an assessment is actively learning, but that does not entail that all (active) learning is or involves assessment. When I reflect on the times that I have learnt the most, they have usually had nothing to do with assessment. The serious fun that happens during experiences such as #CLMooc, #DS106 and the rhizos have led to some of the most meaningful learning that I’ve ever experienced – and assessment most definitely had no part in those experiences.
The bright student in my story above might have ended up using the ‘extra’ learning he had done as part of a future assessment, or he might not – learning is not contingent on past or future assessment. All of this shows that active learning and assessment are not the same thing, and that there can be learning without assessment. And I think I am going to go further and assert that there can also be a type of learning that happens despite assessment. If students are over assessed, as they sometimes can be, then any extra curricula activity – and the meaningful learning that happens as a result of serendipity – will have to be fitted in somehow. And if there is pressure, from whichever direction, to teach to the test, then the keen student who strays beyond the required materials will be learning despite assessment.
Assessment has its place, and accreditation is important. But assessment is not the whole picture, and we should firmly resist anyone who suggests that it is.
from howsheilaseesIT
Following on from the report that has just been published on Approaches to Curriculum and Learning Design in the UK HE sector, Helen Beetham and I are exploring some of the key issues that were highlighted through the survey and the interviews we conducted. Central to this are issues around time, space and place. Earlier this week we were able to start to share some of our initial thinking during a workshop at the Jisc Student Experience Experts Meeting.
In the interviews I conducted as part of the project, there was a general consensus that after the first lockdown most organisations were quite keen, even quite ambitious about their future plans for new approaches to learning and teaching. There was a sense of an appetite to embrace some the changes to practice that being forced off campus had brought about. Assessment was a huge part of that.
Rapid changes to assessments had to be introduced, along with rapid changes to assessment regulations. Student care was high on the agenda – a visible sign of that was the no detriment practices that many adopted. Again in the interviews, it was clear that lots of the changes from f2f exams to online submissions of various types including open book, authentic assessments have now been adopted.
In terms of wider curriculum change, it was also clear from the survey responses and interviews that the appetite for changes to other aspects of curriculum design and delivery had been divisively impacted by the UK Governments’ insistence that everyone needed to be back on campus, at lectures and doing “proper” in person exams. Never mind the lessons that had been learnt from students about the benefits of more flexible, accessible and inclusive approaches. Strategic statements were subtly altered to reflect as a pragmatic response to that political driver.
However, back in the real world, we can’t ignore that our understandings and use of the spaces, places (both physical and digital) and times for learning and teaching have been altered by the pandemic experience. Students have been off campus, on campus, off campus, on and off campus for a bit . . . and now on campus. Typical 1st and 2nd year students have had their final years of school turned upside down in the same way.
I think how “be” a student has changed, and that might be one of the reasons there have been so many issues around engagement. Where (and when) you actually need to be isn’t as clear cut as it was in the “before times”.
Going back to assessment, some of the comments student interns on the Irish EDTL project made during one of their webinars really struck me. Including the student who very eloquently shared how being able to take assessments off campus, in a space that was comfortable for them, massively reduced their stress levels; another who felt that the design of some of the online MCQs exams they had taken were “mean” as they didn’t allow you to go back to a question to answer it. That experience was making them want almost long for pen and paper exams. In the panel discussion at the experts meeting, Deborah Longworth from the University of Birmingham shared how some changes to assessment are now having impact on the mental health of students. She described how some students can think that a 72 hour open book exam means that they need to be working on it for 72 hours. Does this mean taking time to develop more scaffolding around time expectations, or is it an “in” to go back to fixed, in person exam that everyone understands the conventions of ?
Whilst terms such hybrid and hyflex are commonly used and, are they really fully understood by both students and staff? Do we really have effective examples of how these approaches work in practice. This is one area Helen and I want to explore from a pedagogical lens.
We are starting with time, and thinking in terms of synchronous and asynchronous. Then considering what types of activities/interactions that work best in these contexts, and then starting to map the spaces and places that students and staff need to be in as these activities are instantiated. In terms of broadening our approaches to learning design, do we need to be more explicit about time, space and place expectations in?
As the cost of living crisis starts to really kick in, what additional changes do we need/ are we making to make to our physical estate to support our students (and staff). Warm areas, areas with kettles? What choices might commuting students have to make about how many times a week they can be on campus?
As we discussed these issues in the meeting, a dose of pragmatism was injected into the conversation. Whilst it is often said that pedagogy should always come before technology, in reality it’s pragmatism, and the contextual constraints that everyone has to work with that really make have “the power”. Pragmatics always win over everything else.
I know I have run many learning design workshops where some really innovative approaches have been planned, only to find out that 2 weeks before the start of term, the plans have been changed because of timetabling issues or more commonly not enough staff resource or time.
As the sector moves forward is it just easier to cope with increases in student numbers, and the staff/studio ratio to just timetable in lectures? Is it just pragmatically more effective not to change workload models and notions of contact time to reflect the shifts in preparation/contact time and presence needed, and stick with the conventions we are all familiar and comfortable with?
Hopefully not, and that’s what we are working on now, to develop resources that can help provide guidance and exemplars of how the sector can, and is, evolving to allow us to think about pedagogy and place and hopefully start to change some of the pragmatics and constraints approaches to learning design, and in turn the student experience, exist in. I know Peter Bryant’s recent post on the “snapback” discusses many of these issues in more depth so is worth a look if you haven’t seen it yet.
So if you have any thoughts on this, or would like to share any examples, please do get in touch, or leave a comment. We want to provide spaces to have these conversations and hopefully provide some resource to help others have them in their contexts.
from Malcolm Wilson
If you are looking for a way to support learners (or indeed, yourself!) to be clear, expressive and confident when communicating & presenting to others using digital tools, then Microsoft PowerPoint Presenter Coach is a handy feature of Microsoft PowerPoint. It’s built in to PowerPoint whether on the web, on desktop/laptop or mobile device, and …
from Malcolm Wilson
The Inspiring Digital Enterprise Award, known as iDEA, is an award programme which provides access to a range of free on-demand online courses. In completing courses from the wide range available they not only aim to support developing digital, enterprise and employability skills, the completion of the assessments built into each course provide the means …
from NomadWarMachine
A podcast recorded on December 2020 when I talk a lot about assessment, feedback and participatory learning
I am doing a bit of tidying up and catching up on some posts that should have appeared earlier in the year. What a strange yet busy time it has been. In early November I was asked to contribute to this global conference on assessment. You can find my modest contribution from a College perspective in day three of the proceedings.
Beyond Multiple Choice #BMC2020 ( worth clicking on timestamp below videos to see running schedule)
It was a really interesting conference given the world's reaction to CoVid - suddenly on-line testing came into its own - in ways that may surprise you - if you watch one session have a look at final session of the day about how the national testing system in Scotland is operating and the political pressures around this. It's all about putting the learner at the centre.
My slides below - if you have an interest in following this global community you can join the linkedIn Group.