Tag Archives: Feedback

Be More Kind⤴

from

A timely earworm for me this week as I am marking philosophy exam scripts. The lyrics are actually about our broken society, but the title of the song speaks to me as I try to decipher scrawly handwriting and make the best sense I can of the jumbled thoughts written under pressure. Education could, and should, be more kind, in my opinion.

This semester has been particularly hard, with the strike action and weather leading to lost teaching time – and the need to be lenient yet fair while marking seems all the more important. This way of assessing students doesn’t seem at all kind to me.

pupil-and-parent-evaluation⤴

from @ Digital Learning

Pupil and parent evaluation

Digital Prototype Evaluation Survey

At the end of the digital prototype pupils and parents were asked to complete an online survey to collect their views.  This report is a summary of the responses.

Helpful?

80% of pupils and parents thought the digital prototype helped with their learning with 25% of pupils and 55% of parents saying by a lot.  15% of parents and pupils saying it didn’t help at all.

Ease of use

Almost all pupils and parents reported they found the device easily to use, and the resources in OneNote easy to find once they were shown how to find them.

Digital Inking and digital resources

Almost all found the digital inking easy to use and very helpful for physics equations, 50% of the pupils still preferred at the end of the prototype to write on normal paper, 25% of them preferred to use the keyboard to enter digital information and 25% use the digital pen to enter the information.

20% of pupils preferred just digital notes, 30% preferred paper notes and 50% stated no preference.

Software & IT

60% of pupils had no issues with their device with 40% having some issues that were almost all fixed. Issues included pen functionality, sometimes it did not write properly and sometimes the buttons did not work.  One pupil had an issue with charging the Surface as the charger was missing.

Wi-Fi issues occurred for some pupils using the Surface in classes other than physics due to weaker wireless connectivity in some areas of the school.  One pupil had an issue with the auto saving features of Word online not saving files properly, this was resolved through discussions with Glow support.  Sometimes the pupils did not notice that the Wi-Fi was not connected and this mean that sometimes new work was not synchronised from OneNote to their device or they needed to renter their Glow username and password after an update. Sometimes Windows updates meant devices were updating rather than available for use.

Almost all pupils and 70% of parents found it helpful to be able to install software on the surface.  In addition to Office 365 software (Word and Excel), some pupils installed Serif Draw for graphic communication, Music Writing Software such as MuseScore, iTunes, Avid Sibelius, Flash card software for revision.

Wi-Fi Access

All pupils and 85% of parents felt it was really helpful to have Wi-Fi access within school.  Their reasons were;

  • Allowed use of IT if the library PC’s were booked.
  • Allowed them to look up information in class
  • Enabled them to look up information instead of asking the teacher.
  • Ability to look at a range of notes from different subjects whilst they were in school.
  • Internet access during study periods.
  • With everything from Physics on OneNote.  It allowed them to access materials within form tutor time.

Pupil comments about Wi-Fi access. 

  • It’s very beneficial to have access to Wi-Fi in school, especially during times where you’re doing homework or revision for a certain class (like in form tutor time). This is because if you’re stuck and your teacher isn’t there to help you, you can try and look up help or find notes online. You could of course go to the department for help if it’s urgent or necessary, but having the internet to help you as an alternative is also a good option. It’s also good for accessing past papers online in school for revision at any times and being able to write digitally on to the past papers.
  • Allowed access to internet notes in my study periods and helped ability for self-study.
  • I could use online resources for homework and classwork as well as making notes (e.g. Quizlet, OneNote online)
  • It got to connect to the internet wherever you were, so if you ever got stuck in a subject and the teacher was busy then you could just look it up.

Some parent comments about Wi-Fi access

  • Seamless learning – also teaches pupil to access responsibly
  • Internet access is required for modern research and learning.
  • Helpful as easy access to internet information at any point when required.
  • Ability to access the information during school hours when e.g. Form time was quiet.
  • To use the Internet to answer a question if the teacher is busy
  • My daughter could go into the SQA website while at school and access past papers without having to go find a computer to use.

Use of Surface

70% of pupils used their Surface at home for a range of school work with 25% using it only for physics, 5% did not use it for any school work at home.  50% of the pupils said they used it at home every day and 30% every other day.  85% of parents reported their children used it most days.

Other subjects

70% of pupils used the Surface in English, 30% in Maths and Chemistry, 20% in Graphics Communications, PE, Geography, French or Design.

30% of pupils reported that no other subjects provided digital resources for their learning.

Uses in other subjects included, digital textbooks such as dynamic learning in Geography, researching information, writing assignments and reports, finding past paper questions, for getting feedback on written or digital work in class to reduce time wasted at home, for translation and Linguascope listening in languages.

Learning

60% of pupils felt they received more feedback in Physics than in other subjects and all stated they read the digital feedback in OneNote.

85% of pupils watched the learning videos embedded in the OneNote and found them helpful for their learning.

80% of pupils used the OneNote resources to catch up on work missed when absent or for revision.

80% of parents were aware of the digital feedback being provided in OneNote.

75% of parents knew their child could use the resources to catch up on missed work.

90% of pupils and 60% of parents felt the digital resources helped them to be more independent and control the pace of learning.

Only 50% of parents were aware that pupils also had been issued with a physics textbook.

Digital Skills

75% of pupils and parents felt digital skills had improved.

75% of pupils and 60% of parents felt they were now more likely to work independently.

Revision

75% of pupils used the Surface for exam revision in all subjects.  20% used it just for physics revision and 5% didn’t use it at all for any subject.  Activities included, BBC Bitesize, Scholar, downloading past paper questions,   A few copied notes for all of their subjects onto OneNote replacing all their paper resources, pupils practiced typing essays for English and History, Quizlet was used for online quizzes.

Pupil comments regarding revision included:

  • Used the past papers on OneNote for Physics revision, with the advantage of being able to write straight onto them. Also accessed the notes on OneNote. For other subjects like chemistry, I mainly just used the surface at school or at home for quick access to past papers. For English, I got from the internet different notes on the novel I had to do an essay on and the Scottish set text poems for revision. The surface was mainly used for physics though.
  • I copied notes for all of my subjects using the Surface and did a lot of past papers for them all which was easier to do on the Surface as I didn’t need to get paper and a pen and instead was just able to us the tablet itself.
  • I used it for history – typing essays and finding information. I also used it for English – typing essays I also used it go on the SQA website to do past papers.
  • I accessed past papers for my other subjects, also accessing Scholar and OneNote
  • I looked through course notes through OneNote which really helped me as everything gave examples and I had Mr Bailey to look through my work and mark it but also to give me feedback.
  • I used OneNote for physics and English, Linguascope for French and Quizlet for biology, English and chemistry.

Parental comments

  • Although surface was used during year for classwork I believe most of revision appeared to be written in note former and taken from a variety of sources.
  • Accessing the past papers, making use of BBC Bitesize, accessing the listening tests for Spanish, accessing internet to help with research on history topics etc.
  • Accessing past papers and BBC Bitesize and Scholar
  • Revising through each page in OneNote for each unit to prepare for the exam

Future recommendation

80% of pupils and parents recommended that future pupils should be issued with digital devices.

Some pupils would prefer a mix of digital and paper resources.  A few parents stated a textbook would have been easier to flick through (they must have been in the 50% unaware that textbooks had been issued).

Some pupils and parents felt they were distracted with the internet and games and this impaired their learning.  One parent believe that the digital device reduced the time spent with the teacher.  One parent commented that whilst helpful to catch up with work, they were not convinced that exam results would improve.  Another felt that once the surface was provided my child lost interest, it was too easy for them to stray off from the work they are meant to be doing and they lose interest.  If the teacher is working with the children there is more chance of them learning and the teacher is more likely to see what stage the child is at, since getting the surface my child lost all interest with Physics.

Pupil comments

Positive comments

  • I do think the devices are good for having the digital notes, doing questions/past papers and independent learning, etc. However, I do think it would also be beneficial in class to still have jotters for class-written notes and any worked examples, while still doing what we did during Higher Physics on OneNote. I personally feel more of a mix between normal learning and digital learning would be best suited for me.
  • The Surface made it much easier to access notes and when it came to assignments it was a lot better with the tablets as we didn’t have to book a library or anything and saved a lot of time making it easier to focus on the schoolwork. The teacher feedback was also very good as the teacher was able to see everything we did on each note so we always received fast feedback
  • The surface is a great device for obtaining feedback on your coursework for Physics from your teacher. You can ask for help at home when on study leave or on school holidays. The surface will help improve your digital skills for the future.
  • Able to work simply at your own pace. Teachers can mark work in OneNote quickly and easily. Easy to access other online learning resources.
  • Extremely easy to access useful learning resources such as OneNote and Scholar, and unlike paperwork, work on the tablet cannot be lost or misplaced
  • Access to digital notes and past papers is extremely useful and using the devices allows pupils to develop more independent learning skills.
  • I believe the surface allowed me to have a better understanding of the coursework and I think future pupils can have the same experience as far as understanding the coursework goes
  • I think the Surfaces where very effective for home learning and revision and allowed me to study more independently
  • Helpful with learning and much faster and more portable than school computers which are atrocious.
  • The tablets are good for continuing were you left of from at home and you were easily able to get the formula sheet when needed and was also able to get any other question sheet easily which was helpful
  • It was far easier to access school work if I was ill or if I was behind everyone else in a lesson, it also helped because it was easy to access and had lots of resources.
  • You get help from your teachers even if you’re at home and this helps a lot and you won’t need to wait till you in class to ask for help. This can speed up people learning and help them to do better in the subjects also its very easy access school resources.

Negative comments

  • I feel they are a useful tool in learning, but they were sometimes a distraction in class.
  • They are broken too easily and will be expensive to fix.
  • They become distracting for people and make it harder to concentrate the best thing about it was being able to access work at home, but people usually get distracted by things like the internet and games.
  • I personally got distracted sometimes during class with the OneNote and would not finish/do the work given.
  • I found using it in general was difficult for me.
  • Don’t do it again people just want the Wi-Fi to connect their phone and not for learning.
  • We didn’t have the source S folder on our tablets, so if we needed something from there we could not do the work on our tablets.

Parental Comments

  • My daughter has always been on top of her class work and always studied independently but the digital device has been a great help as she could see what they have been doing in class easily and catch up on work if she missed a class.
  • It was a very useful tool. It enabled revision of all different subjects when away from home. The Physics resources provided were extremely helpful.  I think it increases engagement more than learning from a textbook.  It allows access when at school to lots of different online tools e.g. Linguascope.
  • It allows access to a great range of resources which otherwise would be missed.
  • Appropriate for use and learning in the modern era. Will provide opportunities to locate information and to test knowledge. Plus the added advantage of increasing IT skills, beneficial to the job market.
  • It was extremely easy to contact the teacher from home and pupils prefer electronic devices rather than sheets of paper
  • Provides equality of access to digital equipment and improves digital literacy
  • Gives pupils a more independent learning approach which is not classroom specific
  • Surely it is the way forward with cost being the limiting factor. If it frees up teacher’s time for other learning activities then it may be easy to justify the cost.
  • Digital learning is the future and I feel being exposed to this early can only help with my child’s learning.
  • Has given access to wider resources to study and also allowed him to do some background preparation for forthcoming lessons in class.
  • I think this is the way forward for preparing pupils for self-learning, on going to college or university.
  • It’s a great piece of kit that my child has found very useful in all areas of their schoolwork.
  • This has been a fabulous resource provided by the school and believe it helped my son to apply himself much more and become much more likely to study in a more fun way

Negative comments

  • They were too much of a distraction in class.
  • I believe that possibly because of the type of learner my son is, he would have been more suited to completing the course in a more traditional manner. I realise of course that this is not the same for all children and there will be I’m sure huge advantages to delivering subjects on digital devices.
  • The kids need to be disciplined in using it. There should be restrictions for using apps.
  • I don’t think these are a good way of teaching, I feel the teacher should be working with them and not letting them do as they please at their own pace on the Surface.
  • The content was found to be harder to access than a text book and jotter which could be flicked through.

Week 2 Classroom and OneNote, Teachmeet⤴

from @ Digital Learning

During this week pupils have been getting more used to how to use the Surfaces, the inking and O365 applications.

I found out this week that there were still a few pupils that hadn’t got their Physics OneNote setup correctly on their Surface.  The initial setup involves going to Classroom, then the OneNote tab, opening the OneNote Online, then clicking edit to open it in OneNote.  This allows OneNote to know the file location, once done this does not need to be repeated although on the Surfaces it was repeated to link make sure both OneNote 2016 and OneNote app had the class notebook.

It was a little surprising to find out that the installation hadn’t been completed after the setup sessions and using it in class to do work.

Pupils are now getting to grips with the Windows inking in OneNote and this is generally working well although there have been an odd event of non working pen and a few forgotten pens.  Many of the non working pen incidents haven’t been due to the pen but because pupils have tried to write in the content library section of OneNote which is read only.  Pupils can only write or type in their OneNote section.

I am becoming more adapt in distributing content in OneNote after a few times where I sent the same pages twice.  This duplication has allowed us to practice how to delete pages from our notebook and many pupils have learnt to rearrange the order of the notebook if they wished.

This week I have started to leave written feedback in the OneNote and marked some digital work.  This worked pretty well although due to my initially setting up the notebook incorrectly with pupils from another class in with one of my classes I have now deleted the additional sections.  Reviewing work was straightforward with the OneNote class notebook add in.  https://www.onenote.com/classnotebook

classnotebookpower-workreview-student-work

Additional frustrations have been encountered this week with 3 pupils turning up to class without their Surface.  About 4 have also turned up without a charged Surface or a charger.  Some have been used my charger and at least there are plugs in the classroom that can be used for this circumstance.

There have been a few cases of having to remind pupils of appropriate use, one for shoe shopping, another for playing music, another for using the narration tools to say silly sentences.

More positively some have been using the Learning tools to read out information or the narrate function to turn speech into text.

The last task of this week was to ensure that pupils know how to hand in work using Classroom.  To check this I set a task today that involved them opening a Word file from classroom then editing it in Word Online writing in answers and then clicking the hand in work section to send the completed work to me.  I can then check and mark the work, sharing an assessment score.  I can hand the work back if it is incomplete with advice.  I can have a conversation or provide feedback to the pupil too.  The hand-in feature also advises if work was not handed in or is late.  I plan to use the hand in feature to check pupils have done the prior learning before class.  The work that can be added can be in Word, OneNote, a FORM quiz or a link.

At this point the Higher classes has a week of work still on their Classroom.  The N5 classes only have one lesson so I need to work to get more of their learning online with the intention that both groups have 2 weeks of work to look at.

At present their has been little flipped learning and this has been deliberate whilst pupils are learning to use the technology and where to access it.  This will now be gradually increased over the next week or so.  I have however had a number of pupils showing me work they had done earlier to prepare for class or to catch up on work missed so am seeing advantages in having resources more readily available to pupils.  In addition the digital learning has better allowed me to push pupils who have completed their work early into the new section or more complex past paper questions.

Early this week I discussed the prototype, OneNote, OneNote learning tools and the use of the Surface with Angus colleague responsible for pupils with Visual impairments.  As a result of the discussion we are going to trial a Surface device with a S3 pupil.  So I will have something to post in the future about accessibility of the Surface for pupils with visual impairments.

Digital Teachmeet

On Thursday I attended the Falkirk digital teachmeet at St. Mungo’s High School, Falkirk.  St. Mungo’s is now a Microsoft Innovative School and the digital teachmeet was excellent CPD.  I shared 5 minutes about the digital inking in the Surface Pro and how my pupils were using them and OneNote.  Other teachers discussed the O365 applications.  Here is a presentation of what I showed live recorded in Office Mix.

 

There was lots of great practice and information to pick up from my fellow Microsoft Innovative Educator Experts.  Not sure which Tweeted picture is the worst!

You can read more about the event on the Falkirk digital TeachMeet page.

If you are interested in digital learning I would recommend that you follow the above on Twitter.  The event was fantastic and I hope to be involved in organising an Angus digital Teach Meet sometime in the future and invite some of those speaking in Falkirk up to share their ideas.

Angus Teachmeet

I am involved in organising with Kellie Smith an Angus Teach meet on Tuesday 29th November after the Angus Learning Festival at Brechin High School from 4.30 to 6.00 pm.  Please come along to listen or do a 2 minute presentation to share some ideas. Ideas can be any good ideas or practice, not just digital learning and technologies, although my presentation will be on that theme.

Sign up using this link https://forms.office.com/Pages/ResponsePage.aspx?id=oyzTzM4Wj0KVQTctawUZKWdhPNwe8ftPowdGcOI9yIZUOUIyUElPQk9ENzRRT1lLWEpNUDYzWEtXMS4u

 

Feedback and more with Forms⤴

from @ ICT for Teaching & Learning in Falkirk Primary Schools

FormsGathering feedback, taking quizzes to reinforce learning, undertaking surveys of views, signing up or registering for an activity – just some of the ways forms can be used by schools. And now there is the option to use Microsoft Forms – available as a free online tool which uses a Microsoft Office 365 account (available to all Glow users) to set up the form either by going to https://forms.office.com or, if already logged into Office 365, via the Forms tile in the office 365 navigation tiles waffle.  Office Forms can be created by either learners or educators.

Forms work nicely on any smartphones, tablets or PCs. Setting up requires the creator to be logged in to Office 365 but those completing the created form can be completed by anyone without requiring any kind of logging in (if that setting is chosen by the form creator), or they can be anonymous (if that is the setting the creator of the form wishes to use), or if they wish to restrict responses to their class and to ensure their identity they can use the login details of office 365 users too (if that’s how the creator of the form wishes the form to be completed). So the form creator gets the choice to suit the purpose and audience of their form.

Feedback is immediate, real-time, to the form creator and the results can be displayed in different ways to suit the need of the form creator.

For Sway users you can embed a form created with office Forms live in a Sway presentation information can be shared about a topic being studied and a quiz included alongside the content.

Creating your form

  1. Office365waffleEither go to https://forms.office.com and log in with your Office 365 account (for Scottish schools that will be your Glow account) or, if already logged into Office 365, choose the Forms tile in the office 365 navigation tiles waffle.
  2. Click on + New to start creating your new form (you can click on the title of any previously created form in order to edit that, and if you wish to base a new form on an existing form you can click on the … ellipsis to the right of the form title and choose copy – then you can edit the copy to create a new version.
  3. addformJust click on “Untitled form” to edit the name of your form, and click on “Enter a description” to add explanatory text as you may wish to include to explain the purpose of the form and perhaps mentioning the intended audience. Then click “+ Add question
  4. questiontypesChoose the type of question.There are five types of answer formats:
    • multiple choice questions (where you can choose to accept only one answer or multiple responses)
    • free-text (and you can choose either short or long text)
    • ratings (you can choose number or star rating)
    • quiz-questions (where you can provide immediate feedback to anyone filling in the form as to whether the respondent gave the correct answer or not (click on the tick icon to indicate which answer would be the correct answer – and just click on the speech-bubble icon to add comments to any response choice, which may give encouraging comments or suggestions for what to do next in response to the answer given, or any kind of feedback you wish to display when a particular choice is chosen)
    • date-input
  5. You can choose whether there can be multiple responses or only one answer accepted, you can require that specific questions have to be answered before a user can complete the form, and by clicking on the  …ellipsis you can choose whether a subtitle (which could provide explanatory text for each question) is displayed, and whether you wish to shuffle the order of questions so that each time someone sees the form the questions are displayed in a random order.
  6. Add as many further questions as you wish. You can re-order the questions by clicking on the upward or downward facing arrows above each question, and you can copy an existing question (and edit that copy), or delete an existing question.

Previewing your form

mobilepreviewformTo see what the form will look like for people about to fill it in you can click on “preview” at the top navigation bar. You can see how the questions will be laid out on a computer, and you can also choose to see how it will look on a mobile device.

Sharing your form

Once the form is complete click on “Send form” – this will open a side panel with various choices. It will provide a link to share with those you wish to respond to the form. It will create a QR code for quick scanning by users using a mobile device, and it will provide html embed code if you wish to embed the form within a website page or blogpost. This screen also gives you the option to choose who will be able to fill out the form – you can choose only people within your organisation (for Scottish schools using Glow that would be Glow users only), and within that you can choose whether or not to record the names of those responding in the results, or you can choose to make the form available to anyone with the link (where no sign-in will be required for people responding to the form).

If you click on “See all settings” at the foot of this side panel you will get further choices:

Looking at the results of your form

Responsesscreen

When you wish to look at the responses to a form you have shared then simply open the form and click on the responses tab along the top of the screen. You will get an overview of the number of respondents, the average time taken to complete by respondents, and whether the form is still active or expired 9if you’d set it to have a deadline). There is also the option to download to a Microsoft Excel spreadsheet (which comes complete with auto-filter drop-downs to easily sort the information generated to suit your needs).

Example forms

FormLearningHow did you get on with your learning this week? – this form is a mock form just to show how a form might be used for a teacher to get feedback from learners in their class to better support them. This example is based on the form created by Fiona Johnson, headteacher at Kilmartin Primary School in Argyll and Bute, but this link is purely an example so anyone can try it. Similarly here is another mock form (also based on the form created by Fiona Johnson as headteacher at Kilmartin Primary School in Argyll and Bute) – “How did you get on with your learning today?” – feel free to give it a try.

So what have people said about Office Forms?

StevenPayneFormsSteven Payne, an educator in Western Australia, shared the results of a mock use Microsoft Forms – showing the results, and the way in which they can be displayed, which the creator of the form can see once respondents have completed the survey.

Jim Federico commented in a tweet that Microsoft Forms being built into Office 365 for Education means no add-ins are required, and includes question types which auto-grade.

TestingWithOfficeFormsKurt Söser, an educator in Austria, has provided a step-by-step guide to his experience setting up a quiz with Microsoft Forms and using it with his learners.

VicentGadeaFormsVicent Gadea, an educator in Spain, described co-assessment using Microsoft Forms “1st time was complicated then was very powerful for us.”

Zelfstudforms

Koen Timmers, an educator in Belgium, has described in a step-by-step guide, illustrated with screenshots, how to set up a form using Office Forms, and shared what the responses look like for a form he created.

Making use of Forms in the classroom

There is a range of online form tools available, each of which can generally be used in similar ways, so it can be helpful to look at how others have used these tools when thinking about how online forms can support classroom activity.

DavidAndradeFormsChad Raid wrote about the use of forms on David Andrade’s Educational Technology Guy blog – some of which may be applicable in different educational scenarios. Obviously in any use of forms the issue of data security is paramount and guidance from school or local education  authority as to what can, and what must not, be requested via a form would clearly be essential.

 

#100wordTandL Irresistible Feedbac⤴

from @ Pedagoo.org

Research says a fraction of our feedback to students has impact on learning. Knowing this ought to make us look up from our marking labours and try to work out where we might be wasting time. Using a ‘feedback wall’ is immediate and irresistible. Set a challenging discussion-based activity for groups. While they talk, you […]

The power of the red pen!⤴

from @ Pedagoo.org

As a teacher I value pupil voice and understand the importance of quality feedback which needs to be more of a conversation than a statement. In practice though it can be difficult to achieve this without it becoming unmanageable. One change to my teaching practice this week has really made a difference to the quality […]

QAA Scotland Focus On Assessment and Feedback Workshop⤴

from @ Sharing and learning

facebooktwittergoogle_plusredditpinterestlinkedinmail

Today was spent at a QAA Scotland event which aimed to identify and share good practice in assessment and feedback, and to gather suggestions for feeding in to a policy summit for senior institutional managers that will be held on 14 May.  I’ve never had much to do with technology for assessment, though I’ve worked with good specialists in that area, and so this was a useful event for catching up with what is going on.

"True Humility" by George du Maurier, originally published in Punch, 9 November 1895. (Via Wikipedia, click image for details)
“True Humility” by George du Maurier, originally published in Punch, 9 November 1895. (Via Wikipedia)

The first presentation was from Gill Ferrell on electronic management of assessment. She started by summarising the JISC assessment and feedback programmes of 2011-2014. An initial baseline survey for this programme had identified practice that could at best be described as “excellent in parts” but with causes for concern in other areas. There were wide variations in practice for no clear reason, programmes in which assessment was fragmentary rather than building a coherent picture of a student’s capabilities and progress, there not much evidence of formative assessment, not much student involvement in deciding how assessment was carried out, assessments that did not reflect how people would work after they graduate, policies that were more about procedures than educational aims and so on.  Gill identified some of the excellent parts that had served as staring points for the programme–for example the REAP project from CAPLE formerly at Strathclyde University–and she explained how the programme proceeded from there with ideas such as: projects agreeing on basic principles of what they were trying to do (the challenge was to do this in such a way that allowed for scope to change and improve practice); projects involving students in setting learning objectives; encouraging discussion around feedback; changing the timing of assessment to avoid over-compartmentalized learning; shifting from summative for formative assessment and making assessment ipsative, i.e. focussing on comparing with the students past performance to show what each individual was learning.

A lifecycle model for assessment from Manchester Metropolitan helped locate some of the points where progress can be made.

Assessment lifecycle developed at Manchest Metropolitan University. Source: Open course on Assessment in HE.
Assessment lifecycle developed at Manchester Metropolitan University. Source: Open course on Assessment in HE.

Steps 5, “marking and production of feedback” and 8 “Reflecting” were those were most help seemed to be needed (Gill has a blog post with more details).

The challenges  were all pedagogic rather than technical; there was a clear message from the programme that the electronic management of assessment and feedback was effective and efficient.  So, Jisc started scoping work on the Electronic Management of Assessment. A second baseline review in Aug 2014 showed trends in the use of technology that have also been seen in similar surveys by the Heads of eLearning Forum: eSubmission (e.g. use of TurnItIn) is the most embedded use of technology in managing assessment, followed by some use of technology for feedback. Marking and exams were the areas where least was happening. The main pain points were around systems integration: systems were found to be inflexible, many were based around US assumptions of assessment practice and processes, and assessment systems, VLEs and student record systems often just didn’t talk to each other. Staff resistance to use of technology for assessment was also reported to be a problem; students were felt to be much more accepting. There was something of an urban myth that QAA wouldn’t permit certain practices, which enshrined policy and existing procedure so that innovation happened “in the gaps between policy”.

The problems Gill identified all sounded quite familiar to me, particularly the fragmentary practice and lack of systems integration. What surprised most was the little uptake of computer marked assessments and computer set exams. My background is in mathematical sciences, so I’ve seen innovative (i.e. going beyond MCQs) computer marked assessments since about 1995 (see SToMP and CALM). I know it’s not appropriate for all subjects, but I was surprised it’s not used more where it is appropriate (more on that later). On computer set exams, it’s now nearly 10 years since school pupils first sat online exams, so why is HE so far behind?

We then split into parallel sessions for some short case-study style presentations. I heard from:

Katrin Uhilg and Anna Rolinska form the University of Glasgow about the use of wikis (or other collaborative authoring environments such as Google Docs) for learning oriented assessment in translations. The tutor sets a text to be translated, students work in  groups on this, but can see and provide feedback on each other’s work. They need to make informed decisions about how to provide and how to respond to feedback. I wish there had been more time to go into some of the practicalities around this.

Jane Guiller of Glasgow Caledonian had students creating interactive learning resources using Xerte. They provide support for the use of Xerte and for issues such as copyright. These were peer assessed using a rubric. Students really appreciate demonstrating a deep understanding of a topic by creating something that is different to an essay. The approach also builds and demonstrates the students digital literacy skills. There was a mention at the end that the resources created are released as OERs.

Lucy Golden and Shona Robertson of the University of Dundee spoke about using on wordpress blogs in a distance learning course on teaching in FE. Learners were encouraged to keep a reflective blog on their progress; Lucy and Shona described how they encouraged (OK, required) the keeping of this blog through a five-step induction, and how they and the students provided feedback. These are challenges that I can relate to from  asking students on one of my own course to keep a reflective blog.

Jamie McDermott and Lori Stevenson of Glasgow Caledonian University presented on using rubrics in Grademark (on TurnItIn). The suggestion came from their learning technologist John Smith, who clearly deserves a bonus, who pointed out that they had access to this facility that would speed up marking and the provision of feedback and would help clarify the criteria for various grades. After Jamie used Grademark Rubrics successfully in one module they have been implemented across a programme. Lori described the thoroughness with which they had been developed, with drafting, feedback from other staff, feedback from students and reflection. A lot of effort, but all with collateral benefits of better coherency across the programme and better understanding  by the students of what was required of them

Each one of these four case studies contained something that I hope to use with my students.

The final plenary was Sally Jordan who teaches physics at the Open University talking about computer marked assessment. Sally demonstrated some of the features of the OU’s assessment system, for example the use of a computer algebra system to make sure that mathematically equivalent answers were marked appropriately (e.g. y  = (x +2)/2 and y = x/2 + 1 may both be correct). Also the use of text analysis to mark short textual answers, allowing for “it decreases” to be marked as partially right and “it halves” to be marked as fully correct when the model answer is “it decreases by 50%”.  This isn’t simple key word matching: you have to be able to distinguish between “kinetic energy converts to potential energy” and “potential energy converts to kinetic  energy” as right and entirely wrong, even though they have the same words in them. These are useful for testing a student’s conceptual understanding of physics, and can be placed “close to the learning activity” so that they provide feedback at the right time.

Here was the innovative automatic marking I had expected to be commonly used for appropriate subjects. But Sally also said that an analysis of computer marked assessments in Moodle showed that 75% of the questions were plain old multiple choice questions, and probably much as 90% were some variety of selection response question. These lack authenticity (no patient ever says “Doctor, I’ve got one of the following four things wrong with me…”)  and can be badly set so as to be guessable without previous knowledge. So why? Well, Sally had made clear that the OU is exceptional: huge numbers of students learning at a distance mean that there are fewer more cost effective options for marking and providing feedback,  even when a large amount of effort is required. The numbers of students also allowed for piloting of questions and the use of assessment analytics to sort out the most useful questions and feedback. For the rest of us, Sally suggested we could do two things:
A) run moocs, with peer marking and use machine learning to infer the rules for marking automatically, or
B) talk to each other. Share the load of developing questions, share the questions (make them editable for different contexts).

So, although I haven’t worked much in assessment, I ended up feeling on familiar ground, with an argument being made for one form of Open Education or another.

facebooktwittergoogle_plusredditpinterestlinkedinmail

Lawthorn primary school⤴

from @ Glow Gallery

We visited Lawthorn primary school to find out how they have been using Glow

Listen to teacher, Mr English, sharing what he has been doing and which of the Glow services he has been finding useful.

And now we have some pupils talking about what Glow has meant for them