The following is a draft for consultation and will be presented at the next AMC meeting. If you have any comments or observations, please contact Andy Ramsden (a.ramsden@ucs.ac.uk)
- Andy Ramsden (e-Learning Development Manager, UCS)
- 7th July, 2014
Acknowledgements
The author would like to thank the following people for their active role in the Task and Finish Group. Without their input it would have been a longer, harder journey.- Aaron Burrell
- Faith Hicks
- Matt Hirst
Executive Summary
The aim of this review was to answer the following questions, how can we enhance the workflow of EMA at UCS? How can we increase adoption of good practice across the institution?The discussion highlighted EMA is viewed positively by all stakeholders. There are a number of recommendations to enhance the effective implementation across UCS. These recommendations have been appropriately resourced and prioritised.
Background
The Review of e-Submission, e-Grading and e-Return paper was presented to the AMC meeting in March 2014. The paper proposed given there has been a gradual roll-out and take up of electronic processes available to support online submission, grading and return across UCS, and the last review was undertaken in 2011/12, it would be appropriate and timely to undertake another review.The aim of this review was to answer the following questions, how can we enhance the workflow of EMA at UCS? How can we increase adoption of good practice?
Methodology
The Task & Finish Group - comprised of:- Aaron Burrell - Elevate Team
- Faith Hicks - Academic Services
- Matt Hirst - Academic Services
- Andy Ramsden (Chair) - Elevate Team
- Student (n=1) Semi structured interview and desk study from Course Reports
- Lecturers (n=17 (2 from Otley)) - semi structured interviews (1-2-1)
- Course Admins (n=4) - focus group
- Health & Safety (n=1) - email correspondence
- IT Services Helpdesk (n=1) - email correspondence
- Infozone Manager (n=1) - semi structured interview
- Total (n=25)
The Heads of Department were asked to recommend staff to participate.
The data (interviews, focus groups and email correspondence was undertaken during May & June 2014.
As would be expected from this approach, a number of caveats exist:
How can we improve the workflow?
Some people were not aware of the workflow. However, the principle of being explicit about roles and responsibilities was positively received across the stakeholders.
The latest iteration of the workflow is available below.
Some of the suggested enhancements were:
How can we support the implementation of the workflow?
A consistent message emerged from all the stakeholders around needing clarity of “who is responsible for managing portals?” This implies the current workflow has not embedded across all the course teams, and we need an effective marketing and communication strategy aimed at all stakeholders.
Hardware
A common thread throughout the staff discussions were issues around the required hardware and software to complete the e-grading and e-return. Staff did not find the current arrangement of workstations with small monitors in open planned offices conducive to the process. In addition, a number of concerns were raised around health issues associated with spending long periods of time marking online, and the frustrations of excessive scrolling due to small monitors.
Consistency
Lecturers
MUST
The data (interviews, focus groups and email correspondence was undertaken during May & June 2014.
As would be expected from this approach, a number of caveats exist:
- Small sample size means it is not a significant sample
- Sample bias as the Task & Finish Group selected the people to interview
- Interview bias as “they may say what they think we want to here, not what they actually think”
Discussion
The original workflow has been implemented in a variety of models at UCS. The analysis identified a lot of positive feedback for benefits of EMA across all stakeholders at UCS. The Students commented about the ease of submission, and no printing costs. While Course Admins identified efficiency gains and easier monitoring, and Lecturers discussed the benefits of streamlining the marking and feedback cycle.How can we improve the workflow?
Some people were not aware of the workflow. However, the principle of being explicit about roles and responsibilities was positively received across the stakeholders.
The latest iteration of the workflow is available below.
Some of the suggested enhancements were:
- ensure a meetings are undertaken between the course team and the course administrator to agree who is responsible for the various parts.
- clear up misunderstandings between ratified and unratified grades for students
- include external examiners within the process, and ensure they are supported
- re-design the workflow to accommodate the enhancements to the LearnUCS software.
How can we support the implementation of the workflow?
A consistent message emerged from all the stakeholders around needing clarity of “who is responsible for managing portals?” This implies the current workflow has not embedded across all the course teams, and we need an effective marketing and communication strategy aimed at all stakeholders.
A common thread throughout the staff discussions were issues around the required hardware and software to complete the e-grading and e-return. Staff did not find the current arrangement of workstations with small monitors in open planned offices conducive to the process. In addition, a number of concerns were raised around health issues associated with spending long periods of time marking online, and the frustrations of excessive scrolling due to small monitors.
Consistency
Lecturers
- A few issues raised the need for consistency across Departmental implementations. There existed significant differences in practice.
- A question asked by some was, "what does good practice look like? What should I be aiming towards?"
- They acknowledged a variety of approaches were adopted by different course administrators. They saw an opportunity to learn from each other, and move to a more consistent implementation.
- More consistency across UCS with respect to the management of extensions
- More effective and efficient support for External Examiners
- More support sessions for academic staff around inline grading (e-grading, e-feedback)
- Clearer guidance on who to contact with respect to availability or issues with the submission portals.
- Earlier communication if there is an issue with the format of the submitted file.
- Encourage consistency within course teams on when and how EMA is used
- Simplify the need to manage all the other files (cover sheets, marking grids) which need to be submitted with the assignment
- The Course Admins discussed if there existed opportunities to integrate the Student Record System (SITS) with LearnUCS. The hope was for two integration. For instance, the automatic creation of online submission portals based on assessment diets from SITS, and the push of appropriate and selected data from LearnUCS gradebook into SITS.
Recommendations
The recommendations have been organised through a MoSCoW analysis.MUST
- Creation of a dedicated EMA support area for Staff and Students to support and inform them of the process. This will be linked to from each LearnUCS course, and the UCS Assignment Toolkit [Owner: Elevate (Aaron) - By: July 2014]
- Redesign the submission portal to improve access to FAQs, links to academic skills support, clearly identify people / roles who support the process, and make this support material multimedia rich with screencasts [Owner: Elevate (Aaron) & Acad Services (Matt) - By: July 2014]
- Provide student focussed posters at key times of the year within student learning spaces which focus on how to submit work and access feedback. [Owner: Elevate (Aaron) & Acad Services (Matt) - By: December Campaign]
- Provide awareness raising posters (lecturer audience) at key times of the year, distributed within staff spaces and offices which focus on the EMA workflow, the benefits, where material resides and contact names [Owner: Elevate (Aaron) & Acad Services (Matt) - By: December Campaign]
- Improve the communications with new lecturers as part of their induction. Provide a clear and consistent message around talking to their course administrator about the process. [Owner: Elevate (Andy) & Acad Services (Matt) - By: July & Ongoing]
- Run training sessions twice a year for course admins to ensure they have collective ownership the emerging good practice around EMA. Create regular bulletins. [Owner: Elevate (Aaron) & Acad Services (Matt) - By: July 2014 & December 2014]
- Develop online guidance for lecturers on the common issues and questions. Ensure this is easily accessible for lecturers in LearnUCS when they are marking. [Owner: Elevate (Aaron) & Acad Services (Matt) - By: July 2014]
- Be more explicit about the available support and advice on Health and Safety. Include in Online Support Material [Owner: Elevate (Andy) - By: October 2014]
- Group meet to review implementation plan, and write a new short term plan for Jan to June 2015 [Owner: Elevate (Andy) - By: December 2014]
- Dissemination strategy, including Meetings (AMC, LTAG, Faculty), MyUCS Announcements [Owner: Elevate (Andy) - By: Sept - Nov 2014]
- Look at opportunities to create e-grading and e-feedback dedicated workstations or rooms. These would include machines with large monitors, and meet the minimum browser requirements. [Owner: Elevate (Andy) - By: ongoing]
- Train all course administrators on the functionality of the inline grading tool, and collectively author FAQs from the lecturer and student perspective [Owner: Elevate (Aaron) & Acad Services (Matt) - By: October 2014]
- Explore file conversion software for course admins [Owner: Elevate (Aaron) & IT Services - By: October 2014]
- Blog post to engage in the wider UCS discussions around browsers, in particular issues around the performance of LearnUCS in Internet Explorer [Owner: Elevate (Aaron) - By: October 2014]
- Explore the current situation with respect to SITS and LearnUCS integration [Owner: Elevate (Aaron) - By: October 2014]
WON’T
- Pay for Blackboard bespoke developments for SITS integration, and Email Receipts
Image Source: http://www.azproarte.com/wp-content/uploads/2012/05/Online-submission.jpg
No comments:
Post a Comment