AS079 Collaborative E-Assessment


Background

There is a need to ensure the continued quality of e-assessment practices across the VET sector. This is highlighted in the recent E-assessment guidelines for the VET sector. E-assessment continues to be a priority for the VET sector.
E-assessment can be defined as the use of information technology for any assessment-related activity. Currently over forty per cent of Registered Training Organisations (RTOs) and more than sixty per cent of teachers and trainers are using some form of e-assessment (Australian Flexible Learning Framework 2010). It appears that e-assessment is finding its place in the VET sector. However, there is still much work to be done, with the most prevalent e-assessment model being the online quiz (Callan and Clayton 2009).

As the take up of online delivery and assessment gathers momentum, the challenges of authentication and validity of online assessment tasks are becoming more apparent. As a consequence RTO�s tend to relegate e-assessment to the low stakes, formative assessment methods. The current RTO audit process is yet to fully recognise summative online assessment as an alternate to more traditional assessment methods.

This project will explore the possibilities presented by technologies in the use of peer assessment in both formative and summative assessment contexts. Peer assessment can be described as assessment of a learner by fellow learners typically following the same programme of study. Peer assessment encourages the learner to make independent judgements of their own and others' work.

  • Lead organisation: ASC Training & Development
  • Partner(s): Efficacy Consulting

Project Summary

How will the project improve learner access to improve employment outcomes?
Target Group
Our project targets leaders in the Pharmacy Industry, both Pharmacists and Retail Managers.

Learning Approach
A pre course survey will establish each participant's knowledge and current skill level in
(a) Managing team processes
(b) Managing conflict
(c) Building effective teams
(d) Engaging in online learning

Two half-day workshops held 3 weeks apart will give participants the opportunity to work on group activities and develop a collaborative relationship.
Online tasks utilising Moodle 2.1 as our LMS will form part of the learning, including discussion forums, article review and short activities.

A project based on a team improvement is to be completed by all participants and a report provided online. Learning Teams will be formed, and each team member will assess the other members in their team based on criteria developed by the whole group (and guided by the Evidence Guide and Required Skills and Knowledge statements from the competency standard).
Projects will also be assessed by the course facilitators and a representative from the Pharmacy Guild (a) before Peer Assessment is conducted and (b) after the Peer Assessment is completed and participants have had the opportunity to address any gaps raised by their peer assessors.

A final review will bring participants together (or possibly as an online survey) to debrief the project and determine the learning that occurred.

Area of Study
The learning area is Teambuilding and is based on the nationally recognised unit of competency: BSBWOR402A Promote team effectiveness.
Participants will demonstrate improved skill and knowledge in
  • Team development theory
  • Team planning and decision making
  • Building productive relationships and managing conflict
  • Workplace communication processes
  • Identifying and resolving barriers to good performance

In addition to learning outcomes related to the unit of competency, participants will also develop skill and knowledge in online learning and assessment.

How does this project align to the jurisdictional and national priorities?


The Pharmacy Guild (SA) has identified leadership, supervisory and mentoring skills as areas for development in Retail Pharmacy. The Pharmacy Guild (SA) is very supportive of this project as the unit of competency to be delivered (Promote team effectiveness) concerns team development.

Consultation with stakeholders

In providing a letter of support for this project, Meredith Dufty, State Training Manager, Pharmacy Guild (SA) notes:The opportunity for Pharmacists and their management teams to have access to a form of e-learning in a topic that is relevant to them and delivered by a Nationally Recognised Training Organisation, would see an increase of skills and knowledge.
Meredith Dufty, State Training Manager, Pharmacy Guild (SA)

Tony Stone (Project Manager, ASC Training & Development) and Peter Oborn (Facilitator) met with Meredith Dufty and Michael Robertson (Branch Director, Pharmacy Guild) on 6/12/2011 to discuss strategies to identify up to 15 pharmacists and retail managers to engage in the project.

Intended outcomes from your project

What do you hope to achieve with the funding?

This project will show the ways that e-assessment can be used that follows good assessment practices, meets the rules of evidence and achieves excellent outcomes.

This project will achieve the following outcomes:

  • Evaluation of an online assessment methodology that explores peer assessment as a valid summative assessment process
  • Enhanced employability of participants in team leadership roles
  • Development of skills in participants and RTO staff in online learning and assessment methodologies

Information session held for Pharmacy Guild members on Tuesday 7 February




Breakdown of Major Assessment Task and role of 'Peer Assessors'


Actual Outcomes resulting from this project


Progress report as at 15/06/2012
1. Partnership developed with Pharmacy Guild (SA) to explore use of online approaches to enhance leadership capability in the sector.

2. Professional development for ASC T&D facilitators in developing and facilitating a nationally recognised unit of competency through a blended learning approach (face to face, online, phone, email, teleconference)..

3. Enhanced awareness, skill and knowledge in a pilot group of Pharmacists and Retail Managers with respect to learning in a blended environment (face to face, online, phone, email, teleconference).

4. Implementation of peer assessment activity in Moodle (Moodle Workshop)
(i) Participants have conducted formative assessments on their peers. This involved reviewing peer Action Plans for their Team Improvement Projects against assessment criteria based on competency standards for the unit BSBWOR402A Promote team effectiveness.
(ii) 50% of participants have completed and uploaded their Final Reports based on the outcomes of their Team Improvement Projects. Of these, 2 participants have had their Final Report assessed (each by 2 peers); the others have been assessed by at one peer to date.



5. Evaluation of peer assessment decisions to date completed by qualified assessor panel

Benefits experienced by


Lead agent
1. This project has resulted in a concerted focus on blended learning within the RTO. Facilitators have experienced first hand the challenges faced by participants in navigating the online environment and have benefitted from trialling support strategies to keep participants engaged.

2. The project has given us a clear sense of the level of commitment required from learner and facilitator to ensure this approach to learning and assessment is successful.

3. We have a better sense of ways to streamline activities and instructions to allow participants to progress with confidence in the online environment.

4. We have road tested Moodle Workshop and are in a position to consider ways to use this application in other courses.

5. We have explored the implications of peers making judgements on each others work, and have already seen how this process can be used as an important learning tool.

Partner(s)
For our IT partner, Melanie Worrall of Efficacy Consulting, this project provided the opportunity to apply Moodle tools to a real learning task, involving a typical cross section of workers in leadership roles. It was the first time any of the stakeholders in this project had explored the potential of Moodle Workshop. It also provided insights for Melanie, an instructional designer, in the benefits of peer assessment in the learning process.

Learners
The benefits to learners to date include:

1. Enhanced computer skills

2. Engaging successfully in online learning and participating in the peer assessment process
3. Seeing marked improvement in team performance back at work as a result of the Team Improvement Project

4. Participating in forums, especially the sharing of information and documents

5. Being able to initiate a forum discussion

Reflections and Suggestions


Overall, participants managed to navigate the Moodle LMS with relative ease. The majority of participants engaged successfully in discussion forums, document uploading and interaction with the Moodle Workshop activity. This activity was central to the project as it was the vehicle for participants to submit work, have this work assessed by peers and also assess their peers.

Feedback from the group has been positive in terms of learning outcomes achieved through the Major Team Improvement Project. We believe the use of the online peer assessment tool, whilst a secondary consideration to participants, has assisted each participant in better understanding the assessment criteria in the unit of study and consequently a heightened understanding of how to complete their own submissions. This in turn should lead to improved learning outcomes.

From the peer assessments completed to date, there was a fair degree of consistency in marking between individuals; a potential failure in the process is the assessor marking everything off as 'Requirements met' without close scrutiny of the evidence, however we did not see any evidence that inappropriate or unreliable marking had occurred in the small sample to date. The final check by qualified assessors provided the safety net to ensure a valid assessment decision was made in each case.

Time pressures meant we had to adjust and modify the project so participants would not feel overwhelmed. While this did not detract from the overall objectives of the project, we lost the opportunity to enagage learners in some activities that would have aided in their understanding of the assessment process. For example, our initial intention was to have learners help develop assessment criteria based on the unit of competency being studied. As the participant group did not become finalised until late February, we opted to save time by providing the assessment criteria.

Significant facilitator time was spent in one-one support and coaching. In future, we would ensure participants were well practised in using the online tools employed before having them complete any assessment on peers. This would be much easier to achieve with a learner group enrolled (e.g.) in a whole qualification, where there would be ongoing engagement for 12 months or so. This would mean we could drip feed online tools in a more gradual way, and reinforce use throughout the period of study.

In using peer assessment throughout a qualification, it would be more efficient to integrate assessment over 2 - 3 units of competency at a time.

Sustainability of project outcomes


Designing an integrated approach with assessment covering a number of units of competency would aid in making this a more cost effective process and thus more sustainable.
We believe the process is quite sustainable once participants (and facilitators) become comfortable in using the online tools required for peer assessment to be undertaken.

How will you support the transfer of learning from the project to other organisatons, through their own efforts and in cooperation with the business manager and the FLAG Secretariat.

Participation in conferences, presentations; making available templates used in the peer assessment process; posting findings on our website and through the Access to Skills project team

Acknowledgement

This is a an Access to Skills project output, developed by ASC Training & Development and Efficacy Consulting with seed funding from the National VET e-learning strategy.

For more information

For more information on ASC Training & Development
Tony Stone
Managing Director
tony@asctraining.com.au

(08) 8410 2627

For more information on Efficacy Consulting
Melanie Worrall
Senior Consultant
melanie@efficacyconsulting.com.au

(08) 8207 8543

For more information on the Access to Skills program:
Phone: (08) 9229 5200
Email: accesstoskills@dtwd.wa.gov.au
Website: www.flexiblelearning.net.au