Flexible learning questionnaire – WELL project

Carried out with learners of Certificate in Reablement Support and MA Integrated Emergency Management (pilot programmes for the WELL project)





This questionnaire (WELL-project-small-student-survey) was designed to help us understand more about the learners’:

1. familiarity with technologies for learning (Questions 1, 3, 4, 5),
2. level of comfort in using technologies for learning, (Questions 2, 6),
3. interest in technologies for learning (Questions 7, 8, 9).
4. preferred location to study (Question 10),
5. preferred time to study (Question 11),

Familiarity with technologies for learning (Question 1, 3, 4, 5)

Certificate in Reablement Support:


Most of the students in the group do not use a computer at university or at home on a daily basis or a few times each week. Usage of mobile phones varies across the group, with none of them using handheld devices and MP3 players.


The learners’ evaluation of their own abilities to use these technologies varies from good to poor.


Learners report some use of discussion forums, but not on a daily or weekly basis. All other forms of communication (blogs, wikis, chat rooms, facebook, etc.) were reported as never or, at best, hardly used. For some learners this was also the case with emails and SMS text messaging.

MA Integrated Emergency Management:


Most of the group report that they have access to computers almost every day or a few times each week. This is also reflected in their usage of mobile phones. With regards to handheld devices and MP3 players, most learners reported to not use them at all.


The learners’ evaluation of their own abilities to use these technologies is generally good.


Learners report some use of discussion forums, but not on a daily or weekly basis. Other forms of communication (wikis, SMS text messaging, news Web pages) were reported as used on a monthly basis or thereabouts. Emails, however, came out on top for this group with 50% of learners using them a few times each week.

Level of comfort in using technologies for learning (Questions 2, 6) and interest in technologies for learning (Questions 7, 8, 9).

Certificate in Reablement Support:


The learners’ feelings with regards to comfort vary from ‘comfortable’ to ‘not at all comfortable’. Most learners report some fear, anxiety, and frustration when using a computer, and others report a certain amount of confidence.


Most learners believe that technologies for learning can be interesting and fun. Also a desire to learn about new technologies is apparent.

MA Integrated Emergency Management:


The majority of the learners have positive reactions towards using technologies for learning. The majority feel ‘very comfortable’ or ‘somewhat comfortable’ and confident using a computer. This group of learners, by and large, see technologies as interesting and useful for the learning experience. They also show some desire to utilise new tools.

Preferred location to study (Question 10) and preferred time to study (Question 11)

Certificate in Reablement Support:


This group appear to do most of their studying at home and at work, between ‘a few times a week’ and ‘a few times a month’. There appears to be no indication of learning at University, a library, or whilst travelling.


This infrequency is also apparent in answer to the question of when they study. Learners report that most of their study time is in the evenings, but this is only ‘between once a month and a few times a month’.

MA Integrated Emergency Management:


This group study more often than the above group, with most study activity carried out at home a few times each week. Some report never to use libraries and to study whilst travelling less than once a month. The same can be said of studying at the University campus.


Studying is carried out throughout the day, with no particular period of the day standing out as more popular. The frequency of ‘a few times each week’ is also reflected in this category.

Final comment


After we gain some insight into learners’ reactions towards using technologies for learning and their familiarity with computer usage, it would be of interest to look at whether these factors, i.e., computer familiarity, comfort with use of technologies, perceived ability to use technologies, anxiety experienced in using technologies, and interest in the use technologies, would have any effect on their performances throughout the durations of the pilot programmes, and achievement in e-assessment. As some learners are quite clearly new to using technologies, and even show anxieties, this can be an ideal context in which to test their use and evaluate their effectiveness, for learners and teachers (a topic of a previous survey and post).

The WELL project evaluation workshops on 3rd Novemer 2009 showed that WBL arena can be quite a tough nut for HE to crack. The workshops thrashed out some of the key issues in relation to

“success” for all stakeholders and what constitutes a “successful” WBL project; and


developing a stakeholder engagement strategy for WBL projects.

Notes made at the events are available to read. They will also be incorporated into a final evaluation report.

On Wednesday 16th December I travelled down to Middlesex University to find out more about their WBL provisions, and particularly how an institution can ‘roll out’ a model of WBL once it has been piloted as part of a project.

I was subject to the wonderful hospitality of Dr. George Dafoulas (MUSKET project), Dr. Barbara Workman Director of CEWBL), and Yongjun Zheng (MUSKET project) from whom I also gained a great deal of information and advice for the WELL project.

The agenda was fairly informal and open, but I posed the following questions over a smorgasbord of Turkish kebabs:


1.    Does the CEWBL serve as an anchor for cultural change across the whole University? Or is it a discrete initiative?


2.    What about the processes of University validation, how have their programmes accounted for this?


3.    What other processes might require changing in an institution hoping to embed cultural change?


4.    In your initial stages, what were your biggest obstacles (especially in dealing with intransigence), if at all?


5.    What about cost effectiveness of WBL provisions? And a provision being ‘malleable’ as well as ‘flexible’ (i.e. it grows with the employers, or downsizes accordingly)?

I was in for quite a listening activity!


I am currently completing a ‘literature review’ of WBL theories, pedagogies, and practices and how to develop a criteria for a WBL model; this is an important output of the WELL project. Some of what I discussed with the Middlesex people will inform this literature review but the following comments I decided to blog as well:

Should WBL courses follow a separate accreditation and validation process to an institution’s already ‘traditional’ procedure for doing the same thing? This is key to the ‘responsiveness’ of the WBL model, as such a process can take months to finalise through the University’s procedure.

At Middlesex the IWBL accreditation committee meet once every two months and WBL courses follow a separate process, one that is far more responsive and streamlined.

If WBL is to be embraced as a discrete, transdiscipliary pedagogy, as a field of study rather than a mode of study, then should this also necessitate such a separate process, in order to be more responsive to the demands of the tripatite relationship in WBL (the institution, the employer, and the learner/employee)? There are, of course, other implicated stakeolders such as LLNs and Sector Skills Councils as well.

What about attitudes towards the currency of WBL courses, and their equivalence and comparability vis-à-vis ‘traditional courses?

Middlesex is aware of this, and one of the objectives of MODNet is to integrate the WBL activities into the already existing School/Faculty processes, converting existing courses to the WBL model. Naturally, there is some compromise in doing so but what was once wholly discrete and successful can ensure better ‘buy in’ than moving from zero to sum, bearing in mind that Middlesex is  17 years in this process and has successful case-studies to show.

WBL is a radical approach to the notion of Higher Education, but isn’t this where HE belongs? As, the development of traditional courses took their own life from an endeavour to create intellectual capital from vocational situations (the legal, medical, and teaching professions etc.).

In any case, such an informal trip to learn and connect with other another institution and a similar project is certainly something that can reveal more qualitative information than glorified case-studies in a conference paper, for example.

Thanks Vicki and Peter for suggesting this, and I am grateful to George, Barbara, and Yongjun for their excellent company (and food!).

As part of the JISC WELL project, we are conducting research into the views of the students on the two pilot programmes, Cert. in Reablement Support and MA Integrated Emergency Management. A questionnaire carried out helps us to understand more about students’:

1. familiarity with technologies for learning (Questions 1, 3, 4, 5),
2. level of comfort in using technologies for learning, (Questions 2, 6),
3. interest in technologies for learning (Questions 7, 8, 9).
4. preferred location to study (Question 10),
5. preferred time to study (Question 11),

Survey Question:

What can we do to  enhance your experiences of ‘flexible’ work-based learning?

Flexible learning here refers to learning which is negotiated between an employer, learner (employee), and the University, and is not confined to the University’s campus and/or academic timetable. This can include distance learning courses and part-time courses.

Rationale:

The WELL project aims to develop a model for flexible work-based delivery that can be used on any academic course. The results of the survey will be used to inform the development of such a model, as we aim to find out from you what your preferences are.

Part 1: Flexible Learning Questionnaire

Part 2: Semi-structured interviews for selected learners.

The results will be posted in due course and incorporated into the next version of the Baseline Survey.

The WELL project has adopted an ongoing approach to evaluation, using a reflective approach that is designed to secure continuous improvement. The process will have a stong emphasis on evaluating the project against its objectives, its operations and its impact on the institution, sector, employers, students and other stakeholders. The WBL arena can be quite tough for HE to engage with: industry, particularly SMEs, different cultures, mutual suspicions, different pressures/ways of working etc.

Part of this evaluation process involves a series of workshops by our External Evaluator, Prof. Peter Chatterton, on:

1. Impact Evaluation (3rd November 2009) This workshop will thrash out the issues of “success” for all stakeholders and what constitutes a “successful” WBL project.
2. Stakeholder Communications & Engagement (3rd November 2009)This workshop will look at developing a stakeholder engagement strategy for WBL projects.
3. Agreeing a Criteria for the WELL Project’s Model (possibly January 2010)
4. Reflection (possibly September 2010) 
5. Review (possibly January 2011)

Further details are available on the JISC WELL website.

This ongoing reflective approach adopted by the project enables lessons to be learned whilst the project is ongoing rather than afterwards, and helps identify practice worthy of further dissemination, allowing judgements to be made on the success of the project ab initio. This is in contrast to a summative “outputs achieved – job done” approach.

The following file is an “Evaluation Map” based on a “why – what – who – when – how” approach to Evaluation:

Evaluation map V1

JISC funded projects favour a collaborative approach, where cross-fertilisation of ideas and sharing of good practice can lead to innovation. Assemblies are one such example of such collaboration. On 8th October we hosted an assembly at the University of Bradford (co-organised by TELSTAR) on the subject of APEL (accredited prior and experiential learning) and how HEIs are tackling this issue. APEL is an important process which formally recognises any prior and/or experiential learning and work experience of lifelong learners, and how this may be formally recognised and credited in an existing HE academic framework.

The event programme and presentations were as follows:

Sarah Malone & Ann Minton (E-APEL, University of Derby)

This workshop provided an overview of the e-APEL tools designed by the University of Derby to introduce learners to the concept of APEL, familiarise them with key academic terminology and support them in the initial stages of preparation of an APEL claim. The session discussed the rationale behind the need for such tools and considers both the technical and practitioner challenges by UK HEIs today.


 

Rob Stilwell & Harriet Dismore (PINEAPPLE, University of Plymouth)

Rob and Harriet provided an overview of this current project’s progress in developing an APEL system that has a specific focus on University staff. They also discussed their project objectives for the coming year.

Beverly Leeds & Garry George Watkins (TELSTAR, University of Central Lancashire)

Beverly provided an overview of this project’s technology enabled Learning Framework which aims to address the needs of work-based learners and their employers. The framework will include the facilitation of the accreditation of learners’ prior experience, credit recognition of in-house and learning provider courses and negotiated credit awarded learning units.





 

George Dafoulas  (MUSKET, Middlesex University)

The MUSKET project is also concerned with the development of an APEL mapping tool. This will support the mapping of an imported XCRI_CAP description (done by the project’s earlier deliverable in the form of a transform tool) to HE and/or FE courses. A variety of mapping relationships are envisaged and the requirements workshops planned during the project will elicit and refine potential relationships. Currently, there is considerable expertise in the IWBL in accreditation services and we expect to build and “implement” this expertise to support “route planning” for learners in specific work places.





 

Vikki Illingworth (University of Bradford) & Ian McGregor Brown (Leeds Trinity University College), WYLLN APEL project

WYLLN (the West Yorkshire Lifelong Learning Network) has funded five projects with a view to developing APEL processes in partner institutions. Each project will trial/test and introduce learner focused APEL processes, documentation and information to improve the progression of vocational learners and work-based adults into higher education.

This survey was offered to teaching staff between July-November 2009. At the time of writing this (August 2009), the survey received 31 respondents, and results will be incorporated into the next version of the Baseline Survey.

Significant results are summarised as follows:

1. 97% of respondents stated they use Blackboard for various purposes including discussions, course notes, formative assessment, and submission of coursework. Most of this use is at undergraduate (25%) and postgraduate (22%) levels, with roughly half of respondents stating that students engaging positively (55% = good student engagement).
2. 33% use PebblePad mainly for CPD, but also for reflective journal writing. Although there is some use across all levels, it is currently unclear where its use is most popular.
3. 45% use Ning or Facebook mainly for synchronous discussion purposes (via Facebook chat), and establishing virtual communities of practice. This appears to be popular with teachers of undergraduate and post graduate students.
4. 32% use audio podcasts mainly for feedback purposes; however, it is currently unclear where its use is most popular.
5. 15% use mobile device for various purposes largely relating to teacher-student communication. As with podcasts, the survey was unable to reveal where its use is predominant.
6. 4% of respondents (n=1) use video conferencing. Notably, this respondent states having extensive teaching experience using such technology in previous teaching roles.
7. 29% and 15% of respondents use web cams and Elluminate respectively; the former is used mainly for Skype calls.
8. For points 5-7 above the survey does not reveal the reactions of the students to the use of each technology; however, half (52%) suggested that students struggled with the technology used and interestingly 19% state that they were unaware of how the students were coping. This lack of knowledge regarding students’ reactions to technology may also explain the large amounts of “not applicable” answers to questions addressing this issue (question #3b-3i and question #8).
9. Most respondents to the survey were form the School of Health Studies and School of Management.

Naturally, such a survey has limited generalisability due to the number of respondents (so far). In order to use such findings to inform and develop a model of flexible learning, we need a series of ongoing investigations that look at both students’ and teachers’ perceptions and needs.

Initially I will focus on gathering current practice, research, review and evaluation of materials with support from the academic schools and the Centre for Academic Practice (often referred to by erstwhile name, TQEG).

In the later stages the emphasis will shift towards working directly with learners and their employers, developing and testing approaches to delivery and support in the workplace. I hope to continually liaise with relevant parts of the University to ensure the emerging model is feasible and can inform development of the University’s systems and processes.

The project will not be developing new technologies, but will draw upon existing learning approaches/technologies and adapt these as appropriate, particularly those from previous JISC projects.

See below for PowerPoint delivered at the SSBR (Support, Synthesis and Benefits Realisation) programme meeting event on ‘Institutional Impact’ that took place 9 July 2009.