Update

I’m currently focusing on getting all the project documentation finalised, which will be ready by the end of March.

Report from TAG Seminar

I recently attended this seminar which is a good example of wider benefits realisation, and clearly demonstrates the success of wider engagement.

TAG HEA Evidence-based Practice Seminar – March 4, 2010

The seminar was funded under the HEA Seminar series[1] and considered issues of student retention and motivating student learning in relation to the TAG[2] Project at the University of Central Lancashire. 

The seminar was attended by around 15 people including interested participants from University of Cumbria, Manchester Metropolitan University, The Open University, Salford University and University of Central Lancashire.

Richard Brawn, senior advisor at the HEA, provided a welcome, highlighted the other forthcoming seminars running from Feb to June 2010 around Assessment and Feedback, Employability and Employee Learning,  and Professional Learning and Teaching Practice. 

Personalisation of the Learner Experience

John O’Donoghue set the scene with a presentation on Personalisation of the Learner Experience (web ref to podcast). He focused on how personalising the environment for learners can improve retention through motivating learners. The presentation explored the tools and techniques required to change institutional culture and strategies to encourage a change towards personalisation of the learning experience.  He explored different motivations for the personalisation of learning from government policy to pedagogical benefits. A reference provided from Chris Yapp (2006) stating that learning is a personal experience; and the technology is used to achieve this at scale provided a useful reminder of  how Higher Education has been adjusting the increase in student numbers over the past 20+ years and technology has been used in an attempt to maintain the personal learning experience.

John’s presentation considered the different factors that needed to be considered, balancing faculty, teacher and student centred requirements. He highlighted the need for changes to technology, the organisation and culture (people), and recognised that technology can be the easiest of these to implement.

Technology supported personalised learning can provide a shift in the learning paradigm from institutional focussed to learner focussed, removing the boundaries of traditional learning or maybe returning us to a more personal learning experience that can be delivered to a larger cohort.

The seminar discussed the need to change culture to adopt new processes and systems, for example introducing technologies.

 The Alternative Guide to UCLAN

John O’Donoghue was followed by a presentation and demonstration of The Alternative Guide to UCLAN by Lucy Warman, exploring retention issues in the institution. TAG focussed on the student application process, through induction and the first 6 months at University.

TAG identified several issues that relate to supporting learners –

  • Manage life
  • Learn to learn
  • Find the way
  • Making friends

 SLOs (student liaison officers) at UCLAN  undertook an initial review and concluded that University web sites were marketing related and provided poor information for supporting students coming to UCLAN.

 So TAG was developed to support student life at University. The site involves students producing content for TAG and sharing their experiences,  for example  videos of interviews with students on why they should pick a course. Student participation has been critical to content development and has been integrated with student assignments, employed placement students and provided incentives for students to contribute materials.

 These learner generated resources and materials have made TAG an integrated and sustainable resource at UCLAN.  The project has also developed institutional policy around the positioning of TAG at UCLAN as a University owned and managed resource that links to other services including the Students’ Union.  However, more of a challenge has been to engage staff and to promote TAG externally, for example, raising awareness via UCAS, schools, colleges and other recruitment routes.

UCLAN TAG are looking to use mentors to support social networks, building on the success of the Hello Project (Leicester College) and DevelopMe Project (University of Bradford) who have been using student mentors to support learners through transition..

 TAG provides an opportunity to improve retention through support and engaging learners pre entry, but also through their courses of study in the  generation of new resources. The implementation of TAG within an institution can provide a focus for student retention policies that may already be in place but are currently not being implemented.

 TAG is not so much about the content as about the approach the TAG team have used at UCLAN to develop the materials and how this has been integrated into institutional practice. The TAG team are producing a guide for institutions which will provide more information.

 Representatives at the seminar expressed a keen interest in working with TAG to develop similar resources at their institutions.

 It was suggested that a reserved domain name could exist i.e. http:// tag..ac.uk  being reserved for “The Alternative Guide” for a university  and college in the UK.

 Seminar Discussion

 The  group highlighted issues around student retention and considered how these could be supported by TAG. The issues included:

  •  Course Choice
  • Finance
  • Managing expectations
  • Experiences of assessment from school/FE
  • Coping with levels of independence
  • Students used to being able to re-sit exams if fail
  • Personal issues/home sickness/child care, etc.
  • International students – language skills, time to undertake tasks, etc.
  • Lack of peer support network (making friends through social networks)

 The following barriers to introducing TAG into an institution were discussed

  •  ‘Not invented here’ syndrome for content. The TAG content is being made freely available, but some institutions felt that they would want to develop their own. This highlights the importance of process over product for TAG
  • Student voice is important  and TAG will allow student voice to be heard. However, UCLAN has decided to maintain ownership rather than hand over to students
  • Engaging students in developing TAG is crucial to its success and the approaches used at UCLAN may not be as easy to implement at other institutions
  • TAG won’t fix the problem but will help the institution engage with the issues
  • The need for a champion at a high strategic level within the institution, who can engage with senior management.
  • Funding; developing TAG needs allocation of staffing to support this, although it was felt that existing staffing could be used and once in place  the student-generated materials would make it cost effective. Institutions need to also consider the potential benefits in improving retention
  • Ownership of TAG within the institution –an institutional policy and agreement is needed
  • Timing – numbers and evidence would be helpful to convince senior managers

 Seminar representatives then worked to develop actions plans to make TAG work in their institution and to feedback to the TAG team their requirements and how TAG could support them.

 Conclusion

 The seminar provided a useful opportunity for interested institutions to learn about the TAG project and consider the benefits of adopting a similar approach within their own institutions. From discussions with the seminar participants there are several institutions that are prepared to develop an alternative guide for their own institution and are interested in working with the team from UCLAN and learning from their approach and experience.

 The future TAG project activities under Benefits Realisation should provide an opportunity to support these interested institutions and also to provide the necessary information and evidence for institutions to  be able to take this forward on their own.

 The benefits of external evidence were discussed in the seminar and it would be useful to have some tangible evidence of costs and retention from TAG. It was also noted that as well as the team, JISC and the HEA could provide an advocate role in working senior managers on other institutions.

 The TAG team are also undertaking a study on supporting international students and students from ethnic backgrounds and considering the potential for support at other transition points i.e. research students.

 Further information on the seminar outcomes can be found at http://www.uclan.ac.uk/information/services/lis/tag/tag_hea_seminar.php


[1] HEA Evidence-based Practice Seminar Series http://www.heacademy.ac.uk/ourwork/teachingandlearning/alldisplay?type=resources&newid=events/seminar_series_2010_dates_and_host_institutions&site=york

[2] JISC funded project The Alternative Guide  to  UCLAN http://www.taguclan.org.uk/

Marking Workflows and demonstrator information

The marking workflows as described in previous posts are now under resources on the www.jisc-ea.soton.ac.uk web site.

Also worth mentioning is that the old information about the demonstrator has been moved onto the jisc-ea web site from its former home (also on the resources page). Next step will be updating the history document as this will be useful in our upcoming demonstrations in April.

BRII Presentation

On Friday 5th of March I attended a book launch at the University of Hull – Business School. The book is titled “Supporting research students” by Dr Barbara Allan.”The importance of supporting the needs of research students has recently risen higher up the academic agenda around the world. Numbers of postgraduate students have expanded, and the traditional PhD has now been joined by a new range

Even more documenting!

As promised here is the 100-pages document which has finally been finished (in fact, it’s a bit longer than 100 pages).

We even made a 20-page version of it in case you don’t want to go that much into detail.



These documents should give you a detailed overview of our project, telling you all how we went about doing things like research and design.

Our colleague Clay Fenlason of Georgia Tech even asked for this 20-page document to use as a use case example during his teaching.

And so the design phase came to an end

Lets again summarise what we’ve done during the last months…



Creating extreme designs using personas and requirements

We focused on the concepts and requirements which we thought would be most useful and used those to brainstorm and think of ‘extreme’ designs.

This means we were using these extreme ideas to turn them into metaphors which were much easier to use during the design.

E.g. The metaphore or extreme idea of ‘ballroom dancing’ explained the idea that people sometimes spend time together during a project (so dancing together for a while), after a while they split up and work with someone else (splitting up during a dance and start dancing with someone else in the ballroom).





Extreme ideas turned into metaphors and clustered in groups which seemed to have commonalities























User testing session 1 using paper prototypes

At the end of the research phase we ended up with 3 personas. For each of those personas we created a set of frames (representing the pages within a system) so we ended up with 3 main concepts.

These frames are paper prototypes. This makes sure people will give us more honest feedback (and won’t say things like ‘Hmm, I don’t like the colour of that button’ because they’ll see it’s drawn and not finished at all). We wouldn’t be focusing on that kind of detailed feedback yet anyway.

We recruited a set of people again, representing all the different people within the university (subject, age, role, etc). Whilst showing them the paper prototypes during this user testing session, we also used a set of questions so we were sure we kept focus on the right things. The location was in a usability lab, having a one-way mirror where the observers took notes on the other side of the mirror. This feedback would than be used to refine the concept.



Green concept: focusing on Isobel who’s really sociable, and outgoing. Therefore the frames of this concept focus on the extreme idea of ‘events-going out’ as this is very important to Isobel.





Green concept paper prototypes

- homepage –

























Blue concept: Focusing on Peter who sometimes feels a bit lost. He doesn’t always know what to do first because he’s not always talking to the right people either. Therefore the frames of this concept show the idea of the ‘boardgame’: representing a set of steps you need to fulfil in order to reach your goal.

E.g. In order to fulfil this course, you need to go to this talk, finish this paper etc.





Blue concept paper prototypes

- homepage-

























Red concept: Focusing on Kate who’s senior and just wants to save time in any possible way. This set of frames shows the extreme idea of ‘the switchboard’ which makes is possible for Kate just to see the things she’s interested in, not overwhelming her with things which just waste her time.





Red concept paper prototypes

- homepage-





























Refine concept using feedback from user testing session 1

We did some brainstorming and used all the feedback from user testing session 1 to merge all the screens and thoughts from the first set of screens into 1 set of screens, representing all the refinements and usable ideas.





Example of a page which got all the feedback merged together



































Oszkar redesigning



































User testing session 2 using clickable wireframes

This phase was very similar to the first user testing session, which means it was again in the same usability room, with an observer who noted down the feedback. The difference this time was that we used clickable wireframes instead of paper prototypes. This was 1 set of frames instead of 3 because these were merged together. Further, we used again a questionnaire which guided us through the set of tasks.





clickable wire frames

- homepage –
























Refine concept using feedback from user testing session 2

We used the feedback to again refine the last version of the frames.





























Documents

In the text above, you can find the documents we used during the design phase which might give you a better understanding of the information above.

In case you missed out on them, you can also find them below.



User testing session 1 using paper prototypes

> General information about the concepts – This gives a general description what’s meant with each of the concepts

> Green concept – This is the full set of paper prototype frames used within this concept – mainly focusing on Isobel

> Blue concept – This is the full set of paper prototype frames used within this concept – mainly focusing on Peter

> Red concept – This is the full set of paper prototype frames used within this concept – mainly focusing on Kate

> User testing guide during session 1 – This is the set of questions which we used as a guidance during the user testing session.



User testing session 2 using clickable wire frames

> Clickable wire frames – This is the full set of wire frames used within this merged concept and which we also showed to participants during user testing session 2- it contains all the merged feedback and refinements we got from user testing session 1

> User testing guide during session 2 – This is the set of questions which we used as a guidance during the user testing session.

> Sitemap – This is a visualisation of all the different main pages within the system. A sitemap helps you during designing as wel as to focus on the right tasks during the user testing session.



Refine concept using feedback from user testing session 2

> Final wire frames – This is the full set of wire frames which were the result of the feedback we got from user testing session 2

Planning the design phase

Get ready for our design phase! The next months we’ll be doing user centred design (UCD) which basically means we’ll incorporate all the data we gathered from our research. We’ll do that by looking at the personas, requirements etc.



Oszkar and Tjhien will be spending 6 weeks at Flow Interactive where they’ll have the possibility to work together with people from Flow themselves.

Probably they’ll make various versions of designs which will be user tested from time to time.



Exciting!

Bridge between research and design phase

On Friday 13th March until Wednesday 18th March, we had an intensive period of discussions and workshops with the people from Flow Interactive.

These days would be a wrap up of the research phase and a preparation to the design phase, focusing on tweaking the personas so they’re ready for usage during the design phase, writing scenarios based on the personas, and defining the user and business requirements. Defining requirements, based on the personas and therefore also on what real users want, is necessary for the designers to be able to know what approaches should go in the future system in order to answer the users’ needs.



Define user requirements from persona goals

Each persona had a set of goals, focusing on life goals, end goals, experience goals, motivations, and challenges.

We went through each of these goals and rephrased them into requirements.



E.g. From Kate’s persona: ‘Discover what other people are working on.’ Becomes a requirement like this: ‘The user needs to be notified about relevant people or content they wouldn’t otherwise know about.’ K



Writing user scenarios

A user scenario is a good way to feel you one with the persona and to see how this user could eventually use the system. It’s actually a story describing how the user (that particular persona) might use the system and how it’s an answer to his needs. In the end we came up with detailed user scenarios for every persona.



Concept generation

Based on the requirements, we started brainstorming on how these requirements could turn into concepts within the system. This was just a kind of warm up for the design phase, a pool of ideas for future design. Each of those concepts got written down on post-its.





Harriet jotting down concept ideas



























Prioritise concepts for design activity


We made a big grid: ‘user value’ against ‘effort required’. We discussed where every post-it note should fit on the grid.

For the further design, we will pin our minds on the post-its in the area ‘high user value’ vs ‘low effort required’.





Team discussing the prioritisation of concepts on grid



























Finished prioritisation of concept ideas



























Documents

The document below can also be found in text.



> Focus on personas and scenarios – also describing HOW to create these

And so the research phase came to an end…

Pretty songs always come to an end, and so does this interesting research phase. Lets summarise what we’ve done during the last months…



Gathering information through diaries and interviews

We wanted to gather information from people who could be possible users of our ‘Academic Networking’ system. Therefore we thought of recruiting users within 3 types of roles – being undergraduates, postgraduates and academics. For each of those groups, we gathered 8 people.

Some people would think “Hmmm…how can only 24 people possibly be a representation of your whole group of stakeholders ?!” We think that is possible, because we chose them quite carefully, selecting people within the spread of different subjects, gender, roles, age, stage within their career, etc.

First of all we recruited undergraduates and made that selection of 8 people. We gave them the task to keep a diary, writing down every time they had a conversation with someone about their work, studies, or research. When they did that, we had an interview with each of them to talk through their diaries and give us some more insight on other things as well. Whilst doing that, we used a questionnaire as well which guided us through the various topics we needed to focus on.

Next, we did exactly the same thing with postgraduates and academics; recruiting, selecting right people, giving them a diary, and having interviews.





recruiting people through an online survey



































diary entries



























questions used during interview


































Task goal analysis and sorting

During the interviews, Harriet was hiding in one of the rooms which got a camera and audio connection so she could follow the interviews without any problems. [See Harriet doing that here] The reason why she was spying on us, was because she was doing a task-goal analysis of all the things participants were saying. This analysis was also colour-coded, which meant we ended up with walls like this after every interview:





task goal analysis





















When we finished all the interviews, we started analysing all the entries. Clustering all the data into affinity patterns would give us a better understanding of all the data and would allow us to see the overlaying commonalities between the users as well, whether they were a student or academic.

After doing that clustering, we ended up with a room like this:





affinity sorting























Behavioural axes exercise

Now we had a good understanding of all the users and their underlying ideas and commonalities, we started putting people on grids which contained behavioural axes poles (E.g. High network awareness vs. low network awareness). It’s important we had at least 10- but not more than 15 – behavioural axes. After putting each of our participants on every pole, we clustered them into group. We started seeing patterns of people reappearing together over and over again.

E.g. Sarah and Peter would cluster together on pole 1, and on pole 2, 3, 4 etc.

We ended up with 3 big groups of people clustering together. These would later become ‘personas’ in a way.





behavioural axes exercise with: - behavioural axes and behavioural poles - behavioural patterns (clusters)



















Creating personas

The groups we just discovered on the behavioural axes, where the starting points of our persona creation. (If you want to find out more how to create personas, then have a look at our newer posts as we’ll post something more about this in the future.)

After discussing a lot, we ended up with these 3 main personas and 1 ‘negative’ persona as we called it. This means that persona is someone we wouldn’t focus on immediately. Below you’ll see the 3 main personas.



Persona 1 (primary persona) – Isobel: Very outgoing and sociable. If she would have a problem or question, she wouldn’t mind asking someone. She also thinks every conversation is valuable (so not only professors are worth having discussions with).

Her network of people is very valuable to her. She’s therefore actively building and maintaining her network as everyone can be useful sometimes.

She really likes to go to events where she can meet people and learn in an active way.





Persona 1 (Primary persona) – Isobel

































Persona 2 (secondary persona) – Peter: Someone who rather tries to find solutions about a problem himself by looking in books or going through some other sources before actually asking someone. He just doesn’t like bothering other people. Because he doesn’t like to ask other people that much, he’s not always aware of events or other important things which are going on and because of that, he’s sometimes missing out on information.





Persona 2 (secondary persona)- Peter


































Persona 3 (secondary persona) – Kate: Someone senior in her role (can be a senior student or academic), who feels she already learned a lot and now it’s time to give something back to other people. Because she’s so experienced, she perfectly know who to contact in what situation. She’s maintaining her network rather than building it up. Because she’s senior, she has a lot of responsibilities and is very busy. Sometimes she might miss out on some interesting events because she’s too busy.





Persona 3 (secondary persona) – Kate




































Documents

In the text above, you can find the documents we used during the research phase, as well as some videos which might give you a better understanding of the information above.

In case you missed out on them, you can also find them below.



> Diary – This is the diary we asked participants to fill in.

> Interview questions – This is a script of the questions we used during the interview

> Personas – This is a digitised version of the 3 personas we ended up with



> Task-goal analysis – A video showing what’s going on in the task-goal-analysis-room during the interview. Task-goal analysis in action!

> Affinity sorting – A video showing you how to find themes (clusters) within the bunch of post-its.

> Analysis phase in general & profiles – A short video showing you when creating profiles and some discussion while doing analysis

> Behavioural axes – A short video showing you another use case example which helps you to understand what behavioural axes are

Update from the Teaching Office Database System

Here is an update on what has been achieved by the Teaching Office Database (TODB) project as it enters its final stages.



We have worked with the Engineering Department and six other Departments to develop customised versions of the Teaching Office Database (TODB) system for them. Many enhancements made along the way have been ‘retrofitted’ to systems which were visited earlier in the development cycle.



All of these 6 ‘new’ departments plan to use TODB for the next academic year and have claimed for the funding to cover their costs involved in participating in the project. Three of these now have their own copy of the software which they are running on their own servers. The other 3 are in the process of doing this. I have met with or arranged to meet with the Computer Officers from each department to make sure they feel confident to host and support the system. A mailing list for these computer officers will be set up to facilitate interdepartmental communication about TODB queries.



A generic version of TODB, which is designed to be easily customised, has been developed and will be released under an Open Source Licence. Accompanying documentation will shortly be evaluated by an independent member of CARET. This includes a User Guide which is linked to a Help button in the application, an Installation and Configuration Guide, a Developer’s Guide and a ‘How To’ Guide intended to point to the solutions to normal requests and queries.



We have recently had an enquiry from a new department who are interested in adopting the system. They have seen the generic system and feel that they will be able to take this ‘off-the-shelf’. This would seem to be verification that this process has produced a useful and tuneable product.
Copyright © 2007 Institutional Innovation. All rights reserved.