Friday, October 17, 2008

CCK08 - complexity theory we are everything and nothing

This week's topic in the connectivism course gave me permission to engage as it relates really closely to my Doctorate interests. Although I cannot test the theory I hope to include something about it in the discussion. here is the post I sent to the email forum.

people might be interested in Phelp's 2003 report of action research done on "the development of one tertiary course in computer education for pre-service teachers". It is a great practical example of the importance of working with complexity rather than trying to put our students into boxes - and striving to meet pre-determined objectives.

It helps to illustrate an answer to George Siemen's (2008) questions - "Consider learning outcomes created at the start of a course. Can we really ensure they are achieved? Can the complexity of learning be reduced to six or eight broad statements? Many educators feel that outcomes can be achieved. What is overlooked, however, is that much more than planned outcomes are experienced by learners."

Complexity of learning in my view cannot be reduced to "six or eight broad statements" and if students are given freedom, they will indeed learn more than they or the teacher ever imagined.....if they engage of course.

In Phelp's research, "students were prompted to set themselves appropriate goals and were then challenged to engage with content most relevant to them". they engaged in self-directed and scaffolded learning which was authentic and relevant to each student. plus they took their learning to a metacognitive level. well worth a read. however it did happen over three years BUT like a good cheese, learning takes pre-packaged, quick-fix, unit-standard-type courses probably don't cut it for real learning....well not in the long run. they serve a purpose perhaps to get black and white standards across eg. food safety, how to fly a plane etc. but if you do not really understand why it is important not to clean the cheese (yes the cheese theme is rife today) in dirty dish water, the complexity of life starts to confuse the unit-standard educated worker. We have to help teach people to think don't we?

However according to this Buddhist saying by Hsin Hsin Ming - thinking might not be so hot! I was interested in looking at something from the Buddhist tradition due to my husband's talk of Zen and the belief that we are both everything and nothing. also the post by Tech Ticker about Buddhism and complexity theory " Everything, they say, is related and dependant. Nothing is independent." It is hard to be everything and nothing is it not?

To deny the reality of things is to miss their reality;
to assert the emptiness of things is to miss their reality.
The more you talk and think about it, the further astray you wander from the truth.
Stop talking and thinking and there is nothing you will not be able to know.
Hsin Hsin Ming

Developing Online From Simplicity toward Complexity: Going with the Flow of Non-Linear Learning

Renata Phelps, Southern Cross University (Australia)


The Web is a non-linear environment which opens up potential for new approaches to learning and teaching, approaches which in many ways more closely approximate naturalistic and authentic approaches to learning. Yet a large proportion of online courses which have been developed in higher education represent conversions of print-based resources into Web-based delivery formats, the majority of which have replicated traditional linear and directive pedagogy. Such development represents something of a ‘miss-match’, not only to the online teaching environment but to the emergent learning approaches of a younger generation who are ‘at home’ with the online environment. This paper discusses the benefits of maintaining complexity and non-linearity in online learning with reference to the development of one tertiary course in computer education for pre-service teachers. The theory of complexity is briefly explored and its relevance to online teaching and learning is highlighted. An action research undertaking conducted over a four year period is drawn upon to illustrate the importance of future teachers understanding and experiencing non-linear and complexity-based online learning, and the metacognitive processes that can support adult learners to adapt to such an environment.

Tuesday, September 23, 2008

Silence in the online community

Well I did not get very far with keeping up in the Facilitating Online Communities course. I also took a brief peek at the Connectivism course and decided to focus on the reason for my extended study leave - my Doctorate thesis. So you can follow my progress on my blog Doctorate meanderings - set up especially for the purpose and it has been meandering for some time. you can click on the link to the blog on the right.

Tuesday, August 12, 2008

Online communities - what are they? Who makes them - A mouse or a (wo)man?

The topic of the next two weeks is one I am intrigued to follow as I want to see what everyone's take on an online community actually is. Two questions up front and you can read about how I reached them further on:

What do others think?
  • Can an online community thrive through commensurate (equal) facilitation and and symbiotic (feed off each other) relationships, or do you need one person or several people to take on leadership roles to make it work well?
  • How important is the platform chosen for the online community?
So how did I get to these questions? To start with on the email forum, I found that I agreed with Bronwyn Stuckey's example about learning.

"For me a worrying area is when people call a class of learners a CoP and say that learning is the practice. That is really too large and amorphous to be one practice. A class of students learning accounting by engaging in scenarios as practitioners, possibly with real practitioners in the class as mentors is beginning to take on a CoP approach. According to what you state this course is not a community of practice of online facilitators. We're going there. "

Already the email forum discussion has helped us to see the differences between online communities and CoPs and some belive there is not actually a difference.

I believe there is a difference. However surely an online community is "what you make it". Certainly it can turn into a CoP if it is not already one. Sylvia Currie mentions, "often communities evolve from interest -- to learning -- to practice. is it developmental". Perhaps it is but is it as simple as it seems?

The model of Virtual Community brought to us by Greg Barcelon via Violeta mentions that a Virtual Community exhibits - Responsibility, sharing, celebration - in other words members take the initiative, and they have enjoyment while sharing their ideas and knowledge. As Russ says, "we need to understand the way the community functions". Plus a virtual community can be comprised of groups and is usually interwoven by networks (Bee). This is quite nice but is it too simplistic?

Isn't an online community not just about the participants in it, but also about the ways in which participants interact, and the platforms they use to interact?

For example, as many of you already know, and others will find out as you progress through the course there is a multitude of online community platforms - Second Life, FaceBook, MySpace, Google Groups, wikis, blogs, online games etc, etc, etc. Additionally, some platforms can kill a community as mentioned in Sheryl Nussbaum-Beach's blog post - The Art of Building Virtual Communities - if they are not right for the community.

Therefore if a platform is chosen which does not work for the community, then does this mean that the participants and/or facilitators do not have an adequate understanding of the community or that the platform is at fault? Sheryl also appears to make no distinction between online communities and CoPs.

There is some interesting stuff on Sheryl's post about Member Roles, What makes a healthy community and Derek Wenmouth's The 4 Cs of Participation in Online Communities for blogs. It is an extension and new take on Gilly Salmon's Five-step Model for e-moderation.

According to Greg Barcelon's take on virtual communities everyone is in it together and leadership is not important but is it? Is he just referring to active participation and not leadership? According to the writing by Sheryl Nussbaum-Beach, both her and Etienne Wenger believe that leadership is essential.

Bronwyn Stuckey in her post refers to opportunities for leadership as important:
"the opportunity to step up and step into roles - that this event provided. And ..the shared sense of the value of being together - that evolved through the activity"

So maybe we need to do what the fellow in the pic is doing, and take charge of the situation and go for it if we are to create an online community around the subject of this course - online facilitation. I am looking forward to reading more people's opinions about online communities.

Sunday, August 03, 2008

Facilitating online communities

I have decided that although I am supposed to be on leave working on my thesis it would be intriguing to see how the current iteration of the Facilitating Online Communities course goes now that it is truly open. I am having a hard time keeping my fingers off the keyboard and not participating - yeah cos distractions such as FOC will kill my study leave get a draft of the thesis finished.

I am absolutely intrigued by the numbers who have registered in the FOC08 course now it is openly advertised. Last year we ran the course for the first time using wikiEd, however students were given access via Blackboard and a formal enrollment process. This year we got the go ahead to have informal it is proof of the open concept and evidence of the need for people to keep increasing their skills in facilitating online many experienced people.

So my way to manage having meager participation will be to check in on key blogs so I can get a sense of how it is all going...for the next time I get to facilitate the course. Also so I can keep the debate about teaching versus facilitation going with Leigh Blackall who is superhumanly facilitating 80+ participants. Will the 10% rule rule I wonder? :)

Thursday, July 10, 2008

This is a monarch butterfly I spotted in New Plymouth in march - when the sun was hot! I put it here to remind us there is life after the ice and mud. The last few weeks are pretty full on as I tie up loose ends before going on thesis leave. Yes the pressure to finish the 8 year Doctorate is building. So what have I been up to lately...

Power of Design project
Sunshine did a great poster for the final project workshop which we held just before the HERDSA conference in Rotorua. It was a relief to get the project poster done and cutting back the text was like cutting off an much to report. Now I am writing up a case study for the Power of Design project. This was a research evaluation of Designing for Flexible Learning Practice (DFLP) against eLearning Guidelines. The full analysis of data is still to be completed and will be written up near the end of the year - when I come back.

More on the project website.

Designing for Flexible Learning Practice
The semester one session has finished so there is plenty of marking to do. Five people have submitted their assignments; five are very close and six are going to extend into semester two. We lost a couple or three on the way. It has been really enjoyable teaching using the new format - course wiki and blog. The presentations day was really interesting and I was given a great compliment about the high standard of participant work in the course being due to our very good facilitation of the course.

Evaluation of eLearning for Best Practice
I teach this course for MIT, TANZ. It is all online and I am using the same format as DFLP. This course includes a real workplace evaluation and I have participants from the private sector as well as tertiary organisations. I am about to mark assignments for this group as well and again have some people extending into semester two working self-paced with local support from the programme manager, oriel Kelly.

Digital Information Literacy project
This is progressing well and the 10 week series of workshops is coming to a close for Otago Polytechnic participants. Workshops in Albany, Massey University, MIT and University of Otago have a few more weeks to go. I will be ticking over the management of the project while I am on leave. You can read more on the project website.

Wikieducator projects
  • The HIVAIDS project has stalled but I have been in talks with the Commonwealth of Learning about collaboration options.
  • Anatomy and Physiology for vet nursing - we are awaiting the project plan from the COL learning designer and this should progress with funding in Semester Two. We are meeting on 21 August to progress the project.
  • There is going to be an open educational resources (OER) Thinktank (Heywire8) on 22 August at Otago Polytechnic. Wayne Macintosh from COL is going to visit us here and lead this. There will be several people from around the country converging on us. It is pretty important so I will be making a special trip to take part.
Spotlight Teaching and Learning Colloquium
Terry and I have been helping organise this with University of Otago HEDC staff. It is coming together nicely and you may want to get your abstracts in. All information is on the website and invites are going out to staff and teaching award recipients and tertiary providers in the Southern region.

It is on 19 and 20 November, 2008, hosted by the University of Otago and Otago Polytechnic. The event is entitled “Spotlight on Tertiary Teaching and Learning: Colloquium for the Southern Region”. This event is sponsored by AKO Aotearoa - National Centre for Tertiary Teaching Excellence.

The event will highlight best teaching practice and research into teaching in Otago.

And as if there wasn't enough to do.... I sometimes sleep.

Sunday, July 06, 2008

HERDSA conference - Rotorua 2 – 4 July 2008

Traveling to Rotorua for the Herdsa 2008 conference was a smooth hop to Christchurch and Rotorua. The first day we spent in the eLearning Guidelines workshop sharing our project experiences with the other project leaders. I was pretty proud of the poster created by Sunshine - spot the one with the arrow below. The vertical brochures looked great as well. NorthTec's poster was pretty good as well - very little text, eye-catching and with a separate handout, an A4 brochure with all the info about the project. More on this later...

The highlights at HERDSA 2008 were the key note speakers - Dr Pita Sharples, Dr Barbara Holland, Judith Ramaley, Dr Stuart Middleton and Dr Etienne Wenger – and the Inquiry-based research workshop. I will concentrate on the key note presenters in this post and do another post around the IBL presentation.

  1. Dr Pita Sharples spoke at the Powhiri and shared his perspectives on Maori learners and participation – it is okay to take a long time to get your education – he did. His talk was enjoyable and set the scene for the conference about disadvantaged learners such as Maori, however, only the keynotes addressed this area with regard to engagement, and I only attended one presentation which dealt with this – Selena Chan (given by Nick) and the use of mobile technologies so that bakery students had access to resources. A lot of speakers spoke about engagement and communities of practice, but not about access and equity and diversity. Dr Sharples spoke about the importance of “language nests” for Maori children and also schools where they could carry on the learning. Taking children from Te Kohanga Reo and putting them in mainstream schools did not work. Te Wananga were another advance for improving the participation of Maori students.

  2. Dr Barbara Holland – spoke about communities of engagement. She mentioned how contemporary students crave active and experiential learning eg mobile learning and social networking – I was curious and want to investigate the research on this. She suggested a conference - International Service-Learning conference - Hong Kong 2009. There are different perspectives on what engagement with the community means. For example, an exchange of ideas and information is heading towards engagement, whereas Incubators, continuing education, internships and clinical work which are work-integrated and which promote networking are examples of community engagement. In other words engagement is connected learning and discovery. Dr Holland framed this as community-based learning which enables participants to explore their interests leading to greater self-esteem, motivation, engagement, empowerment. I liked the ideas Dr Holland put forward, as the theoretical basis of a practical framework which Judith Ramaley explained later on. However in vocational education community-based learning is not new. Certainly in nursing and other health professions there is a strong connection with professional communities which is linked to learning objectives and reflection. The development of academic/clinical positions in the School of Nursing at Otago Polytechnic has been dually beneficial. I wonder if they still exist?

  3. Judith Ramaley – was charming and funny as she spoke about how a framework of engagement in an organisation can lead to transformative change where “adaptive expertise” is foremost. That is proponents have the capacity to “learn on the job” and solve problems creatively as they arise. Anyone and everyone can lead change as people work together collaboratively using inquiry and emotional learning to form communities of practice. I particularly liked her analogy to social networking where “institutions become equivalent to a social network or open source model in which anyone may offer suggestions, contribute to advancing the institution and feel an integral part of the enterprise as a whole” - a “transformational change dynamic”. It is not a top down model and I believe OP has the beginnings of this capacity although our managers are immersed in a model of strategic change. The work we are doing with the Commonwealth of Learning through WikiEducator is an example of community-based learning and this is being led from the “coal-face”. It is disruptive, complex and collaborative and transformative for anyone who becomes involved and capacity is building around the nexus of open educational resources (Holland & Ramaley, 2008, p45). References: Holland B. & Ramaley, J. (2008). Creating a Supportive Environment for Community- University Engagement: Conceptual Frameworks. HERDSA Annual Conference 2008 Proceedings, p33-47.

  4. Dr Stuart Middleton – was also funny and it was obvious he was not very impressed by politicians and the impact of government policies on access to education. We now have a situation where education has become more academic and has lost flexibility – the one size fits all approach means there is no longer a differentiated curriculum. The result is that disengagement is now occurring, and even though there is growth in education a large number of people turning are turning their backs on education and there are serious skills shortages. He likened the educational system to a nut which was resistant to change (the nut cracker), with disengagement showing up as one of the cracks as the arms of the nut cracker (changing demographics and economies) squeezed the nut. As Dr Middleton said, “Given the filters of failure that currently operate within education up to the point of entry into further and higher education, the group that presents itself for postsecondary education has become such a distorted reflection of the community that even the most carefully attentive and sensitive selection procedures would simply not achieve equity nor provide access for a majority of students.” We have a “leaky education system” with large numbers dropping out of secondary school, truant and leaving with no qualifications. Maori participation has increased thanks to the Te Wanangas, however, participation and completion rates in mainstream universities continue to be low. The diagram below depicts equity and access, and shows where some of the cracks lie in the transition from secondary to higher education. I will link to the paper once the proceedings are online. In the not too distant future, white Europeans will be in the minority so those groups of learners who have traditionally been under-represented are increasing, meaning HE needs to work differently. For example, the Pacific Island population is predicted to increase to make up 51% of the population in Auckland in the next five years. He also referred to examples in the literature around student retention and success stating, “Supporting students is critical to their success”.

Dr Middleton’s presentation was very powerful and I particularly liked the following quotes: “Simply by sailing in a new direction, you could change the world” (Curnow). “We have to be careful not to cross a crevasse in two steps” (Middleton), this was regarding the new funding regime – which he believes is a good move as we have moved away from the previous system but it is happening too quickly. For example, the Adult Community Education funding should never have been dismantled as 40% of people participating in Community Learning Centres in Auckland go on to HE. And the best quote, “Resources have to be distributed unevenly when the need is uneven” (Middleton).

5. Dr Etienne Wenger – rearranged the main venue and put us in circles facing each other. This was tricky for some who were unable to nod off, though believe it or not, they tried. Etienne tried to engage the audience in reflection on some of the presentations about Communities of Practice which worked a little but people had come to hear his wisdom and they were tired from the conference dinner and dancing so not particularly responsive and there was no real discussion. Personally I would have liked to hear more about his new theories and frameworks for COPs, but apparently this would have repeated his workshop. There were a couple of good diagrams about the social discipline of learning, an approach also
covered in his latest book, so I hope the diagrams will appear on the conference web site
The conference was closed by Rotorua's Deputy Mayor and his closing speech and prayer sent us on our way after he led a rendition of pokare kare ana.

The poster (spot the one with the arrow) will be travelling to Wellington with John Milne and then to the DEANZ conference in Wellington.

Tuesday, June 17, 2008

Open educational resources

Crossroads sign by Bill Owen
 The importance of our relationship with WikiEducator and the significance of open educational resources (OER) is really beginning to sink in. We are part of a vision and it rocks!

I attended a DEANZ web-conference (Elluminate) presentation by Wayne MacKintosh from the Commonwealth of Learning today - WikiEducator: DE reincarnated or new innovation? I realised after Wayne mentioned how Otago Polytechnic was a leader in OER in NZ, how far behind other organisations actually are in this area. The whole thing has taken on a life of its own since we started developing courses on WikiEd.

I agree with Wayne the structure of WikiEd does enhance asynchronous interaction. For one thing in the Designing for Flexible Learning course, participants can study self-paced using the course schedule as a guide, and they can leave each other messages in the Discussion (Talk) area. So far I haven't experienced many students using the wiki collaboratively though some participants in the Facilitating Online Communities did create pages for an assignment though I am not sure what will happen in the next iteration.

Wayne mentioned that social phenomenon of our time - Wikipedia - and how well it does in providing free access to the sum of our human knowledge - well it is the largest encyclopedia in the history of human kind. Well it is looking like WikiEd might be getting even bigger. Their strategy of access to free education for all countries by 2015 is phenomenal and worthy of our attention. He explained how community is pivotal - and involves content, capacity, connections;
sustainability is important for the OER project or they will fail.

Some stats
More than half of the registered users on WikiEd are over 50 yrs, and the majority are educators. Wayne's slide show has all the detail. A very interesting statistic - 12% of Wikied users are actively editing whereas in Wikipedia only 3% of the users are active editors - probably because more people access it than develop the articles. WikiEd is the first wiki platform to enable generation of pdf from wiki pages. There is also a facility to generate content cartridges of wiki pages for importation into LMS. And last but not least - Learning4content - is the largest wiki training initiative in the world.

WikiEd team
graphic designer
learning designer
critical reader
subject matter expert

A return to tradition
There are several levels of certification available following training:

It is possible to get certification loaded on your user page after having done the training; maybe I will work harder on getting this. One thing I do need to do is create a decent profile page. Leigh has been nagging me for ages to do this.

In the future, WikiEd is looking at developing the extra capability of exporting an IMS package. It enables direct import of content to LMS without creation of a cartridge. Not sure why this is necessary but I guess it will satisfy people who are unwilling to share and feel cheated if others use their content.

A school initiative is racking up the action: Another school example - There is a big opportunity for us to work with schools in helping them develop content on wikiEd. I wonder if the push is still there to do this? Not being on Academic Board any more I don't hear about these things and the info is not filtering to us.

Utah state is going to create an open school - an OER handbook is being developed to help educators create OERs. Perhaps they could use some of the stuff we have developed on digital information literacy on the WikiEd platform and which we are using in the Digital Information Literacy project also located on WikiEd. much to do and so little time.

It was a good presentation and although discussion was slow and had that halted factor due to the "push to talk" feature of Elluminate, I was glad I made the time to attend. It has motivated, mortified and made me petrified....will I ever get my thesis written?

Monday, May 19, 2008

So what sort of help is there to help learners access flexible learning equitably?

Thank you Pam for a very interesting presentation last week about the Disabilities Service. There was a small but very interested group at the Elluminate session and the discussion which followed covered a range of things about flexible learning. The disabilities Service supports people with a range of health issues as well as people who are visually impaired, deaf, or have physical immobility.

It turns out there is lots of support students can obtain to help them access learning opportunities. As well as note-takers, peer tutoring and arrangements for taking exams, students may also need support to access online materials and use computers.

For example, Dragon Voice Recognition software is useful for people to use to create electronic documents, if they are unable type on a computer for long periods, or even if they want to work more efficiently. It does take a while to train yourself to use this software but there are some very skilled trainers offering this service e.g. Brian Treanor at otago Polytechnic.

JAWS (Job Access With Speech) is a screen reader software which people who are visually impaired can use to have an online page read to them (Wikipedia, 2008.) I know my father-in-law uses something like this for his emails and searching on the Net. He does a lot of work on his computer and is for official purposes rated as blind. Being a Veteran in the USA means he gets lots and lots of government support, and special training to use the products he is supplied the track here, but I do wonder if there is a similar service in NZ which is just as comprehensive - special Veteran hospitals etc. Anyway the support he has had for computing means he has been able to record the family history and write books.

There is a raft of products available to help people who are visually impaired with their computing needs. You might like to take a look at some of them on the Freedom Scientific website.

Training can be organised in different types of software by the Disabilities Service. One thing Pam has asked is that lecturers provide easier access to documents for people who are visually impaired. For example, make sure documents are in Word-like formats (we prefer Rich Text Format) as well as pdf because the font can more easily be increased for printing.

Staff at the Disabilities Service are available to give advice to teachers as well so they can find out how to better support their students. One thing I think that lecturers at the polytechnic do quite well is providing extensions for assignments for students with or without health problems - sometimes life just gets too busy. However, the Disabilities Service is available to negotiate extensions on behalf of students.

This leads me to the idea of flexibility for assessments - how important is it that students have to submit assignments under strict deadlines? My belief is that due dates should be a guide only, and throughout my years of study I have been very lucky to have flexibility with my studies. I would never have been able to complete anything otherwise. Due to flexibility and being treated as an autonomous learner, I have completed a Diploma of Teaching (Tertiary), a Certificate in Clinical Teaching, and nine Doctoral courses and a pilot research study. No flexibility, no completion. :(

Is inflexibility in assessment conducive to learning?

I think we need to design assessments that enable learners to document what they are learning rather than encourage rote learning. Exams have long been a thorn in my side and yet there are people who believe that some subjects can only be tested through exams. For example, bioscience which is very fact-based. However, I have used "open book" formative quizzes and learning portfolios very successfully for this subject. The students preferred these types of assessment to exams because it took the stress off them, and they were really able to think about how the bioscience facts they were learning could be applied in real, clinical situations. Their learning was more holistic, and the formative self-tests enabled them to work through facts step-by-step as well.

However some NZQA assessments for unit standards encourage the use of Mastery learning - I am not a fan of this method as you may have guessed. Repetitive practice for certain skills is necessary for safety, e.g driving and flying, but critical thinking skills are much more important in the long term. When the crunch comes and you have to decide for example, how to land the plane which has gone out of control or has been hijacked.

Do you agree with me? And I ask the following questions as well and hope some of you can continue this discussion on here.
  • Is the structure of deadlines inhibiting or is it necessary to provide structure in a course?
  • Is it unfair to others if some get extensions?

Tuesday, April 29, 2008

Teams are they more innovative and fun

Thank goodness for Leigh and his ability to keep us updated with the latest developments. The BBC interview where Peter Day talks to Professor Lynda Gratton of the London Business School was very interesting. Prof Gratton believes that good teamwork is what will make companies successful in the 21st century.

I like the ideas they were discussing e.g ignite teams with "big questions" to explore and solve - innovative teams are where there is a small core of people and "volunteers" come in because they are passionate about the project. Hence a team leader's role becomes not so much to direct and choose and but to inspire. The ability to cooperate is the most important characteristic for people to have in modern times so they can work collaboratively in teams. The focus is on the good of the team not on the individual's needs. Also the ability to be able to make connections with others, locally and globally. Graduates with these attributes are highly sought after in big companies.

People in a business if left to their own devices will sign up to project teams which excite and inspire them. eg. Google. Innovation is created when people who are very different share their ideas and they often do not agree with each other but they can cooperate.

So perhaps the idea of the teams and committees we have in the poly may not be the best for cohesion and for spreading the passion eg. the quality in teaching team was chosen - perhaps if people were invited to join and did so because of their passion about quality in teaching and learning, a very different mix would have been created, perhaps a more innovative (no offence meant here) combination may have been achieved. Having "experts" or higher levels of staff on a team is not necessarily more productive and effective - you need people to challenge because they have a different perspective.

Big teams can be very productive and everyone has the chance to shine. Gender and ethnicity mixed teams are much more productive. Some of the examples discussed were interesting such as Google, Nokia, BP and a Scottish bank. If they can do it surely we can as well.

One of the things which made my job in eLearning more exciting in the early days was the collaborative projects and networks I was involved in. Being able to have the freedom and autonomy to choose projects means I can be part of several teams, and I have learned so much from working collaboratively. The networked learning team is an excellent example of a collaborative association where ideas are discussed and shared, and people join because they are passionate about the area and want to develop resources to share with students and staff alike.

Now we have the Digital Information Literacy project and collaboration across Otago Polytechnic, Manukau Institute of Technology, University of Otago and Massey University. Watch this space for the project wiki page - it will be appearing soon.

Thursday, April 17, 2008

Observation Questions and Usability Checklist Eval08

In response to a request by participants in the Evaluation of eLearning Best practice course, I have made some observation questions available. The actual document can be accessed and modified for your own use and is on Google docs - Observation Questions eval08

There is also a Usability Checklist eval08 available. Feel free to modify and use in yur own context.

The observation questions and usability checklist were used in the Online Information Literacy - OIL - evaluation project in 2006 and 2007. I also discovered how to send documents to my blog. Google is so clever.

Observation Date: Time Taken: Observer:

Observation Questions


1. Are there any difficulties in getting the user started?

2. How does the user engage with the module and follow the icons and instructions?

3. If the user gets lost how and why does this happen?

4. What suggestions or comments does the user make about the module?

5. Other observations.

Bronwyn Hegarty, Otago Polytechnic, 2007

Sunday, April 06, 2008

My reflections on a presentation of an evaluation project - DVD resources in Nursing

It was great to catch up with Bonnie Schroyen and Suzy Poppe from the Nursing Department at NorthTec again and I was very appreciative that they agreed to give a presentation to the 2008 class in the class I am teaching/facilitating, Evaluation of eLearning for best practice. It was especially valuabel because they were alumni from the course. The presentation only can be accessed by clicking: Evaluating the effectiveness of a DVD in the flexible mode of delivery in nursing and I have loaded the presentation with synced audio on to a web-based facility called MyPlick. The full web-conference session can be found by clicking here.

So what happened?
I introduced Bonnie and Suzy to the class, eight of whom attended the session on Elluminate, and then encouraged the class to introduce themselves to the presenters. Half the class had microphones so the rest were asked to introduce themselves on the chat facility. However this was probably unnecessary as I had also initiated introductions on the Whiteboard, and most people had written their name and area of work there for everyone to see. I felt this worked quite well and next time, I will do the same with the Whiteboard introductions and only introduce the presenters using voice.

The class will get their chance to ask questions after the presentation, or during it if the presenters prefer this, and I will need to ensure that everyone gets a chance to speak. Anyone who asked a question on chat, was either responded to by the presenters or I asked the question on their behalf.

Bonnie and Suzy had a short PowerPoint presentation describing the evaluation project which was easy to follow. They explained how they used a mixed methods approach (survey and online discussion) in a formative evaluation to investigate a new DVD resource which had been designed to enhance an "online nursing module in the Bachelor of Nursing (BN)".

They investigated the "student’s feelings and opinions about the design and ease of use of this resource", so they could make recommendations about improvements which could be made to the design. My understanding is that the DVD provided students with guest speaker presentations. This was trialled as a way to reduce costs and travel for guest speakers.

The sample size for their evaluation was quite good with 20 respondents. They used likert scales in the survey and collected quantitative data which was analysed using a tool in the Moodle Learning Management system. It would have been good to see some of the graphs they produced - I may be able to obtain them from the presenters. Qualitative data was collected from the discussion forum, and the comments were very positive. Overall, the presenters found that the students believed the DVD was effective.

Bonnie and Suzy had not found the task of conducting the evaluation project an onerous one, and enjoyed doing it. That was great to hear, and they thought it only took them around 20 hours. Participants in the course have an allocation of 50 hours for the project, and providing they do not make their project too big, they should be able to remain within the time frame.

I did not ask them what their "big picture" questions were, but this might have made the main thrust of the project more understandable. Some graphics on the slides would have been good but the design was clear and simple and their explanations good so pics might have deflected our focus. Some graphs of the results would have been good.

Discussion time
Following the presentation we had some very good discussion about the usefulness of DVD resources in a flexible course, ongoing use and currency of the resource and modifications. For example, the nursing course is now primarily online and short video clips are being used rather than the longer versions previously presented on DVD. People were interested in the idea of version control being added to the DVD, which would save having to collect them all in at the end of the course. This was necessary to prevent people using outdated material which might contain techniques which were no longer safe. I am having trouble uploading the discussion to the web and will add this later.

I missed Suzy's statement about the examples of video clips so did not get them to show an example of what they are doing now. That was a pity - perhaps a link to one or two would still be useful, so people can see they type of resource they are using and the quality.

My feeling was that the session's focus was on the resource rather than the aspects of the evaluation project itself, but I believe it was still useful in getting across the message about how important it is to evaluate eLearning resources. I will be interested now to read about others' thoughts about the session, and will run a poll to find out how useful, the class have found the presentation in helping them get going on their own evaluation projects.

There has been one positive comment about the presentation so far and someone is actually excited about the next one. At least it is pleasing one person, and it was great to see eight people there on the night. Right on!

Monday, February 25, 2008

Research evaluation has good news for blogging

It was great to discover some research with good things to say about using blogging for learning. This is also a great example of how evaluation can provide evidence for teaching and learning innovation.

"This article received an Outstanding Paper Award at ascilite Singapore 2007 Conference, gaining the additional recognition of publication of an expanded version in AJET. " (
Farmer, Yue & Brooks, (2008)

Reference: Farmer, B., Yue, A. & Brooks, C. (2008). Using blogging for higher order learning in large cohort university teaching: A case study. Australasian Journal of Educational Technology, 24(2), 123-136.

The evaluation is still being carried out and preliminary results indicate there is potential in using blogging to enhance learning.

The blogging trial was conducted and evaluated with 200+ first year students being team taught in a "Cultural Studies Program at the University of Melbourne"(p125). To ensure security and because the blogs were being assessed, a purpose-built platform called "CultureBlogging" was created, and the authors also mention it aided compatibility with a proprietary learning system.

It appears the exercise was overall well supported by students though grades were a little lower than those obtained through traditional assessment methods. Student feedback indicated that more scaffolding in the way of guidelines on how to write on a blog and how to write reflectively about themselves was needed.

"Although most students were generally media savvy in their uptake of new technologies and some were already familiar with the tools of blogging, most however were not necessarily familiar with the nature and possibilities of blogging as a self reflexive practice. More guidance on the pedagogical aims of blogging would possibly have helped make the exercise more user friendly and critically transformative"(Farmer et al., p8).

Only 56/200 responded to the questionnaire about the use of blogging in the course, but the researchers also have the blogs to examine and content analysis of these is continuing. There was positive and negative feedback about the blogging exercise but overall the researchers report on the use of blogging in a large group of students as worthwhile. They did pick up a conflict between students' perceptions of blogging and educational use and this is an area where they recommend some strategies will need to be applied.

"We would hope that by supporting, guiding and modelling use of modest Web 2.0 tools such as blogs we will be enablingstudents to take on ‘prosumer’ identities that are more significant and self aware than the simple phatic discourse of online sociability and the prosaic ‘daily diary’ experience that many students seem to associate with blogging" (Farmer et al., p12).

There are more articles waiting for me to read in AJET (Australian Journal of Educational Technology) .

Sunday, February 24, 2008

Evaluation of eLearning for Best Practice course is starting

Well it has been a mission getting the course material ready. The picture of Kaihaihai beach from the zigzag track near the Heaphy track is a reminder I was on holiday this year.

I decided to transfer everything to WikiEducator for ease of updating and currency. Also it is a great opportunity to put some of the facilitation ideas and tools we used in Facilitating eLearning Communities course into practice.

How did I get interested in evaluation?
My interest in evaluation comes from spending a number of years teaching and learning online and designing and developing online courses. I have also used a number of multimedia resources, and the quality of them has been rather variable. I studied evaluation for interactive learning systems with Professor John Hedberg when he was lecturing at the University of Wollongong. Yes I was a distance student, and that is the best place to sit if you want to critique best practice.

Thomas Reeves and John Hedberg felt so strongly about the need for better evaluation practices that they wrote a textbook about the subject. The reference is - Reeves, T. & Hedberg, J. (2003). Interactive learning systems evaluation . Educational Technology Publications: Englewood Cliffs, New Jersey.

I was able to put evaluation into practice when I was part of a two year collaborative project (eLearning Collaobrative Development Fund - eCDF) to develop online information literacy modules. My experience as Chair of the Analysis and Evaluation committee was invaluable - I led several evaluation projects as part of the development and learned a heap about evaluation in action. There are several reports in the Research and Evaluation section on the project website at:

I also build in evaluation in all the flexible development projects I manage and I am always intrigued by the lack of awareness for the process of evaluation. I am really looking forward to working with a new group in the class and to help them complete an evaluation project relevant to their work.

Tuesday, January 29, 2008

Ideas for Dunedin International Computer Mediated Social Networking Conference

I am really interested in contributing to a paper for the Dunedin International Computer Mediated Social Networking Conference in June with Leigh Blackall and Sarah Stewart. It is a great idea to base it on the Facilitating eLearning Communities course.

I have read Leigh and Sarah's posts and ideas and have come up with some thoughts for a paper for this conference. Looking at the list of people in the conference programme committee, and the place where papers will be published - this looks like a very computer science-focussed conference. For example, papers will be published in the "Springer-Verlag in the LNCS/LNAI series" - I found this for the Springer LNCS series.

So what will this audience be interested in? In the introductory blurb they mention Social Network Systems (SNS) and have provided a list of questions they want answered. I have added them to the wiki for consideration and discussion as well.

I believe we could answer the following Q from an educational point of view: The other Qs don not seem relevant to the online communities course experience - correct me if I am wrong.
  • How can various Web 2.0 tools be integrated to satisfy the needs of electronic communities in a holistic manner?
Are we preparing a case study as an example of what happened? This could be done from the perspective of whoever in the Facilitating elearning community collaborates to write the paper - that way permission will not be a problem - their were insufficient evaluations and we were not asking the right Qs to get any useful info for the paper.

Perhaps if we describe how we tried to integrate the various web 2.0 tools with the aim of developing an online community and modelling different communities - we can also include the issues which cropped up. Our perspective of how the community developed and what facilitation was needed to assist this. we do not have enough data or time to do a proper evaluation of participants to get it from their perspective unless they contribute to the paper.

This would enable a discussion of how we used blogs and wikis and email, SL etc to communicate and present the course content - The set up and maintenance of the Facilitating Online Learning Communities course. we can also bring in the class experiences (Experiences of the participants and examples of how their new learning is being used in their work - if they are contributors) and include the different types of web 2.0 communities to which we exposed them and discuss how the sense of community developed in the course. (Outstanding issues and considerations arising from the course.)

I do not think we will have time to include teacher/facilitator issues or the rest of the items suggested by Leigh (italicised above) - these might have to be addressed in another paper.
  1. Further work we will do in developing education generally at Otago Polytechnic using socially networked media and communications.
  2. Frank and honest discussion on the probable and existing issues with this vision and Otago Polytechnic
Re the presentation on the day, I think it would be of more value to beam in or play some course participant comments about how they thought the web 2.0 integration thing went, rather than comments from the 10 min lecture series presenters as this should really be about the experiences of the course facilitators and participants if this is a case study that is.