Adopting a Design-Based Research approach to harnessing the power of social and participatory media
Here is a draft of a peper we are writing: Gráinne Conole, Rebecca Galley and Juliette Culver, The Open University, UK
This paper describes a new social networking site, Cloudworks, which has been designed to support the sharing and discussion of learning and teaching ideas and designs. A new framework on community indicators for describing and evaluating user behaviour will be introduced. The paper will describe the development of the site and outline the new emergence patterns of behaviour we are observing on the site and associated discourses.
The affordances of new technologies appear to offer much to support learning, however there is a gap between this potential and their actual use in practice. Jenkins et al. (2009) argue that there are twelve skills needed for full engagement in today’s participatory culture: play, performance, simulation, appropriation, multitasking, distributed cognition, collective intelligence, judgment, transmedia navigation, networking, negotiation, and visualization – the ability to interpret and create data representations for the purposes of expressing ideas, finding patterns, and identifying.
To make full use of the potential of new technologies both teachers and learners need to reskill to embrace these new literacies. This paper contends that learning design can be used as a methodology to help teachers and learners to develop these new skills. It will outline some of the research in this area being undertaken by the OU Learning Design Initiative (http://ouldi.open.ac.uk). It will focus on one aspect of this work – the development and evaluation of a new social networking site (Cloudworks) for discussing and sharing learning and teaching ideas and designs. It will describe how Cloudworks is attempting to harness the power of new technologies and in particular web 2.0 practices for an educational context; specifically as a means of facilitating greater discussion and sharing of learning and teaching ideas. The site is attempting to address three inter-related issues:
· The lack of uptake of technologies for learning and teaching (despite the fact as outline above that they have immense potential).
· The new skills needed for engaging in a participatory digital landscape.
· When asked the question ‘what do you need in order to make better use of new technologies in your teaching?’ teachers invariably say they want examples and they want to be able to share and discuss their ideas with others.
We have developed a social networking site (Cloudworks) to enable teachers and learners to discuss and share learning and teaching ideas and designs to address these issues. An overview of Cloudworks will be provided, along with a definition of key concepts associated with the site. We will focus is on the new patterns of user behaviour that are emerging through use of the site, along with mapping these to a number of theoretical perspectives. We have adopted a Design-Based Research approach, which is closely aligned to agile development described by Cockburn (2001) to development of the site. The initial phases of development and evaluation of the site are described elsewhere (Conole & Culver 2009; Conole and Culver, 2010). The initial underpinning theoretical basis builds on Engeström’s (2005, 2007) notion of social objects and Bauman et al.’s (2007) framework for sociality the site. We have recent begun exploring other theoretical perspectives to consider how they might help us explain the patterns of user behaviour we are seeing in the site (Alevizou et al., 2010). We argue that Cloudworks represents a new direction for designing for learning; by providing a space for both learners and teachers to make learning designs more explicit and sharable, and as a web 2.0-based dialogic space for critiquing learning and teaching ideas.
The OU Learning Design Initiative
Despite the fact that there are numerous repositories of good practice, case studies, learning objects and Open Educational Resources (OER), their impact on practice has been limited (McAndrew et al., 2009). The vision behind the development of Cloudworks was to harness web 2.0 practices specifically to foster dialogic exchange between educational practitioners. In order to get a better understanding of the extent to which technologies were being used across the university, a set of 45 case studies were captured (Wilson, 2007). In addition a series of semi-structure interviews were carried out with 12 teachers across the university to get a more in-depth understanding of they design practices (Clark and Cross, 2010). The interviews focused around a number of themes: i) how teachers went about the design process, ii) where they got ideas or inspiration from, iii) how they represented their designs, iv) how and with whom they shared their design with, v) where they got additional help or support, and vi) what kinds of evaluations activities did they undertake to access success. From the case studies and the interviews it was evident that teachers design practices were creative, messy and iterative; and primarily based on prior experiences and inherent believes, rather than any formal, set of design principles. Resources and more information on the OU Learning Design Initiative are available at http://ouldi.open.ac.uk and Conole (2010) provides a detailed description of the theoretical underpinnings of the initiative and describes the various tools and resources we have developed.
Part of the aspiration of the OU Learning Design work was to help make the design process more explicit and hence sharable. We have developed three types of tools/resources with this in mind. The first is a set of conceptual design tools – to help teachers think beyond content when designing learning activities. We have developed a series of ‘views’ which foreground different levels of design and aspects of the design (See Conole, 2010 for a more detailed discussion and also http://cloudworks.ac.uk/index.php/cloudscape/view/1907). The second, CompendiumLD, is a visualisation tool for guiding teachers through the design process (See Conole, Brasher, et al., 2009). It includes templates of the conceptual tools, as well as in-built help and guidance. The third, Cloudworks, is a collaborative tool, aimed at helping teachers to share and discuss learning and teaching ideas and designs. This paper concentrates on Cloudworks and in particular some of the dialogic discourses and practices, which are now emerging on the site.
An overview of Cloudworks
Cloudworks is a specialised social networking site for sharing, debating and co-creating ideas as well as designs and resources for teaching, learning and scholarship in education. Conole and Culver (2009) provide a description of the original vision behind the development of Cloudworks and the associated theoretical underpinnings. The site is essentially object- rather than ego-centered in nature (Dron and Anderson, 2007). Figure 1 provides a screenshot of the homepage. The core object in the site is a ‘Cloud’, which can be anything to do with learning and teaching (a description of a learning and teaching practice, an outline about a particular tool or resource, a discussion point).
Figure 1: Screenshot of the Cloudworks homepage
Clouds combine a number of features of other Web 2.0 technologies. Firstly, they are like collective blogs, i.e. additional material can be added to the cloud, which appear as sequential entries under the first contribution. Secondly, they are like discussion forums, there is a column under the main cloud where users can post comments, i.e. they are ‘social’. This aligns with Engestrom (2005) notion of the importance of social objects as the key focus of social networks. Thirdly, they are like social bookmarking sites, i.e. links and academic references can be added. Finally they have a range of other functionalities common on Web 2.0 sites, such as ‘tagging’, ‘favouriting’, RSS feeds, the concept of following, and activity streams (See Conole and Alevizou for a review of Web 2.0 practices). Collectively these features provide a range of routes through the site and enable users to collectively improve clouds in a number of ways. Clouds can be grouped together into aggregations, termed Cloudscapes. The homepage of the site, in addition to providing standard navigation routes (such as browsing of Clouds, Cloudscapes and People and searching), lists currently active Clouds and five featured Cloudscapes. All recent activities on the site (newly created Clouds and Cloudscapes, comments, additions, etc) are listing in a site Cloudstream.
The Community Indicators Framework
A key issue in the evaluation of social and participatory sites, such as Cloudworks, is understanding what types of user behaviour are emerging. In order to understand this we have reviewed the literature on different frameworks for describing ‘communities’ in online spaces (Galley, 2010a; Galley et. al., 2010). Galley et al. (2010) suggest that the notion of ‘communities’ in social and participatory spaces is different and argue that:
participatory web processes and practices have more recently opened up new spaces for, and styles of, interaction – social spaces which enable transient, collaborative, knowledge building communities, and the development of shared assets such as interests, goals, content and ideas.
They looked at various frameworks for describing communities such as: Communities of Practice (Wenger, 1998), ‘Networks of Practice’ Brown and Duguid (2001), ‘Network Sociality’ Wittel (2001), ‘Communities of Inquiry’ (Garrison et al., 2000), and ‘Communities of Interest’ (Fischer, 2002). From these a new Community Indicators framework has been developed, which consists of four broad aspects (or indicators) associated
with: behaviours and attributes of participants (e.g. turn taking, tolerance, and playfulness), situational factors (e.g. clarity of purpose, cross-boundary participation, culture) and how participants feel (e.g. sense of ownership, trust, enjoyment or engagement). Each of these aspects is interrelated and the whole reflects the multifaceted complexity of what we experience as community.
Figure 2: The Community Indicators Framework
Figure 2 illustrates the framework. The indicators are: participation, cohesion, identity and creative capability. Users participate in different ways in social and participatory spaces; it is often important to have someone adopting a leadership role, whilst others might comment or post links and some choose to simply read and not post. The cohesion indicator is concerned with the types of social interactions that occur and highlights the importance of emotional and peer support in such spaces. Identity relates to the groups evolving self-awareness. Galley et al. (2010) draw on Bouman et al’s (2007) notion of sociality, which argues that that: online environments, like Cloudworks, need to accommodate both the evolution of practices and the inclusion of newcomers, both individual identity and group formation are important and people are more inclined to use software systems that resemble their daily routines, language and practices than to adopt whole new concepts. Finally, creative capability relates to how far the community is motivated and able to engage in participatory activity, and is of particular importance to us in our work in terms of helping teachers think differently and enabling them to be more creative in their practice. This theme relates to participants’ skills, qualities and experience (including those relating to digital literacy and inter-and intra-personal skills – such as conflict management), community and individual motivations, and the capacity of the emerging community to mediate between these aspects, and exploit the cultural, ethnic, social, and personal differences between participants within the community.
Evaluation of Cloudworks
Use and development of the site is being monitored in a number of ways. We have adopted a Design-Based Research (DBR) approach to the development of the site, through a series of design phases, where each phase has consisted of a series of design decisions and subsequent evaluation (Conole and Culver, 2010). Data collection has included web stats and Google analytics, analysis of site activities and discussions, collation of references to Cloudworks elsewhere (such as in the blogosphere and Twitter), and use and evaluation of the site at workshops and conferences. A Cloudworks questionnaire is also available online. This multi-faceted evaluation strategy has gathered data that has then been used to inform the next design phase, thus ensuring an alignment between technical developments and user needs. The data, and particularly the user feedback, has given us a rich understanding of how the site has evolved and how it is being used. At key points we have commission an expert review of the site and have to date undergone three site redesigns, commissioning an expert external designer.
A range of standard statistics is gathered routinely (Figure 3), along with an administrative Cloudstream, which in addition to listing activities on the site chronologically (in the way that the main site Cloudstream does), it also documents when new users register with the site (the site is open, but users need to register if they wish to post anything or create Clouds or Cloudscapes) and when users choose to ‘follow’ others. We will also be capturing on a 6 monthly basis: the number of users who have posted clouds, the number of users who have posted comments, and the number of unique users posting a cloud or comment in last 60 days. To measure sustainability and longevity of contribution, we are also capturing: the number of registered users who have posted a cloud or comment at least one month after registration (this way we don’t count the initial use of the site for say a conference or workshop) and the number of registered users who have posted a cloud or comment at least a year after registration.
Figure 3: Statistics for the site – 01052010
The site is also linked to Google analytics (Figure 3), which shows the growth of the site since its launch in July 2009. As is evident with other Web 2.0 sites, the number of active contributors to the site (currently 2, 275 registered) is less than the number of unique visitors (59, 171 visits from 163 countries). The top five countries are UK, United States, Canada, Australia and Italy). We have also undertaken a number of qualitative studies of the use of the site; including explorations around how the site is being used by a particular community or theme and through a series of interviews with users.
Figure 4: Google analytics July 2009-April 2010
Users of Cloudworks have been encouraged to complete an online survey after workshops and conferences, a
nd in April 2010, 299 people registered on the site were randomly chosen to participate in a survey. In total, approximately 100 surveys have been completed during phase-one and two. All data used from the surveys is anonymous. A series of interviews have been conducted, these include some short unstructured three-minute interviews asking for perceptions of the site (where permission was given these are publically available here: http://cloudworks.ac.uk/cloudscape/view/1900). Other more in-depth, semi-structured interviews have also been conducted particularly with established users, with a focus on exploring how Cloudworks is used, and the perceived advantages of using the site. Two usability tests of the site have also been conducted. We have kept reflective logs, documenting the process of development and use of the site. These reflections are available in public blogs and links to the full postings have been included where extracts have been used in reports and evaluations. Occasionally users will also discuss their perceptions of Cloudworks in their own publicly available blogs or on the site itself, where these have been referred to direct links have not been made in our reports or papers as we recognise that our use of the postings in this way could not have been anticipated when the posting was made. In these cases we have also removed names and any identifying information. User activity data has been collected from the site relating to: content: number of Clouds in the Cloudscape, items of extra content, embeds, comments and links, people: number of followers, distinct people contributing, number people marked as attending and views: number of views of the Cloudscape page, number of distinct people logged in and viewing Cloudscape Clouds, number of distinct guests (i.e. distinct IP addresses) viewing Cloudscape Clouds. Types of interactions have been collected and analysed, with a focus on those that may indicate increases of knowledge and understanding, and sense of community. Interactions will be categorised into the following types:
· informational (sharing of resources, links, annotations of presentations, live blogging, etc)
· practical (sharing of practice or experience)
· social (information modes of address, personal narratives, suggestions to recommendations), that lead or relate to:
o discursive (affirmations, welcome notes, supportive interchanges, humour and word plays, etc)
o deliberative (instigating debates, asking probing questions etc)
The Design-Based Research Approach
We are adopting a Design-Based Research (DBR) approach; starting with a stated problem we were trying to address, a proposed solution and then an iterative cycle of developments and evaluation. Design-Based Research has emerged in recent years as an approach for studying learning in context through systematic design and study of instructional strategies and tools (Brown, 1992; Collins, 1992 cited in Design-Based Research Collective, 2003). Wang and Hannafin (2005:5-6) define it as ‘a systematic, but flexible methodology aimed to improve educational practice through iterative analysis design, development and implementation, based on collaboration between researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories’. Reigeluth and An (2009:378-379) articulate the following set of characteristics of DBR. We map the ways in which we are addressing these principles for the design of Cloudworks.
1. It is driven by theory and prior research. In our work, we are building on the substantive body of prior research on instructional design, learning sciences, learning objects/Open Educational Resources and more recently learning design. The approach we adopt is socio-cultural in nature, with a focus on the design and use of a range of mediating artefacts involved in teaching-learning processes (See Conole, 2008 for a more detail account of this). Cloudworks is an example of a mediating artefact that can be used to facilitate the discussion and sharing of learning and teaching ideas.
2. It is pragmatic. Our aim is to develop tools and resources that are useful in actual practice by practitioners to address real educational challenges. Our intention is to be theory-driven, but pragmatic, recognising the complex, messy and often craft-based nature of teaching practice. Cloudworks has been designed based on a number of theoretical frameworks, including Englestrom’s (2005) notion of social objects and Bouman et al.’s (2007) concept of sociality.
3. It is collaborative.
We see working in close connection with end users as a vital part of our approach. Our initial interviews with teachers confirmed our view that teaching practice is complex and situated. Changing practice will only occur through close working with and understanding of practitioners’ needs. Cloudworks provides a space for practitioners to communicate (through the discussion spaces) and share/collaborate (through the notion of adding resources and references).
4. It is contextual. Our vision is to change actual practice, to achieve this it is important that the development activities occur in real, authentic contexts. In analysis of user behaviour in Cloudworks we are seeing examples of this; users are supporting each other in the development of knowledge co-construction; through building on the discussion of others and providing back up evidence to support arguments through the sharing of resources and links.
5. It is integrative. Wang and Hannifin (2005: 10) state that ‘DBR uses a variety of research methods that vary as new needs and issues emerge and the focus of the research evolves’. We have adopted a mixed-method approach (see below for more details) to evaluating our developments, matching the methods we use to the specific sub-research questions and the context that we are focusing on.
6. It is iterative. Our approach consists of an interactive cycle of identification of problems to be addressed, suggestion of proposed solutions, development, use, evaluation and refinement. Because user behaviour co-evolves in social and participatory media like Cloudworks, it is important that we adopt an iterative approach, with evaluation of emergent patterns of user behaviour informing future social and technical interventions developed for the site.
7. It is adaptive and flexible. Because our work is closely tied to actual practice, we need to ensure that the approach we are adopting is agile in nature, so that we can adapt based on evidence from changing practice. As above we need to be responsive to the ways in which users are using the space.
8. It seeks generalisation. In addition to the practical, pragmatic nature of our work, we are also attempting to develop a coherent underlying learning design framework of concepts and approaches. We believe the Community Indicators framework we have developed to inform the design and evaluation of the site has relevance for other social and participatory media. In addition, the underlying architecture could be applied for other topics around sharing and discussing ideas and to this end we now have an open source version of the site (available to download from https://bitbucket.org/cloudengine/cloudengine/wiki/Home).
Emergent patterns of behaviourAs a result of the new functionality and redesign we have seen a significant increase in use of the site, new patterns of user behaviour emerging and evidence of the site acting in distinct ways. It has been possible to identify eight types of Cloudscape emerging, although it is worth noting that some Cloudscapes fall into more that one category:
· Event Cloudscapes. The site provides an excellent mediation space pre-, during and post events. These can include both face-to-face and virtual events, such as one-day seminars, workshops and conferences. Conferences have also been a highly effective way of securing new community engagement. We know that using Cloudworks to support events is a highly effective way of introducing new users to the site and showcasing functionality. Clouds can focus around presentations; to enable live blogging about the talk, a shared discussion space, or a mechanism for aggregating related links or references. Clouds can also be set up to support workshop activities or to act as discussion spaces for particular topics. A recently added feature of the site is the list of ‘events’ (http://cloudworks.ac.uk/events/events_list). At the time of writing thirty-eight events are listed between May –December 2010, and fifty-two Cloudscapes have been labelled as past events. Cloudworks has been used to support more than 25 conferences this year. These Cloudscapes tended to grow organically as participants added Clouds and related materials, as they needed to, and consisted, primarily, of informational postings and archival content – for example live-blogs and links to papers.
· Debate Cloudscapes. A number of Cloudscapes have now been established acting as discussion spaces. ‘Flash debates’ are sparked from questions that aim to provoke and began to appear on the site in September 2009. Most typically a range of comments and activities will erupt almost immediately after initial postings, and will cross a variety of different communication platforms (e.g. Twitter, email lists, blogs, Facebook). The Flash Debate Cloudscape (http://cloudworks.ac.uk/cloudscape/view/1896) includes a range of topical issues such as ‘Citizendium versus Wikipedia’, ‘Has Twitter already peaked?’, or ‘What will the University of Tomorrow look like?’ The first example of this use was a cloud entitled ‘Is Twitter killing blogging?’ (http://cloudworks.ac.uk/index.php/cloud/view/2266). This was set up following a Twitter posting on this topic and had 719 unique views at the time of writing. Quickly the Cloud became a shared space for people to discuss the topic and to aggregate resources. Many of them then went to their own personal websites such as blogs to write more individual reflective pieces, posting links back in the Cloud:
Twitter is increasing the connections between us and in effect bringing more people into the conversation which can only be a good thing. The recent VLE-PLE debate is a great example of this. XX kicked off the latest round on his blog but it was his (& others) use of Twitter that brought people into the conversation, some of whom went on to blog, including myself, with that blog post I’d saving up since April (see above)!
The blog-twitter discussion was an appetiser for the Great VLE-PLE Debate™ at ALT-C 2009. Having eaten too much of the appetiser I opted out of that session but what has been great is the way I have been able to re-visit it thanks to Cloudworks”.
· Design Cloudscapes. Part of the original aspiration around the development of the site was to act as a channel for fostering more debate around design practices. A number of Cloudscapes have now been established that are focusing on learning and teaching issues around a particular course. These include spaces for those involved in designing courses (see for example http://cloudworks.ac.uk/index.php/cloudscape/view/1919) as well as those who have a tutoring role in delivering courses (http://cloudworks.ac.uk/index.php/cloud/view/3342). There are also some good examples of design collaboration (for example see http://cloudworks.ac.uk/index.php/cloudscape/view/688), however these examples have all been stimulated by a face-to-face learning design event, and are not yet happening spontaneously. To date, although there has been a great deal of very productive sharing of ‘snippets’ of practice on Cloudworks (discussing and sharing a new teaching tool, or a teaching and learning experience, or asking a tricky and interesting pedagogical question) there has been little sharing of what might be described as ‘worked designs’.
· Learning and teaching Cloudscapes. The site is also being used to some extent to support learners. For example students on the Masters in Open and Distance Education course at the OU have been exploring the site by taking part in a Cloudquest challenge (http://cloudworks.ac.uk/index.php/cloud/view/2699), contributing H800 flash debates (http://cloudworks.ac.uk/index.php/cloudscape/view/1937) and using the site to find relevant resources for particular teaching contexts (http://cloudworks.ac.uk/index.php/cloudscape/view/2057). In addition to these, there are a number of Clouds aggregating resources for informal professional development/ teacher education courses both across the HE and FE sectors. We recognise that use of the site in such courses will be important in supporting sustainability and use. We see Cloudworks as a space which offers excellent opportunities to engage learners in the learning design/ re-design process through sharing visualised designs and pedagogical discussion, checking assumptions and collaborative, co-creative development.
· Reading Cloudscapes. A relatively new type of Cloudscape to appear on the site is reading Cloudscapes. For example the 800-strong community of researchers interested in exploring students use of technologies have set up a space to aggregate and discuss relevant readings from the field (http://cloudworks.ac.uk/index.php/cloudscape/view/1968). Users post references to papers as Clouds and then use the Cloud to discuss the paper and add relevant links and references, essentially acting as a form of virtual reading circle.
· Resource or topic Cloudscapes. Cloudscapes have also been established that act as aggregators around particular topics or resources. Examples include the Horizon report Cloudscape (http://cloudworks.ac.uk/1957), the online research tools Cloudscape (http://cloudworks.ac.uk/cloudscape/view/2046) and the Learning Design toolbox (http://cloudworks.ac.uk/index.php/cloudscape/view/1882).
· Open reviews. The site has been used successfully a number of types to support ‘open reviews’, whereby a Cloudscape is set up to support a standard literature review process. The core questions being explored are posted, along with a space to aggregate resources and references. Examples include a review of the use of Web 2.0 tools in HE (http://cloudworks.ac.uk/cloudscape/view/1895) and a review of pedagogical models (http://cloudworks.ac.uk/index.php/cloudscape/view/2009). At the time of writing, the Web 2.0 Cloudscape has generated 465 views, and 9 comments. This includes descriptive content, an embedded video showcase and a link to a video that provided the inspiration for repurposing. Six months later, the same lecturer repurposed this Cloud as an entry for a virtual conference on teaching and learning that was organised by the Open University, and which was supported by Cloudworks. The new Cloud: ‘Experimenting with the pedagogy of creativity and openness’ has generated 256 views, and contains 9 comments, 3 embedded videos, and 6 references and links. This use of Cloudworks is similar to use observed on other social sites. Twitter for example enables ‘just-in-time’ learning moments where a query can be posted and several suggestions or explanations posted in response within minutes. This kind of interaction replicates the important and informal ‘coffee conversation’ that is such a core part of teacher practice. Sharing ideas and short snippets of practice is a very valuable way that teachers get new ideas, develop their practice and inform their learning designs (Conole, 2009).
· Expert elicitation and consultation. Finally Cloudworks works well as a space to elicit expert views around a topic or as a space to valid and discuss research outputs. One example was a literature review and expert elicitation around the role of educational technologists (http://cloudworks.ac.uk/cloudscape/view/1872). Another example was a major consultation process around Open Educational Resources and their associated practices (http://cloudworks.ac.uk/cloudscape/view/2105), following the gathering and analysis of a set of international OER case studies (http://cloudworks.ac.uk/cloudscape/view/2085) and articulation of a set of associated Open Educational Practice dimensions (http://cloudworks.ac.uk/cloudscape/view/2086).
Galley (2010b) maps these types of activities in terms of examples of evolving trajectories of use/activity in the site (Table 1). As can be seen from the table each type of activity has its own particular pattern of behaviour.
Table 1: Evolving trajectories in use/activity
Levels of activity
We are beginning to see evidence of the site being self-sustaining, with the emergence of Cloudworks champions both from within the university and outside who are actively using the site within their community. Use of the site has increased significantly; by the end of July 2010) there were ca. 3000 registered users, on average 4000 unique visitors per month from up to 165 countries each month. There has been steady increase in interest in Cloudworks with numbers of registered users moving from 1005 to 2997 between the launch of the Beta version in July 2009 to the end of phase-two in July 2010 (up more than 198%). The Cloudworks team have gradually reduced their facilitation and moderation of the site over the year, and as can be seen below activity levels have been maintained (not withstanding the seasonal fluctuations over the winter and summer breaks), with non-team members increasingly taking on ‘champion’ roles.
Cloudworks team intervention has been reduced over time and activity has not been significantly impacted but it is evident that Cloudworks team activity continues to impact on non-team activity – for example, when the team is active in creating Cloudscapes and comments in one month, non-team activity can be seen to rise in the following month. During phase-two of development the site was visited just over 90,000 times from 167 different countries with just over half of all visitors (54.26%) coming from the UK. Take-up in the Open University itself has been slow to be established but can be seen to be increasing as the site is used by more university groups, and for events such as the ‘Learn About Fair’ (http://cloudworks.ac.uk/cloudscape/view/1963) a university staff development event which received 3164 distinct guests (i.e. distinct IP addresses), 179 distinct people logged in and viewing Clouds, and 22 active participants, and the ‘Open University annual Learning and Technology conference’ (http://cloudworks.ac.uk/cloudscape/view/2012) which received 4417 unique guests, 474 distinct people logged in and viewing Clouds, and 54 active participants. During phase-two, the numbers of registered users who told us on registering that their institution was the Open University rose from 208 to 651 (up 213%). However, a number of Open University communities can be seen to be using the site. Some examples include:
· Mobile technologies special interest group http://cloudworks.ac.uk/cloudscape/view/1889, 26 distinct people commenting, 1922 distinct IP addresses viewing.
· Olnet http://cloudworks.ac.uk/cloudscape/view/562, 60 distinct people commenting, 2489 distinct IP addresses viewing.
· eLearning Community http://cloudworks.ac.uk/cloudscape/view/899, 3 distinct people commenting, 164 distinct IP addresses viewing.
· Associate Lecturers http://cloudworks.ac.uk/cloudscape/view/1934, 6 distinct people commenting, 446 distinct IP addresses viewing.
· Classical Reception Studies Network (CRSN) Learning & Teaching Group http://cloudworks.ac.uk/cloudscape/view/21678, 8 distinct people commenting, 183 distinct IP addresses viewing.
· K802 Students and staff http://cloudworks.ac.uk/cloudscape/view/2171 8 distinct people commenting, 1009 distinct IP addresses viewing http://cloudworks.ac.uk/cloudscape/view/2161 7 distinct people commenting, 294 distinct IP addresses viewing.
· Teaching and Learning Librarians http://cloudworks.ac.uk/cloudscape/view/2035 22 distinct people commenting, 2353 distinct IP addresses viewing
Alevizou et al., (2010) looks in more detail at the use of the site by one Open University led group (OLnet) for sharing and the discussion of issues relating to the use and uptake of Open Educational Resources (OERs). They conclude that there is substantial evidence to indicate that Cloudworks is being used by this as a means of sharing and discussing and adopting more scholarly and evidence based approaches to practice.
Applying the Community Indicators Framework
In this section we will show how the Community Indicators framework described above can be applied to the evaluation of the Cloudworks site across the different types of Cloudscapes described in the previous section. Table 1 provides examples of evidence of the four Community Indicators and demonstrates examples evident from our evaluation of Cloudworks.
Table 2: Community Indicators in Cloudworks
The previous section gives some indication of the breadth and richness of the types of activities we are seeing on the site. New practices are emerging as users begin to colonise and appropriate sections of the site for their own interests. We are beginning to apply additional theoretical framework to gaining an understanding of this new patterns of behaviour, such as Goffman’s notion of ritual performance (Goffman, 1974), the concept of collective intelligence (see for example Lévy, 1997) and expansive learning (Engestrom, 1987); see Alevizou et al. (2010) for more on how we are using these frameworks.
Early evidence suggests, that Cloudworks is one of the sites blurring formal and informal cultural and networked learning about being an educationalist, scholar, practitioner or indeed a learner (in limited examples) with online interactions and experiences allowing roles to be learned, experiences to be shared, values to be exchanged and – to an extent – identities to be performed and (re)shaped, and communities to gather (Alevizou et al., forthcoming). It is too early in our research to demonstrate empirically more than glimpses of emerging patterns but we have now developed clear ideas about research questions that will inform Cloudworks position within this landscape of practice, as well as guide implications for further systematic research.
We continue to recognise the complexity and challenges inherent in supporting and promoting a collaborative and open approach to design and reflection in learning and teaching practices, but also would argue that mechanisms to facilitate these are essential if learners and teachers are going to develop the necessary new literacy skills they will need in order to harness the potential of new technologies. We have argued in this paper that Cloudworks is a platform where we are starting to see evidence of expressive interactions, crowdsourcing and archiving of issues relating to learning designs and the process of design. We are also staring to see new connections and interactions emerging within Cloudworks (on a given time, for a given purpose, or randomly and serendipitously), which we believe are key in supporting the dialogic and creative process of design. The idea of Cloudworks functioning as a hub between several virtual and physical learning design spaces is both powerful and vi
sible: we have pointed to evidence whereby designs can be seen to be both negotiated and improved. We do however recognise that we have significant work to do in encouraging and supporting designers in sharing, discussing and archiving worked designs, and promoting the shifts in culture and practice necessary for many educational practitioners in order to achieve this, and benefit from it.
This paper has attempted to consider the challenges associated with rapidly evolving social and participatory media and has argued that the design and evaluation of such sites needs new approaches. We have chosen to adopt a Design-Based Research approach to designing Cloudworks and have introduced the Community Indicators framework as a mechanism for evaluating user behaviour in the site. We have also strengthened our understanding of interactions on the site through the development and use of the Community Indicators framework described in this paper. We think the Community Indicators framework has a number of benefits. Firstly, it is built on relevant research literature on the development and sustainability of online communities, drawing on related frameworks, but extending beyond these in order to support communities in new social and participatory media. Secondly, it provides a framework for designing social and participatory sites. Thirdly, the framework can be used to map emergent and evolving patterns of user behaviour on the site. We think the framework could be useful in terms of designing and evaluating other social and participatory sites.
The OU Learning Design Initiative was funded through strategic funding from the OU and also the JISC as part of the Curriculum Design programme.
Alevizou, P., Conole, G., Culver, J. and Galley, R. (2010), Ritual performances and collective intelligence: theoretical frameworks for analysing emerging patterns in Cloudworks, Networked Learning Conference Proceedings, Aalborg, Denmark, 3-4th May 2010.
Alevizou, P., Conole, G. and Galley, R. (2010), Using Cloudworks to support evidence-informed OER activities, Report for the HE Academy commissioned project PearlsintheClouds, The Open University: Milton Keynes.
Alevizou, P., Galley, R. and Conole, G. (forthcoming), Innovating design for learning in the networked society, in L. Dirckinck-Holmfeld, V. Hodgson and D. McConnell (eds.), ‘Exploring the Theory, Pedagogy and Practice of Networked Learning’, Springer: New York.
Bouman, W., Hoogenboom, T., Jansen, R., Schoondorp, M., de Bruin, B. and Huizing, A. (2007), The Realm of Sociality: Notes on the Design of Social Software, PrimaVera Working Paper Series, Amsterdam: Universiteit Van Amsterdam, http://choo.fis.utoronto.ca/fis/courses/lis2176/Readings/bouman.pdf, [7/8/08]
Brown, J.S., and Duguid, P. (2001) Knowledge and Organization: A Social-Practice Perspective. Organization Science. 12(2), 198 – 213.
Clark, P. and Cross, S. (2010), Findings from a series of staff interviews about learning design, representations of design, design process, evaluation and barriers, Appendix 2, The Learning Design Initiative Institutional Project Phase two Report – embedding learning design and establishing a reversioning culture, The Open University: Milton Keynes
Cockburn, A. (2001), Agile Software Development, The agile software development series, A. Cockburn and J. Highsmith (eds), Addison Wiley: New York
Conole, G. (2010). Learning design – making practice explicit, keynote and paper, . Paper presented at the ConnectEd – 2nd International conference on design education. Retrieved from http://cloudworks.ac.uk/cloud/view/4001
Conole, G., Brasher, A., Cross, S., Weller, M., Clark, P. and White, J. (2008), Visualising learning design to foster and support good practice and creativity, Educational Media International, Volume 54, Issue 3, 177-194.
Conole, C. G. and Culver, J. (2009) Cloudworks: Social networking for learning design. Australasian Journal of Educational Technology 2009, 25(5), 763-782. [last accessed 19/11/09]
Conole, G. and Alevizou, P. (2010), A review of the use(s) of Web 2.0 tools in Higher Education, The Open Univesity: Milton Keynes.
Conole, G. & Culver, J., 2010. The design of Cloudworks: Applying social networking practice to foster the exchange of learning and teaching ideas and designs. Computers & Education. Available at: http://www.ascilite.org.au/ajet/ajet25/conole.html [Accessed November 24, 2009].
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for
educational inquiry. Educational Researcher, 32(1), 5-8.
Dron, J. and Anderson, T., 2007. Collectives, networks and groups in social software for e-Learning. In Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education Quebec. Retrieved Feb. p. 2008.
Engeström, J. (2005), Why some social network services work and others don’t — Or: the case for object-centered sociality, blog posting, 13th April 2005, http://www.zengestrom.com/blog/2005/04/why_some_social.html [1/8/08].
Engeström, J. (2007), Microblogging – tiny social objects on the future of participatory media, cited in K. Anderson (2008), blog entry, 13th June 2007, http://strange.corante.com/archives/2007/06/13/nmkforum07_jyri_of_jaiku.php, [7/8/08]
Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research (Helsinki, Orienta-Konsultit).
Fischer, G., (2002) Communities of Interest: Learning through the Interaction of Multiple Knowledge Systems [7/08/10] http://l3d.cs.colorado.edu/~gerhard/papers/iris24.pdf
Galley, R. (2010a), Indicators of community – a framework for evaluating relational and transient communities on Cloudworks, OULDI workshop paper, 19th March 2010, The Open University: Milton Keynes.
Galley, R. (2010b), Summary report for Cloudworks development – phase one and two, JISC OULDI report, The Open University; Milton Keynes.
Garrison, D. R., Anderson, T., and Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105.
Goffman, E.(1974). Frame analysis. Middlesex: Penguin Books.
Jenkins, H., 2009. Confronting the challenges of participatory culture: Media education for the 21st century, Mit Pr.
McAndrew, P., Santos, A.I., Lane, A., Godwin, S., Okada, A., Wilson, T. Ferreira, G.; Buckingham Shum, S.; Bretts, J. and Webb, R. (2009), OpenLearn Research Report 2006-2008. The Open University, Milton Keynes, England. available online at http://oro.open.ac.uk/17513/, last accessed 21/04/10
Reigeluth, C.M. and An, Y. (2009), Theory building, 3865-386, in C.M. Reigeluth and A.A. Carr-Chellman (Eds), Instructional-design theories and models – building a common knowledge base, Volume III, RoutledgeFalmer: Oxford.
Wilson, P. (2007), ‘Progress report on capturing eLearning case studies’, Internal report, The Open University: Milton Keynes.
Wang, F. and Hannafin, M. (2005), Design-based research and technology-enhanced learning environments, Educational Technology Research and Development, 53(4), 5-23.
Wenger, E. (1998). Communities of practice: learning, meaning, and identity. New York: Cambridge University Press.
Wittel, A. (2001) Toward a Network Sociality Theory, Culture & Society 2001 (SAGE) 18(6), 51-76.