Networked Learning Conference 2004

NLC2004 /Proceedings / Individual Research Papers

Is the problem with implementing Managed Learning Environments social, technical or educational?

Sarah Holyfield, University of Wales Bangor, s.holyfield@bangor.ac.uk

Professor Oleg Liber, Bolton Institute, o.liber@bolton.ac.uk

Abstract

The concept of the Managed Learning Environment (MLE) has evolved over the last few years to describe the vision of a rich and ‘seamless’ environment for learning, however the complexity of the task and the implications for organisations are rarely fully recognized. In the business world there is a large literature and body of research in the implementation of technology and its impact on the business processes of organizations. The use of a systems approach to understanding how organizations work, and a socio-technical approach to the implementation of technical systems can offer valuable insights and help us to ensure that MLEs deliver the promise of enhanced student learning.

Introduction

At the start of the new millennium, the Higher and Further Education sectors in the UK are under greater pressure than ever to change. They are expected to respond to a range of demands including teaching increasing numbers of learners with wide ranging backgrounds and needs, offering courses at all sorts of times and in all sorts of ways, competing with courses offered across the globe, and across the traditional boundaries of HE and FE, and doing all this on shrinking levels of funding. Alongside these pressures there is the central desire to provide an enhanced learning experience.

The last few years have seen enormous interest in, and experience with, Virtual Learning Environments (VLEs), and it is becoming possible to provide technological solutions to many of the above demands involving linking VLEs with other central systems. Achieving this, however, is not a simple process. The recent DFES consultation paper ‘Towards a Unified e-learning Strategy’ (DfES 2003) recognizes and explores this complex area, and there now are enormous pressures on institutions to implement technological solutions and provide ‘joined up’ and ‘seamless’ systems for learners.

A considerable level of understanding is developing of the pedagogical implications of supporting the learning process with technology, and technology itself is more tractable in that achieving technical interoperability is well elaborated. But the impact on the structure and processes of educational organizations, in our view, is not well understood. We argue in this paper that the use of a systems approach to understanding how organizations work (e.g. Beer 1985), and a socio-technical approach to the implementation of technical systems (e.g. Mumford 1983) are proving to be highly significant and can offer valuable insights for the educational world. Technology change will have a profound social, cultural and structural impact, and any process of implementation must take account of this and build it into its methodology. This paper examines these theoretical approaches and applies them to illuminate the concept of the Managed Learning Environment (MLE), using the experience of MLE development in the UK over the last three years, in order to explore how they can help us ensure that MLEs deliver the promise of enhanced student learning.

Background

The term Managed Learning Environment (MLE) was first used by JISC in 1999, and the concept has evolved over the last few years. It is still the case that there is no commonly agreed definition for the term, other than that it is concerned with technology, administration, the learning process and the organization itself. There is also still some confusion over the distinction between an MLE and a VLE. A VLE is specifically concerned with the learning process and only with other systems in as far as the VLE exchanges data with them. The MLE is concerned with whole institutional systems, such as Student Record Systems, Library Systems, Management Information Systems (MIS), VLEs, timetabling systems and so on, and the joining up or interoperation of some or all of these. As a further illustration of the difficulty of defining the concept, a number of experts in the field gathered together in the summer of 2002 to explore the possibility of producing a definitive diagram for an MLE. It was recognized that there was a wide variety of perspectives, and there were inherent issues with the process of diagramming itself, yet diagrams were a powerful tool in the process of developing a shared understanding of the concept (Holyfield 2003).

Significant effort has been expended in recent years across UK HE, FE and beyond to achieve interoperability between existing institutional systems to support learning and the management of learning. These have included the exchange of information between administrative, library systems and emerging Virtual Learning Environments, and the development of institutional or learner-focused portals providing access to a range of resources. It has been reported that “Over 70% of institutions are currently engaged in some kind of MLE development activity” (JISC Study of MLE activity, 2003).

The JISC MLE Programme[i] has funded a wide range of exploratory projects in the UK in this area; and whilst much work took place focusing on technical integration, associated with this work were a number of studies which observed the process and drew out the lessons to be learnt. Almost all projects concluded that the technical part was relatively easy compared to addressing the organizational and cultural implications and recommended that anyone developing an MLE should focus great attention on these areas. In her report to JISC Boys concluded that - “MLE development inherently demands a level of cross-institution collaboration and commitment far beyond other ICT projects. It is thus essential that the HE/FE sector both has guidance on the implications of MLE development in terms of organisational, pedagogic and technical re-structuring, and that any future projects are more firmly embedded in their institutions from the start” (Boys 2002). The work on MLEs funded by JISC has been very substantial and has produced a rich seam of data to be mined, and forms the basis of much of the rest of this paper.

Most UK HE and FE institutions are increasingly tending to adopt commercial systems for their VLEs and administrative systems. Vendors of these systems have themselves been significantly affected by the emergence and growth of the Internet. VLE vendors, often new companies selling new products, have experienced rapid growth and have been subject to significant changes as larger companies have bought their stock and influenced their strategic directions. Some, like FirstClass, have been taken over more than once. Others have expanded their system’s functionality to provide more administrative functions (e.g. WebCT Vista) as they compete to take more market share. New systems offering support for different pedagogic approaches are emerging (e.g. Moodle, LAMS). Open Source systems and tools are becoming more sophisticated, providing increasing flexibility (e.g. Plone). This constantly changing situation in the system provider community adds to the problems institutions face in trying to develop and implement their eLearning strategies and to join up systems.

Whilst the MLE appears to offer an attractive vision, and one that could offer solutions to a wide range of education’s problems, there is still no clear model for technical integration, organizational change or for understanding the pedagogical possibilities and implications[ii] . Implementing MLEs is new territory for all institutions, and the complexity of the task and the implications are rarely fully recognized. There is a large literature and body of research on the implementation of technology and its impact on the processes of organizations in the commercial world and in other parts of the public sector, and vast sums of money are spent on this process; there is also, however, a great deal of concern about the associated extremely high levels of failure[iii] .

This experience is not traditionally seen as of relevance to the educational community, perhaps because education is considered by some as involving far more complex processes than business, however we suggest that it is highly relevant, especially because of the high level of risk in the context of a great shortage of funds, and perhaps unrealistic ideas about the role technology can play.

Why a systems approach

Systems theoretic approaches to organizational change have been around for several decades, and there are a number of varieties, including systems dynamics (e.g. Senge 1990), soft systems methodology (SSM) (e.g. Checkland 1981), management cybernetics (e.g. Beer 1981), total quality management (TQM) (e.g. Deming 1993) and socio-technical systems design (STS) (e.g. Mumford op. cit.). What they have in common is the view that organizations have to be treated as a total system, and that interventions made in any part of an organization will have effects elsewhere. Introducing technical systems can destabilize existing procedures, and lead to a loss of effectiveness. In their seminal paper on the British Coal Industry, Trist and Bamforth (1951) elaborated how the introduction of automation reduced productivity – the increased noise of machines interfered with communication within teams of miners. This led to the formulation of the now well established socio-technical systems design approaches (e.g. see Trist and Emery 1960).

Socio-technical systems design was popular until the 1980s, when it was overtaken by Business Process Re-engineering (BPR) (Hammer and Champy 1993), a radical and aggressive approach to organizational change through technology adoption. However despite its popularity, there have been few successes based on BPR, and critics have suggested that the lack of concern for social aspects of organizational operation has been the reason (Mumford 1994). In recent years there has been a renewed interest in STS: for example, the British Computer Society now has a thriving Special Interest Group for STS[iv]. The key tenets of STS are that technical systems introduction needs to be accompanied by organizational change; that everyone needs to be involved in the change process; and that all stakeholders must experience benefit from the change. These ideas underpin the work elaborated in the rest of this paper.

Applying a socio-technical approach to MLE development

In 2003 it was recognized by JISC that their MLE Programme had generated a great deal of valuable experience and materials, and that it was important to make these accessible to those who were embarking on the route to an MLE. A pack of materials was produced which would gather these case studies, tools and experiences together within a framework which was informed by a socio-technical perspective (JISC Creating a Managed learning Environment Infokit 2003). This framework proposed a lifecycle approach to the process of developing an MLE, whilst recognizing that this would be an iterative and ongoing process, and individual contexts will vary widely. It applied the principles of STS design.

Seven key stages were identified for the framework moving from the first stage of deciding whether an MLE is a good idea or not, through the process of understanding how things are done at present, to deciding by a process of requirements gathering what the MLE should ideally be able to do and enable. The technological options should be considered and evaluated, then the MLE can be designed taking into account the technologies chosen, the requirements identified and the identity of the organization itself. Finally the design can be implemented and then embedded within the organisation. Whilst this appears to be a serial model, there should in fact be iterations all through the process, earlier stages should be revisited at any time, and once the embedding stage is reached the first questions should be revisited. As there is now so much experience in this territory, this analysis enables the identification of case studies, tools and other relevant materials to be gathered, and not just from the education sector, which will support those involved in this work and help develop a community of practice.[v]

Each of these stages is briefly described below.

Why an MLE?

There are many drivers, including political and peer pressure, on FE Colleges (FECs) and HE Institutions (HEIs) to develop an MLE, but it is essential that each institution finds a way of asking itself about its identity, whether it really wants to take this route, what it would gain from doing so, and what the costs and risks might be. The drivers may be concerned with efficiency and effectiveness, and most institutions in surveys have indicated that their main concern is to improve the learning experience of the students (JISC MLE Study op.cit.). However, as already discussed in an earlier section of this paper, the introduction of technology based systems may lead to all sorts of unpredictable outcomes, so it is essential that everyone who will be affected by the MLE, should be involved in the discussion from the beginning. Mumford’s ETHICS method provides a methodology for ensuring that ‘the future users of computer systems at all organizational levels play a major part in the design of these systems' (Mumford, op. cit.). Whilst this widespread consultation is of critical importance, it is necessary to also make the process manageable and so representatives should be identified for all stakeholder groups, and processes for working should be decided, perhaps involving working groups, committees and so on. The ultimate aim is a shared understanding about what the MLE will be like, and what it will mean for their organization.

How does our organisation do things now?

Once this consensus has been reached, the next stage must involve developing a good understanding of how things are done now in order to see what needs to change and where technology might fit in. Organizations already have a well elaborated set of systems and procedures for management, administration, service provision and teaching/research, some of which include technical systems. In many cases these have evolved over time, and are firmly grounded in the day-to-day practice of members of the organization, including how teaching and learning takes place. Any change needs to be based on a rich knowledge of these from multiple perspectives. Everyone is familiar with systems they use which depend on conversations with administrators at their desks, or with lecturers in corridors, and which help to provide ‘work-arounds’ if the formal system doesn’t seem to work smoothly, however it is not easy to capture this process.

There is a need to develop models of ‘how things work now’, and these need to provide a simplified but inclusive picture. A wide range of techniques are available for modeling processes and systems; some forms of systems analysis which develop relatively simple flow charts will miss vital information, but systems theory approaches such as Checkland’s Soft System Methodology (op. cit.) and Beer’s Viable System Model (op. cit.) are particularly useful. It is also essential to understand wider organizational questions such as what the drivers and barriers to development might be, and the wider national and political context in which the organization finds itself. Again the question arises about the process of conducting and coordinating this work, who should be involved, and how everyone will be able to participate.

How can we find out what we want?

Once it has been determined that the institution wants an MLE, and it has a clear picture of how it currently operates, it is necessary to begin a process of establishing the detailed requirements of all the stakeholders – teachers, managers, service and support staff, students and funding bodies and so on. These need to be gathered in a process which develops mutual understanding, but which results in a specification of the functionality of the system to be implemented. This is a subtle and difficult task, and there is a rich history of this being badly done, evidenced by the many IT projects that fail or go way over budget.

There are many methods of requirement gathering in the software world (e.g. see Beynon-Davies 2002). UML provides a language for capturing and expressing requirements, and socio-technical approaches ensure that there is wide participation across the institution. The danger is that new systems are designed by people for others to use without involving them, or being aware of what the ‘knock-on’ effects might be.

A draft set of requirements for an MLE for discussion purposes may be found in Holyfield’s report to JISC (op.cit) which illustrates the complexity of this process.

What are our technical options?

Any development needs to be grounded in what is possible and affordable, it needs to reflect what the organisation wants to achieve but also be feasible. Before any decision is made concerning technologies, there needs to be an understanding of what current commercial systems do. For example, many Student Record Systems bring with them a range of other tools, including discussion boards, course gazettes and others. This extension of functionality, which is a result of increasing competition between suppliers, leads to overlap between systems from different categories, for example between record systems and VLEs, and can lead to implementation difficulties. Also, several companies have formed alliances, and others have adopted standardized data formats (e.g. IMS specifications), which affect the costs of integration significantly.

Some of the basic questions to be asked will be whether to develop in-house or purchase from a commercial vendor. Other issues include how the technologies will integrate with existing systems at the institution and any partner institutions, how they will meet the requirements and adapt with these as they change, the total costs including adaptation, support and training, and the long term viability of these systems. Developments presently taking place include the creation of a technical framework and components approach thus illustrating the speed of technological change.

Design

Once the requirements have been established, and the technical options explored it is now possible to begin to design an MLE to realize these requirements. Part of this involves producing a model of the planned MLE. Models typically consist of a number of diagrams that represent the organization structure, a process model, and a number of technical models, with mappings between these. Ultimately these need to result in the description of an integrated system of component subsystems, and how they interoperate. . This process of design will inevitably lead to changes in job roles, including the loss of some roles and the creation of new ones. This process must therefore also include wide consultation to ensure that the new systems will really do what people expect and need, and these changes are acceptable and potentially rewarding.

Firstly, what is in a design? The answer is that there is never only one design but a whole raft of different views of the architectural whole. In the world of architecture, for a new house, we could expect to see an overview for the clients, a plumbing diagram for the plumbers, a wiring diagram for the electricians and so on. The problem with an MLE is that it is not one single thing but a series of agreements between systems, all of which may be evolving at the same time. (Newton-Ingham 2003)

Whilst the design process will produce specifications and documentation, it is also a highly iterative process which must involve all stakeholders.

Implementation

If preparation has been well undertaken, implementation should lead to a successful system being in place. There may be different approaches to implementation: for example an incremental approach to change or the design and development of new systems. Any implementation will mean the production of a deployment strategy which is likely to involve a phased approach to the integration of all the systems involved. It will involve a range of working relationships, will need to pool expertise, and requires committed involvement from senior management. Implementation is a highly political process, because MLE development cuts across traditional organisational boundaries. There are likely to be several different groups involved in eLearning development, of a formal or informal kind - with different cultures, different perspectives and priorities.

The issues will include who should be involved and how the project will be managed (e.g. Steering Group and Project Team), the wide range of legal requirements (disability, data protection etc), how support will be provided (service level agreements etc), establishing effective dialogue between all those involved, and ensuring the reliability of these ‘mission critical’ systems.

Embedding

Successful implementation of an MLE represents a significant change in the way an institution carries out its primary activities, and will result in changes in the practice of almost all staff, academic or service/support. Inevitably, this will lead to further change in light of experience, as strengths and weaknesses emerge, and as technology changes. This requires that an institution is able to adapt its strategies to be able to support evaluation and on-going development, and to be adaptable to change. It also needs to put in place processes that allow new staff and students to be inducted into the full use of the MLE.

The embedding of the MLE will involve managing change in all the areas affected from administration through to teaching and learning, and widespread staff development. The MLE for each institution will have its own character, and there will be many new possibilities. There will need to be new policies, strategies and protocols and constant review and evaluation.

Summary

No institution has yet achieved a fully functioning MLE although many are well on the way. As already indicated, the process will not end with the embedding of a system which will then remain static for a long period of time. Change will be a constant factor and must be anticipated and prepared for, which is why the stages outlined above will have to be regularly revisited. The common factor stressed throughout is a focus on how the organisation works, its identity, and the people who bring it into being. The use of a systems approach enables an organisation implementing technology to take a holistic approach (Beynon-Davies (op. cit.) to the process and ensure that the new systems in place are of benefit to everyone.

References

Beer, S. (1981) Brain of the Firm. Wiley, London,

Beer, S. (1985) Diagnosing the System for Organizations. Wiley, London

Beynon-Davies, P. (2002) Information Systems: An Introduction to Informatics in Organisations. Palgrave, Basingstoke

Boys, J. (2002) Managed Learning Environments, Joined up Systems and the Problems of Organisational Change: http://www.jisc.ac.uk/index.cfm?name=mle_related_joined

Checkland, P. (1981) Systems Thinking, Systems Practice: Wiley, London

Deming, W. Edwards (1993), The New Economics For Industry, Government, Education. MIT Press, Cambridge USA

DfES (2003) Towards a Unified e-learning Strategy http://www.dfes.gov.uk/consultations2/16/)

Hammer, M. and Champy. J. (1993) Reengineering the Corporation : a Manifesto for Business Revolution. Allen & Unwin, St Leonards, NSW

Holyfield, S. (2003) Developing a Shared Understanding of the Managed Learning Environment – the Role of Diagramming and Requirements Gathering: http://www.jisc.ac.uk/index.cfm?name=mle_diagram_report

JISC (2003) Study of MLE activity, http://www.jisc.ac.uk/project_mle_activity.html

JISC (2003) Creating a Managed Learning Environment http://www.jiscinfonet.ac.uk/InfoKits/creating-an-mle

Mumford, E. (1983) Designing Human Systems – the ETHICS Method: Manchester Business School, Manchester, England.

Mumford, Enid (1994): New Treatments or Old Remedies: Is Business Process Reengineering Really Socio-Technical Design?, Journal of Strategic Information Systems, 3 4

Newton-Ingham, G. (2003) Designing an MLE in The Creating an MLE Infokit, http://www.jiscinfonet.ac.uk/InfoKits/creating-an-mle/mle-6-design

Senge, Peter M. (1990) The Fifth Discipline : the Art and Practice of the Learning Organization. Doubleday/Currency, New York,

Trist, E. and Bamforth, K. (1951) Some Social and Psychological Consequences of the Longwall Method of Coal Getting, Journal of Human Relations 4 pp 3-38;

Trist, E. and Emery, F.E. (1960) Socio-technical Systems, in Churchman, C.W. & Verhurst, M. (Eds), Management Science, Models and Techniques Vol 2. Pergamon Press: London.