Computing and Library Services - delivering an inspiring information environment

Between analysis and transformation: technology, methodology and evaluation on the SPLICE project

Johnson, Mark, Lager, Peter, Pollard, Bill, Hall, Graham, Edwards, Miranda, Whitfield, Pete and Ward, Rupert (2009) Between analysis and transformation: technology, methodology and evaluation on the SPLICE project. In: "In dreams begins responsibility" - choice, evidence, and change. Conference Proceedings, 16th International Conference, 8-10 September, Manchester, UK. Association for Learning Technology, pp. 40-50. ISBN 978 0 95 458 709 3

[img] PDF (Splice Project - Research Paper 0084) - Published Version
Available under License Creative Commons Attribution Non-commercial.

Download (2MB)


This paper concerns the ways in which technological change may entail methodological development in e-learning research. The focus of our argument centres on the subject of evaluation in e-learning and how technology can contribute
to consensus-building on the value of project outcomes, and the identification of mechanisms behind those outcomes.

We argue that a critical approach to the methodology of evaluation which harnesses technology in this way is vital to agile and effective policy and strategymaking in institutions as the challenges of transformation in a rapidly changing educational and technological environment are grappled with.

With its focus on mechanisms and multiple stakeholder perspectives, we identify Pawson and Tilley’s ‘Realistic Evaluation’ as an appropriate methodological approach for this purpose, and we report on its use within a JISC-funded project on
social software, SPLICE (Social Practices, Learning and Interoperability in Connected Environments). The project created new tools to assist the identification of mechanisms responsible for change to personal and institutional technological practice. These tools included collaborative mind-mapping and focused questioning, and tools for the animated modelling of complex mechanisms. By using these tools, large numbers of project stakeholders could engage in a process where they were encouraged to articulate and share their theories and ideas as to why project outcomes occurred. Using the technology, this process led towards the identification and agreement of common mechanisms
which had explanatory power for all stakeholders.

In conclusion, we argue that SPLICE has shown the potential of technologicallymediated Realistic Evaluation. Given the technologies we now have, a methodology based on the mass cumulation of stakeholder theories and ideas about mechanisms is feasible. Furthermore, the summative outcomes of such a process are rich in explanatory and predictive power, and therefore useful to the immediate and strategic problems of the sector. Finally, we argue that as well as generating better explanations for phenomena, the evaluation
process can itself become transformative for

▼ Jump to Download Statistics
Item Type: Book Chapter
Subjects: L Education > LC Special aspects of education > LC5201 Education extension. Adult education. Continuing education
L Education > LB Theory and practice of education > LB2300 Higher Education
Schools: School of Computing and Engineering
School of Computing and Engineering > Pedagogical Research Group
Related URLs:
References: Archer, M. (2000) Structure, Agency and The Internal Conversation. Cambridge Argyris, C. and Schön, D. (1974) Theory in practice: Increasing professional effectiveness, San Francisco: Jossey-Bass. Beer, S. (1981) Brain of the Firm Second Edition, John Wiley, London and New York. Bhaskar, R. (1979) A Possibility of Naturalism Sage Bhaskar, R. (1975) A realist theory of science Sage Beer, S. (1994) Beyond Dispute: the invention of team syntegrity Wiley Campbell, D. T. and Stanley, J. C. (1963) Experimental and Quasi-Experimental Designs for Research Chicago: Rand McNally. Checkland (1990) Soft Systems Methodology in Action Wiley Eden, C. and Ackermann, F. (1989) Strategic options in development and analysis (SODA) — using a computer to help with the management of strategic vision Knowledge based management support systems Glaser, B.G. and Strauss, A. (1967) Discovery of Grounded Theory. Strategies for Qualitative Research. Sociology Press Harré, R. (1984) Personal Being: A Theory for Individual Psychology Harvard University Press JISC (2007) Six Steps to Effective Evaluation: a handbook for programme and project managers [accessed on May 22nd 2009] Johnson, M. and Liber, O. (2008) The Personal Learning Environment and the Human Condition: from Theory to Teaching Practice Interactive Learning Environments, v6, no. 1 Luhmann, N. (1995) Social Systems Stanford University Press Mingers J. (2004) Realising Systems Thinking: Knowledge and Action in Management Science Springer Pawson, R. and Tilley, N. (2004) Realistic Evaluation Sage
Depositing User: Rupert Ward
Date Deposited: 14 May 2010 11:49
Last Modified: 28 Aug 2021 22:18


Downloads per month over past year

Repository Staff Only: item control page

View Item View Item

University of Huddersfield, Queensgate, Huddersfield, HD1 3DH Copyright and Disclaimer All rights reserved ©