Systemic Evaluation Methodology: The Emergence of Social Learning From Environmental ICT Prototypes

This paper investigates why and how systems approaches can help in evaluating the design of new Information and Communication Technologies (ICTs) as social learning platforms. It focuses on the prototypes created by the research project Virtualis, whose objective is to promote social learning on environmental concepts and practices amongst a variety of stakeholders. The paper presents the principles of systems thinking and practice that did help in formulating such evaluation processes. It illustrates how both a peer systemic evaluation process (within the research team) and a participatory evaluation process (involving potential future users of the ICTs) were carried out.     

You are Seldom Born with a Drum Kit in Your Hands: Music Teachers’ Conceptualizations of Knowledge and Learning Within Music Education as an Assessment Practice

Due to an increased demand for evaluation and accountability, the focus on assessment in public education has become stronger. Already leading to teachers’ deprofessionalization, another risk is assessment leading to criteria compliance and becoming a tool for measuring teaching quality. Those whose learning is affected are thereby not only students, but also teachers. One major factor to restore professionalism and focus assessment on learning is to change practitioners’ ways of thinking and not only their behaviour. Therefore, a group of seven music teachers and a researcher in music education have carried out a research and development project using participatory action research as approach as well as method. This article explores and presents these upper secondary school music teachers’ conceptualizations of musical knowledge, learning and educational communication working with peers. Using Deweyan pragmatism as a lens to interpret the qualitative data, the results show that while a professional language does exist, the music teachers’ conceptualizations of musical knowledge and learning and educational approaches differ. A key aspect for the teachers to develop their professional language, concepts and assessment practices is the teachers’ opportunities to communicate—both factual as well as perceived.     

An Analytical Framework of Social Learning Facilitated by Participatory Methods

Social learning among different stakeholders is often a goal in problem solving contexts such as environmental management. Participatory methods (e.g., group model-building and role playing games) are frequently assumed to stimulate social learning. Yet understanding if and why this assumption is justified is quite limited. Difficulties arise from the complexity and context-dependence of processes influencing social learning. Furthermore, continuing discussion of the exact meaning and theoretical basis of social learning result in a limited capacity to assess and evaluate whether social learning has occurred. In this paper we introduce an analytical framework to develop an in depth understanding of essential processes underlying social learning facilitated by participatory methods. Concepts from different fields of science are discussed and integrated, including resource management, small group research and learning research. The individual and group perspectives are brought together via mental models and emergent roles. We added the direction of learning, being either convergent or divergent, to be able to explore if and when personal views on a problem converge into a shared understanding of a problem. The analysis of convergence and divergence of learning is facilitated through the use of the mental model concept. Methods for measurement of proposed indicators for social learning are also discussed. The framework developed provides a conceptual basis for the analysis of social learning facilitated by participatory methods and an operationalization for application in empirical research.     

Using Action Research to Implement Selective Waste Collection Program in a Brazilian City

The objective of this paper is to study and report upon the elaboration of a selective waste collection program in a Brazilian city through action research; the overarching goal is that all knowledge generated may serve as a basis of information and reference to aid in decision making for similar programs globally. The municipality chosen was São Lourenço in the state of Minas Gerais. The implementation of the selective waste collection program happened over the course of multiple improvement and learning cycles, and enabled many environmental, economic and social benefits for all of the project’s stakeholders. Eighty-nine tons of potentially recycled material were sent to recycling processes rather than being disposed of in a landfill. No great difficulties were observed for the use of action research as a method, but some challenges were encountered and overcome during the whole process, due to the lack of structure and confidence on part of the population, in relation to the selective collection program. The choice for action research as the methodology and the use of improvement and learning cycles proved to increase the project’s overall efficiency, making it possible to make decisions more quickly, generating better results, and enabling replications of good decisions and correction of errors from one cycle to the next.     

The Participation Imperative in Co-operative Inquiry: Personal Reflections of an Initiating Researcher

Participation has become an imperative in international development. In particular, participatory approaches to development research are believed to support empowerment through collective development of knowledge and action. Yet there are broad interpretations of participation, ranging from passive participation and information exchange to empowered participation through self-mobilisation. As such, development researchers may claim to use a participatory approach without yielding power and agency to citizens involved in the research. This is a reflexive article about a development researcher’s experience in applying a participatory worldview through co-operative inquiry, a research approach that attempts to engage emancipatory forms of participation. The article begins with a critical analysis of literature regarding participation, the participatory paradigm and participation in the co-operative inquiry. The author then analyses her personal experiences, challenges and strategies as initiating researcher to actualise empowering forms of participation in three co-operative inquiries. The article emphasises that researchers must “let go” of their power, control and personal agenda to support empowering participation.     

Action Research as a Bridge Between Two Worlds: Helping The NGOs and Humanitarian Agencies Adapt Technology to Their Needs

This paper reflects upon the process of introducing social media systems into supported employment programs for peoples with severe mental illness in Taipei. Such intervention represents the encounter of two different worlds that are often-unknown to each other, i.e. IT student engineers and job coaches in NGOs. With the participatory and cyclic feature of action research approach, the process can be divided into three stages: technology-led stage, mutual infusion stage, and service-led stage. The focus of this research is to illustrate how action research provides a site for the infusion of two different horizons, that is technology experts in academia and social service providers in NGOs. The existing problems in the use of IT within NGOs were studied and changes through co-inquiry and interventions were implemented towards better utilization of IT and higher levels of service qualities. The researchers utilized an action research approach to investigate the problems, plan the actions, intervene, assess the effectiveness resulting from the actions, and specify learning outcomes.     

Smart Cities and M<Superscript>3</Superscript>: Rapid Research,Meaningful Metrics and Co-Design

The research described in this paper is undertaken under the banner of the smart city, a concept that captures the way urban spaces are re-made by the incursion of new technology. Much of smart is centred on converting everyday activities into data, and using this data to generate knowledge mediated by technology. Ordinary citizens, those that may have their lives impacted by the technology, usually are not properly involved in the ‘smartification’ process. Their perceptions, concerns and expectations should inform the conception and development of smart technologies at the same extent. How to engage general public with smart cities research is the central challenge for the Making Metrics Meaningful (MMM) project. Applying a rapid participatory method, ‘Imagine’ over a five-month period (March – July) the research sought to gain insights from the general public into novel forms of information system innovation. This brief paper describes the nature of the accelerated research undertaken and explores some of the themes which emerged in the analysis. Generic themes, beyond the remit of an explicit transport focus, are developed and pointers towards further research directions are discussed. Participatory methods, including engaging with self-selected transport users actively through both picture creation and programmatically specific musical ‘signatures’ as well as group discussion, were found to be effective in eliciting users’ own concerns, needs and ideas for novel information systems.     

Learning by Action Research: A Policy for School Development

In this article I define a project organization as action research. Thus defined, the approach presents a method for experiential learning among educators who develop as learners in an educational setting. Second, the approach describes a method for focusing on research related to change in classroom didactics brought about by the mediating instruments educators discover and use so as to overcome their own learning anxiety. Third, action research and academic investigation are described as a method for integrating theory with practice. Finally, action research is outlined as an effort for social justice—the way knowledge usually develops by social construction.     

Intersections of Critical Systems Thinking and Community Based Participatory Research: A Learning Organization Example with the Autistic Community

Critical systems thinking (CST) and community based participatory research (CBPR) are distinct approaches to inquiry which share a primary commitment to holism and human emancipation, as well as common grounding in critical theory and emancipatory and pragmatic philosophy. This paper explores their intersections and complements on a historical, philosophical, and theoretical level, and then proposes a hybrid approach achieved by applying CBPR’s principles and considerations for operationalizing emancipatory practice to traditional systems thinking frameworks and practices. This hybrid approach is illustrated in practice with examples drawn from of the implementation of the learning organization model in an action research setting with the Autistic community. Our experience of being able to actively attend to, and continuously equalize, power relations within an organizational framework that otherwise has great potential for reinforcing power inequity suggests CBPR’s principles and considerations for operationalizing emancipatory practice could be useful in CST settings, and CST’s vocabulary, methods, and clarity around systems thinking concepts could be valuable to CBPR practitioners.     

网络环境下组织学习测度的实证研究

首先对组织学习测度研究进行了综述,然后确定了组织学习的测度模型-EPCETVLS模型,并应用网络环境下企业调研的实际数据对测度模型进行了确定性因子分析.最终对组织学习测度的EPCETVLS模型的深入分析,为以后的组织学习相关分析奠定了基础.     

Coproduction in Practice: Participatory Action Research to Develop a Model of Community Aged Care

Coproduction has become synonymous with innovative approaches to public service delivery in European Union countries as well as in Australia. Coproduction has the potential to bring together individuals, communities, and organisations in a process to collaboratively develop new models and services which improve public services. Yet, Australian policy makers and practitioners who would like to deploy coproduction within the context of older adult social care can only draw on a handful of papers and reports that could guide implementation. This paper fills this gap by reporting on the implementation of a multi-stakeholder coproduction approach to the development of a consumer directed care model for older people with complex health issues. The paper describes and critically highlights methodological challenges encountered during the 12 month-long participatory action research phase of a larger project involving older people with complex care needs, their carers, and government and non-government stakeholders. The paper outlines key considerations regarding (1) the involvement of older people with complex needs, (2) collaboration with industry partners, (3) engagement of government representatives, and (4) reflects on implementing participatory research projects within a context of outsourcing and interlinked supply chains. While not all challenges encountered could be resolved, the coproduction approach was successful in bringing together a wide range of stakeholders with competing agendas in an iterative process geared to resolve a plethora of concerns raised by older people, carers and services providers. This paper provides an example for others seeking to use coproduction and participatory methods to provide person-centred care services for older people.     

Learning with participatory monitoring and evaluation in Dir district,northwest frontier province,Pakistan

This paper is concerned with the development of a monitoring and evaluation system for the second phase of a United Nations Drug Control Programme (UNDCP) in Pakistan and the manner in which participatory structures are developed and built on in a complex development context. Initially the paper introduces the context. The project is concerned with rural development issues as part of a structured programme to substitute opium poppy growing with a range of agricultural and social innovations. During the first phase of the project (up to 1993) the main focus of attention for the project was measured in terms of physical progress with road building, electricity supply, irrigation works and concentration on improved agricultural practices. Phase II has set out to build on this, most specifically in terms of gaining the greater participation of local people in the project activity. The paper describes the monitoring and evaluation (M&E) structure for the project and notes that it is intended that M&E further encourage participation and beneficiary analysis (analysis of project benefits by those who are intended to derive most from the project outputs). One of the primary means set out for achieving this was by inducing the professional staff working in the project to adopt new mind sets, adjusting their focus from being technical experts supplying undoubted technical improvements to being co-operative partners, bringing ideas to the area but equally being willing to work through these ideas and their implications and learn from the communities. The process for the development of the monitoring system is described, and the methodology ‘TeamUp’ discussed. Links are made to related areas of study—most specifically the literature of Rapid and Participatory Rural Development (RRA and PRA). Finally, the paper discusses the results of the research to date and the likely extension of the ideas to other projects.     

The Role of Formalisation, Participation and Context in the Success of Public Involvement Mechanisms in Resource Management

In the face of complex and uncertain issues, one important goal of public participation in resource management and research is to foster communication and the inclusion of non-expert knowledge—thus the effective flow of information between project organisers and stakeholders. We compare different methods (instruments, tools) that were employed in the German–Austrian ‘PartizipA’ project to structure information flows in participatory processes. Depending on their goals and context, more or less ‘formalised’ and ‘participatory’ methods were applied, the most important being guided interviews, focus groups, agent-based modelling, nutrient modelling, cognitive mapping and group model building as well as the development of a common document. Two regional case studies, both concerned with European-induced institutional change, are portrayed in which the specific participatory methods were embedded. The Austrian case study involved the analysis and modelling of agricultural land use in the region of St. Pölten against the background of the reform of the Common Agricultural Policy, while the implementation of recent European water policy was the issue in the German agricultural region north of Osnabrück. Presenting both cases in their regional context, the applied methods are first described according to the logic of the entire respective process. Subsequently, the specific methods are systematically analysed and compared according to their objective, context and degrees of participation and formalisation. Finally, we evaluate all methods regarding their effectiveness in terms of goal attainment and their potential generalisation, seeking to respond to the question of when a particular method might best be used.     

基于元网络的项目组织学习能力对任务完成影响评价仿真方法

组织能力是影响项目任务完成的决定性因素,而学习能力是组织能力形成的关键基础.基于元网络理论和动态仿真方法,将组织(人)、知识和任务视为多智能体复杂交互系统,提出知识扩散度和任务完成水平两项关键指标,研究项目组织学习能力对任务完成水平影响的内在机理.以通用品牌别克汽车4S店建设项目群管理为例,验证评价指标的合理性,并进一步通过仿真实验,识别项目组织的关键知识、关键组织以及关键学习阶段,进行组织学习能力提升和任务分配的优化.为项目组织设计、任务分配及组织学习的研究探索了一种有效的方法.     

Formalised and Non-Formalised Methods in Resource Management—Knowledge and Social Learning in Participatory Processes: An Introduction

The participation of non-state actors in public decision-making and transdisciplinary research is increasingly regarded as an effective means to cope with growing uncertainties and complexities in human–nature interactions. The management of natural resources is expected to profit from a broader knowledge base and processes of social learning, thus allowing for potentially more informed and creative decision-making. Communication is a key element of transmitting knowledge and fostering social learning. This article introduces the special issue, which assembles contributions that discuss different methods, instruments, tools, and models that have been developed in order to facilitate the transmission of information as well its selection and aggregation. Each of the contributions is briefly reviewed. The approaches discussed here and in the individual papers aim to foster learning in participatory processes. We argue that a key aspect is the degree to which methods are formalised. Formalisation refers to the extent to which information is channelled in a certain way, leaving more or less scope for open communication. Depending on the goals and context, more or less formalised methods can be employed. We conclude by highlighting the context-dependency of participatory processes in natural resource management and indicate some directions for future research.

Systems Thinking and Systems Practice: Working in the Fifth Dimension

This paper describes how a messy organizational problem was tackled using a systems approach which led to a resolution of the problem and to an improvement in the working relationships of the people involved. The example also shows how people who are not familiar with systems thinking, as such, can quickly be helped to develop a shared systemic view of a complex situation using activity modeling. At a deeper level, the paper also shows how the underlying dynamics of the problem situation was altered as a result of using a cooperative, dialogical working method, leading to an improvement in the quality of service provided by that part of the organization. The paper uses Peter Senge's five disciplines of a learning organization, together with various systems concepts, as a framework for describing how the process of change was brought about through the simultaneous management of task and social processes, leading to both individual and organizational learning.     

When Social Workers Meet Special Education Teachers: Action Research to Implement Curricular Changes in Taiwanese Special Education Systems

Non-governmental organizations (NGOs) offering supported employment services in Taiwan recently have identified a missing link between their programs and the vocational programs students with disabilities received in secondary special education system. Sponsored by a national initiative aimed to enhance public education through crossover trials and social inclusion under the umbrella of ??HighScope Taiwan??, academics and mainly senior social workers devoted to supported employment for persons with intellectual disabilities started a two-way outreach program through action engagement in 2007. Action research became the common focus of the joint task forces across social work, public education systems, and academia. The article provides field-based reflections of the action research project and suggests ways in which practice of action research may increase social impact in the professional development of individuals.     

The Role of Action Research (AR) in Environmental Research: Learning from a Local Organic Food and Farming Research Project

We expand on the FMA model of inquiry elucidated by Checkland and Holwell (Syst Pract Act Res 11:9, 1998) and identify three kinds of research: interactive Action Research (AR), extractive participatory research, and traditional science, that were integrated into an interdisciplinary agro-environmental research project involving researchers from eight countries around the Baltic Sea. The research project held the goal of creating scientific understanding of the ecological, economic and social effects of the sustainable development of local food systems, while also having the explicit goal of implementing change in the studied systems. Based on our experience working as action researchers in the project, we highlight the role that normative goals and facilitation process competence play in complex interdisciplinary research projects. Our reflections focus on project design, including the relation between AR and other kinds of research when employed in a single project, and includes reflections on preconditions for stakeholder participation in AR projects.     

Student, Curricular and Public Agency Needs: A Focus on Competency Achievement

This paper presents a framework that addresses a public child welfare agency??s need for a highly trained child welfare workforce with specialized competencies, situating these competencies within the human services educational profession??s accreditation standards, while simultaneously meeting the learning needs of professionals returning to school for additional graduate training. The framework integrates three disparate literatures. The first that frames employees returning to a university setting for additional education and training as a human resources benefit. The second body of literature that addresses the annual review of employee performance via employment assessment systems. And the final body that examines parallel organizational processes related to the development of a product, in our case, a professional with a graduate degree. The framework, developed using a soft system methodology approach, and evaluated by means of an electronic portfolio, integrates these three bodies of literature and allows for the real time curricular and agency assessment of competency achievement. While using a professional graduate training program as an exemplar, implications for broader systemic practice are discussed.