𖧹/△ Fulcra

Unknown

user: Surprise me with some novel ideas about how to evaluate progress in systemic strategies.

user: Surprise me with some novel ideas about how to evaluate progress in systemic strategies.

ai: Evaluating progress in systemic strategies requires innovative approaches that can navigate the complexity and dynamism of systems. Here are some novel ideas for evaluating progress in systemic strategies:

1. Dynamic Systems Modeling and Simulation: Utilize dynamic systems modeling to simulate interventions and predict their impacts over time. This approach allows for the testing of different strategies in a virtual environment, providing insights into potential outcomes and unintended consequences. Tools like System Dynamics models can help in understanding how changes in one part of the system might ripple through others.

2. Real-time Data Analytics and Visualization: Leverage real-time data analytics to monitor system behavior and the impact of interventions continuously. Advanced data visualization techniques can help stakeholders understand complex system dynamics and track progress against goals. This approach enables rapid adjustments to strategies based on current system performance.

3. Participatory Evaluation Methods: Engage system stakeholders in the evaluation process through participatory methods. Techniques such as Most Significant Change (MSC) and Outcome Harvesting can capture the perspectives of those affected by systemic strategies, providing qualitative insights into changes and their significance. This approach ensures that evaluations are grounded in the lived experiences of stakeholders and can uncover unexpected outcomes.

4. Adaptive Management and Learning Loops: Implement adaptive management practices that incorporate continuous learning and adjustment of strategies based on evaluation findings. Establish feedback loops that allow for the iterative testing of hypotheses about system behavior and the refinement of interventions. This approach acknowledges the uncertainty inherent in systemic change and fosters a culture of experimentation and learning.

5. Network Analysis and Social Network Mapping: Use network analysis to evaluate changes in relationships and information flows within the system. Social network mapping can reveal shifts in collaboration patterns, power dynamics, and the diffusion of innovations. This method provides insights into the social infrastructure of the system and how it evolves in response to interventions.

6. Counterfactual Reasoning and Scenario Planning: Apply counterfactual reasoning to consider what would have happened in the absence of the systemic strategy. Combined with scenario planning, this approach can help evaluators understand the unique contributions of their interventions amidst other influencing factors. It also aids in anticipating future challenges and opportunities, allowing for proactive strategy adjustments.

7. Leverage Point Analysis: Focus evaluations on identifying and assessing interventions at leverage points—places within a system where a small shift can lead to significant changes. This approach prioritizes understanding the effectiveness of strategic actions in altering system behavior in desired ways and can guide the allocation of resources to the most impactful interventions.

8. Resilience and Sustainability Indicators: Develop and monitor indicators of system resilience and sustainability. These indicators can help assess whether systemic strategies are enhancing the system’s capacity to withstand shocks, adapt to change, and maintain functionality over time. Evaluating progress through the lens of resilience and sustainability ensures that systemic strategies contribute to long-term system health and well-being.

By integrating these novel ideas into the evaluation of systemic strategies, changemakers can gain deeper insights into the complexities of systems change, make informed decisions, and adaptively manage their interventions for greater impact.

Sources: ⟁ Evaluation of systemic change Systemic strategy–Systemic design methods for complex systems change Systemic strategy Systemic Evaluation Systemic strategy hypotheses testing Testbeds for a systemic strategy lab

user: Say more about number 3?

user: Say more about number 3?

ai: The participatory evaluation methods mentioned, specifically Most Significant Change (MSC) and Outcome Harvesting, are innovative approaches that actively involve system stakeholders in the evaluation process of systemic strategies. These methods are particularly valuable for capturing qualitative insights into the changes occurring within complex systems and understanding their significance from the stakeholders’ perspectives. Here’s a detailed look at each method:

Most Significant Change is a form of participatory monitoring and evaluation technique that involves the collection and systematic selection of significant change stories emanating from the field level. The process is participatory as it engages a wide range of stakeholders in deciding which of these stories represents the most significant change among them. Here’s how it works in the context of evaluating systemic strategies:

1. Collection of Change Stories: Stakeholders at various levels are asked to report stories of what they consider the most significant change that has occurred as a result of the systemic strategy.
2. Selection Process: These stories are then shared among stakeholders, who are involved in a systematic process of selecting the stories that they feel are most significant. This often involves discussions and deliberations, which in itself is a reflective and evaluative process.
3. Analysis and Feedback: The selected stories are analyzed for insights into how and why changes are happening, and the findings are fed back into the decision-making process for the systemic strategy. This helps in adjusting strategies and actions in real-time based on stakeholders’ perceptions of change.

Outcome Harvesting is another participatory method that collects evidence of what has changed and, then, working backward, determines whether and how an intervention has contributed to these changes. It is particularly useful in complex settings where changes are non-linear and not directly attributable. Here’s how Outcome Harvesting can be applied:

1. Identification of Changes: Engage with stakeholders to identify specific changes that have occurred within the system. These changes can be intended or unintended, positive or negative.
2. Harvest Outcomes: Collect detailed information about these changes, including who was involved, what exactly changed, and how the change came about.
3. Validate and Engage: Share the collected outcomes with stakeholders for validation and further engagement. This step ensures the accuracy of the information and deepens stakeholders’ understanding of the changes.
4. Analyze and Support Strategy: Analyze the outcomes to understand their implications for the systemic strategy. This analysis can help identify leverage points, inform strategic adjustments, and provide evidence of impact.

Both MSC and Outcome Harvesting are powerful because they focus on changes that are valued by stakeholders, thus ensuring that the evaluation is grounded in the realities and priorities of those most affected by the systemic strategy. These methods also support adaptive management by providing timely, qualitative insights that can guide strategic adjustments. Engaging stakeholders in this way not only enriches the evaluation process but also builds ownership and accountability for the outcomes of systemic strategies.

Sources: Systemic Evaluation Systemic strategy–Systemic design methods for complex systems change ⟁ Evaluation of systemic change A potential article from the PANA work Who is the system- Open Social Mapping Systemic Design for Systems Change - RSD9 abstract

user: Describe a step-by-step procedure that links the development and implementation of systemic strategy with these techniques for monitoring and evaluation.

user: Describe a step-by-step procedure that links the development and implementation of systemic strategy with these techniques for monitoring and evaluation.

ai: Developing and implementing systemic strategies that are both effective and adaptable requires a comprehensive approach that integrates innovative monitoring and evaluation techniques. Here is a step-by-step procedure that incorporates such techniques, including Dynamic Systems Modeling, Real-time Data Analytics, Participatory Evaluation Methods (including Most Significant Change and Outcome Harvesting), Adaptive Management, Network Analysis, Counterfactual Reasoning, Leverage Point Analysis, and Resilience and Sustainability Indicators:

1. Identify Key Stakeholders: Engage with stakeholders who are part of or affected by the system. Their insights will provide a diverse perspective on the system’s components and dynamics.
2. Develop System Maps: Use participatory methods to create causal loop diagrams or influence maps. This collaborative mapping helps in understanding the system’s structure and identifying leverage points.

1. Dynamic Systems Modeling: Use the system maps to develop dynamic models that simulate the system’s behavior over time. This helps in understanding complex interactions and predicting the impact of potential interventions.
2. Leverage Point Analysis: Identify points in the system where small changes could lead to significant impacts. This analysis guides the focus of interventions towards areas with the highest potential for systemic change.

1. Develop Strategy Trees: Based on the leverage points and dynamic models, outline various pathways (strategy trees) that lead from interventions to desired goals.
2. Combine into Strategy Forests: Assess and combine these strategy trees into a comprehensive strategy forest, evaluating each for feasibility, impact, and sustainability.

1. Adaptive Management: Implement the strategies with an adaptive management approach, allowing for flexibility and adjustments based on real-time feedback and changing conditions.
2. Real-time Data Analytics: Utilize real-time data analytics to monitor the implementation process and outcomes, providing timely insights for adaptive management.

1. Participatory Evaluation Methods: Incorporate methods like Most Significant Change and Outcome Harvesting to capture qualitative outcomes and impacts from the perspective of different stakeholders.
2. Network Analysis: Use network analysis to evaluate the changes in relationships and interactions within the system, assessing the strength and reach of the implemented strategies.
3. Counterfactual Reasoning: Apply counterfactual reasoning to assess what would have happened in the absence of the intervention, helping to attribute changes directly to the strategies implemented.

1. Dynamic Systems Modeling (Revisited): Revisit the dynamic models to incorporate data collected during the implementation phase, refining predictions and understanding of the system’s behavior.
2. Resilience and Sustainability Indicators: Develop and monitor indicators that measure the resilience and sustainability of the changes, ensuring that the strategies contribute to long-term systemic change.

1. Feedback Loops: Establish feedback loops that continuously feed learning back into the system mapping, analysis, and strategy design phases.
2. Scale Strategies: Identify successful strategies for scaling up or replicating in other contexts, using the insights gained from monitoring and evaluation to guide the scaling process.

This procedure integrates systemic strategy development with innovative monitoring and evaluation techniques to create adaptable, impactful, and sustainable systems change initiatives. By continuously learning from and adapting to the system’s dynamics, changemakers can more effectively navigate complexity and drive meaningful change.

Sources: Systemic strategy–Systemic design methods for complex systems change A potential article from the PANA work ⟁ Evaluation of systemic change ADMC Paper - Systemic theories of change- using leverage analysis to design and manage complex change Crafting systemic strategies - Identifying actions that create systems change Systemic strategy

user: Not bad, but too unfocused. Try again and only use your ideas from participatory evaluation methods.

user: Not bad, but too unfocused. Try again and only use your ideas from participatory evaluation methods.

ai: Integrating participatory evaluation methods such as Most Significant Change (MSC) and Outcome Harvesting into the development and implementation of systemic strategies offers a nuanced approach to monitoring and evaluating progress effectively. This integration can enhance the systemic evaluation process by providing deeper insights into the impacts of systemic strategies and ensuring that these strategies are responsive to the complexities of the systems they aim to change. Here’s a detailed exploration of how these methods can be integrated effectively:

Firstly, it’s essential to align participatory methods with the Systemic Theories of Change (SToCs). SToCs, as discussed, bridge conventional Theories of Change (ToCs) and systemic design, embracing complex systems dynamics. MSC and Outcome Harvesting can be integrated into SToCs by identifying and documenting significant changes and outcomes that emerge from systemic interventions. This alignment ensures that the evaluation methods are not only tracking predefined indicators but are also open to capturing unexpected changes and outcomes, which are often prevalent in complex systems.

The strategic planning process, particularly when using the “seeds,” “trees,” and “forest” metaphor for systemic strategies, can benefit from the inclusion of MSC and Outcome Harvesting. These participatory methods can be used to gather data and insights at different stages of systemic interventions (from seeds to forest), providing real-time feedback that can inform strategic adjustments. This approach allows systemic designers and strategists to remain adaptive and responsive to the system’s dynamics, enhancing the effectiveness of systemic strategies.

Participatory evaluation methods are inherently inclusive, making them well-suited for engaging a wide range of stakeholders in the evaluation process. By involving stakeholders in identifying, documenting, and analyzing significant changes and outcomes, these methods can help build a shared understanding of the system’s complexities and the impact of systemic strategies. This collaborative approach to evaluation not only enriches the data collected but also fosters a sense of ownership and accountability among stakeholders, which is crucial for sustaining systemic change.

The development of a “Systemic Impact Framework for Transformation” (SIFT), as outlined, emphasizes the importance of monitoring signals of change in large-scale, long-term systems change work. MSC and Outcome Harvesting can be instrumental in this process by providing mechanisms for capturing and analyzing qualitative data on significant changes and outcomes. These methods can help identify early indicators of systemic impact, allowing evaluators and strategists to track progress and make informed decisions about the direction of systemic interventions.

Finally, integrating MSC and Outcome Harvesting into systemic strategies acknowledges and embraces the complexity and uncertainty inherent in systems change work. These participatory methods are flexible and adaptive, enabling evaluators to navigate the non-linear and emergent nature of systemic change. By focusing on significant changes and outcomes, rather than rigid indicators, these methods can accommodate the unpredictability of complex systems, providing valuable insights that can guide the continuous refinement and adaptation of systemic strategies.

In conclusion, integrating participatory evaluation methods like Most Significant Change (MSC) and Outcome Harvesting into the development and implementation of systemic strategies offers a dynamic and inclusive approach to monitoring and evaluating progress. This integration enhances the ability of systemic strategies to be adaptive, responsive, and effective in fostering positive social gains at scale within complex systems.

Sources: Systemic Evaluation Systemic strategy–Systemic design methods for complex systems change ⟁ Evaluation of systemic change A potential article from the PANA work Systemic Design for Systems Change - RSD9 abstract ADMC Abstract Submission - Systemic theories of change