Methods for Evaluating Existing Solutions to Social Problems

What is Evaluation?

Let's start with the basics. Imagine you're trying to figure out how well a solution (like a program, policy, or idea) works in addressing a social issue. That process of checking it out, understanding its strengths and weaknesses, and deciding if it's actually making a positive difference – that's what we call evaluation. Think of it like giving something a thorough check-up to see if it's doing what it's supposed to.

📺 Watch Michelle Molina, a professional evaluator, explain what evaluation is and discuss a five-step guide for evaluation.

From the video, we learn that evaluation is a systematic process designed to determine the value and worth of activities, programs, and initiatives. It's a versatile tool that can be applied to a wide range of contexts, offering insights that pave the way for improvements, better decision-making, and the sharing of successes. Michelle elucidates that evaluation serves as a conduit for organizations and groups to decipher success, collect pertinent data, and subsequently refine their strategies.

Like every other process, there are steps necessary to conduct a proper evaluation. In the video, Michelle outlined five of them.

❓ Pop Quiz: Can you mention the 5 steps involved in evaluating a program or initiative?

- Step 1: Consider the context*
- Step 2: Develop an evaluation plan*
- Step 3: Data collection analysis*
- Step 4: Sensemaking
- Step 5: Take action*

Why is Evaluation Important?

Good question! Imagine you're driving to a place you’ve never been to, and all you have is Google Maps. You want to keep checking the Map to see if you are making progress towards your target location. Similarly, with social problems, if we don't evaluate the solutions in place, we won't know if they're really helping people or if there's room for improvement.

Evaluation helps us make informed decisions about what's working and what needs to be improved. It helps us ask important questions like Is our solution really helping people? Are we using our resources wisely? Could we do things better? It's like checking the engine of a car to make sure it's running smoothly. By checking and measuring, we can adjust our plans and make sure we're truly making a positive impact.

📺 Learn more about the importance of evaluation by watching this United Nations Evaluation Group video montage

From the video, we learn that evaluation within the United Nations is not merely an administrative task. It's a transformative process that underpins democracy, accountability, and sustainable development. It is important to evaluate existing solutions to social problems in order to:

• Determine whether the solution is actually working
• Identify any gaps or areas for improvement
• Make informed decisions about how to allocate resources
• Share lessons learned with others

🪞 Reflect: Think about a school project you've worked on.

• How did checking your progress along the way help you make it better?
• How might evaluating our efforts in the community have a similar effect?

Types of Evaluation

Now let's take a closer look at the various ways we can evaluate solutions for social problems. Think of these types of evaluation as different lenses through which we examine things. Each type gives us a unique perspective and helps us gather a well-rounded understanding.

📺 Watch this explainer video on types of evaluation for an introductory overview of evaluation methods.

From the video, we learn that there are two main types of evaluations: Formative and Summative evaluations.

Formative evaluation takes place before or during the project's implementation to facilitate improvements in its execution. This type encompasses several sub-evaluations, including:

• Need Assessment: A process evaluation conducted prior to project initiation to identify requirements and gaps.
• Process Evaluation: Ongoing assessment during project implementation, aiming to enhance the execution process.
• Pre-Implementation Evaluation: Conducted before project commencement, addressing implementation strategies and design.

Formative evaluation essentially focuses on refining the implementation process, design, and strategy to ensure that the project or program operates more effectively.

Summative evaluation, on the other hand, occurs at the project's conclusion, aiming to evaluate the overall success of the project, program, or intervention.

❓ Pop Quiz: Can you recall the types of summative evaluation mentioned in the explainer video?

Methods of Evaluation

Evaluation methodologies refer to the specific techniques or tools you use to gather information, data, and insights during the evaluation process. These methods are the practical steps you take to assess and understand the effectiveness and impact of the solutions.

📖 Read the Eval Community article on evaluation methods for more insight.

In the context of evaluating solutions for social problems, the methods can be categorized into two main approaches: quantitative, qualitative, or a mix of both. Some frequently used methods include:

• Surveys and Interviews: Engaging with individuals through questionnaires, interviews, or discussions to gather their opinions and experiences.
• Data Analysis: Examining quantitative data, such as statistics and numbers, to identify trends, patterns, and correlations.
• Observations: Directly observing the solution in action to see how it's being implemented and whether it aligns with the intended goals.
• Comparative Analysis: Comparing different solutions or groups to determine which one is more effective in addressing the social problem.
• Case Studies: In-depth analysis of specific instances to gain insights into the real-world impact of the solution.
• Document Analysis: Reviewing written materials, reports, and documents related to the solution to understand its progress and outcomes.

❓ Discuss: Imagine you're evaluating a social entrepreneurship venture that provides job training and employment opportunities for individuals who have faced homelessness. Which of the evaluation methods will you use and why? Share your answers in the padlet below.