Monitoring and evaluation plan for kalamazoo Airzoo STEM Program Pre-K

Identify how you will measure your program's performance and track data to stay informed about how well your program is doing.

What data will you collect, and how will you analyze the data?

*Guide monitoring in practice.

The monitoring process will involve continuous assessment of the program's performance against its objectives. We will measure the program's performance using both qualitative and quantitative data.

Data Collection: We will collect data on the number of participants, their engagement levels, their feedback, and their learning outcomes. This will be done through surveys, observation, and tests.

Data Analysis: The collected data will be analyzed using statistical methods to identify trends, patterns, and areas of improvement. We will use software tools for data analysis to ensure accuracy and efficiency.

Guidance for Monitoring in Practice:

Regularly collect data: This ensures that we have up-to-date information about the program's performance.

Use reliable data collection methods: This ensures that the data collected is accurate and reliable.

Analyze data promptly: This allows us to quickly identify and address any issues.

Evaluation

Develop a brief evaluation plan using your approach based on TC and the evaluation approach assignment. At a minimum, identify

The purpose of the evaluation

Evaluation question

Evaluation methods

BONUS: Draft instruments

The evaluation plan will involve a comprehensive assessment of the program's effectiveness and impact.

Purpose of the Evaluation: The evaluation's purpose is to assess the program's effectiveness in achieving its objectives and to identify areas for improvement.

Evaluation Question: The main evaluation question will be "How effective is the STEMS Kalamazoo Michigan Air Zoo program in achieving its objectives?"

Evaluation Methods: We will use a mixed-methods approach for the evaluation, combining quantitative data from the monitoring system with qualitative data from interviews and focus groups.

M&E Management

This should include guidance on the following:

Reporting and dissemination plan

Responsibilities

A timeline for M&E implementation

Reporting and Dissemination Plan: We will prepare regular reports on the program's performance and the results of the evaluation. These reports will be shared with all stakeholders, including program staff, participants, and funders.

Responsibilities: The program manager will be responsible for overseeing the M&E activities, with support from the program staff.

Timeline for M&E Implementation: The monitoring activities will be ongoing throughout the program, while the evaluation will be conducted at the end of the program.

NOTES from class!

What type of info does the...

Funding

Management

Public

*Coverage data of which schools attend vs who does not = target population of who is and who is not being reached.

*Equity data looks at subgroups of the population being served on disability, race, income, age, etc. = during the implementation or end of a program.

*Process data to make sure the program is implemented as designed = feel welcomed and learn something from it (survey?)

*Effort (output) data = provides feedback on the number of products and services, quality, and number of service completions. (Track annually for funding? Is it cost-efficient?)

*Cost-Efficiency Data provides feedback on the costs of providing these programs (The difference between planned costs with actual costs) and can be used to assess a program's productivity at the end of a program year required for performance measurement reporting.

- Opportunity costs are the next best thing.

*Results (Outcome) Data provides feedback on the extent to which a program achieves its intended results

- can be monitored during implementation to compare actual results

- can be used at the end of a program year to document results achieved by a program and used for policy and planning purposes.

*Cost-effectiveness data provides feedback on the costs of achieving program results

- for every dollar spent, what is the actual outcome/change

*Impact data provides feedback on what happened to clients because of program participation that would not have happened in the program's absence

Evaluation

Develop a brief evaluation plan using your approach based on TC and the evaluation approach assignment. At a minimum, identify

The purpose of the evaluation

Evaluation question

Evaluation methods

BONUS: Draft instruments

LOGIC OF EVALUATION

Criteria- indicators--> who has access vs who goes (barriers being lack of funding or time?)

Standards

Collect data

Synthesize

MONITORING

Performance Indicator

Why

How

When

Responsible Party

Goal

Participant satisfaction

Reach the targeted audience.

10 Questions regarding evaluation for the Air Zoo

What did the Air Zoo find as areas of concern?

What was your primary focus within elementary groups?

Do the goals and objectives align with the Air Zoo funders and board of directors' goals and objectives?

What metrics or benchmarks do you use to determine scholarship awards?

Because Kalamazoo is a very diverse area, how well do you advocate for different ethnic/racial/language groups?

How do elementary students respond to programs? How do we receive that feedback?

How are we allocating funds and determining expenses for elementary age group versus the other grade levels?

How is the Air Zoo staff trained on...

DEI initiatives

Hands on learning

Educational standards

9. How frequently is the educational content for the pre-k through 5 updated?

STAKEHOLDER LEVEL QUESTIONS....

Upstream Stakeholder	Mid-Level	Down Steam stakeholders
How is the Air Zoo staff trained on... DEIA initiatives Hands on learning Educational standards	How well do elementary students respond to the context of educational programs? How do we receive that feedback?	How are we allocating funds and determining expenses for elementary age group versus the other grade levels?
Do the goals and objectives within the Air Zoo education programs align with funders and board of directors' goals and objectives?
How frequently is the educational content for the pre-k through 5 updated?

Re-examine the five questions and refine them to be more evaluative

What are the main criteria? (satisfaction/implementation/ DEI concerns)

Participant satisfaction

Indicators= recurring visits/ positive feedback/ self-report

Methods-->

monitoring data (organizational or archival data) real time

Survey (immediately after)

Review (annually or quarterly)

Timing: Possible timing of data collection (ex. post program)

Criterion of each question	Indicators	Methods	Timing	Data Collection
Satisfaction
Implementation Fidelity
DEIA
Learning Quality

Input	Criterion	Indicators	Standards	Benchmarks	Method	Timing
Activities
Outputs	Diversity
Immediate Change
Intermediate Change
Impact
Cross Cutting-Questions

How is the Air Zoo staff trained on...

DEIA initiatives

Hands on learning

Educational standards

Do the goals and objectives within the Air Zoo education programs align with funders and board of directors' goals and objectives?

How frequently is the educational content for the pre-k through 5 updated?

How well do elementary students respond to the context of educational programs? How do we receive that feedback?

How are we allocating funds and determining expenses for elementary age group versus the other grade levels?