Request For Aga Khan Foundation
Request For Endline Evaluation of EC Climate Change Project
1. STUDY RATIONALE
AKF Afghanistan’s (AKF (Afg)) Monitoring, Evaluation, Research and Learning (MERL) Unit is looking for an experienced and qualified Individual/firm (hereafter referred to as “consultancy”) to conduct the EC climate change end-line evaluation study which will contribute to the project’s completion report. To understand the project’s contribution and achievements against the proposed desired situations and overall objective; the consultant is required to use the available and collected data in addition to primary data collection to further understand the project effectiveness, efficiency, relevancy, impact, and sustainability in targeted areas. The consultant is also required to map the areas of success, for instance, greater lessons learned, case studies and apply outcome harvesting evaluation approaches. Moreover, the firm is expected to map the results of outputs immediate effects, intermediate outcomes, related indicators and changes on the communities’ life as a result of project interventions. The end-line evaluation’s report will be shared with the donors and other relevant stakeholders in the form of presentation and narrative report as assigned.
2. BACKGROUND
Active in Afghanistan since 2002, AKF (Afg) delivers integrated multi-input development programs in eight provinces of the country's 34 provinces. With over 2,000 staff members engaged in 53 districts, AKF (Afg) focuses on community development and governance, civil society, market development, natural resource management, infrastructure, education, and health. AKF’s vision of Afghanistan is a stable, peaceful and more prosperous country where resilient communities have access to quality basic services and the knowledge and tools to uphold inclusive and effective institutions to support a democratic and pluralistic society.
The AKF (Afg) is part of the Aga Khan Development Network (AKDN), a group of private, international, non-denominational agencies founded by His Highness the Aga Khan. As the overall goal of the AKDN is “To improve quality of life) in the areas where its member institutions operate.
2.1. About EC Climate Change:
Aga Khan Foundation (AKF) in Afghanistan in close coordination with Wildlife Conservation Society (WCS) and Aga Khan University (AKU) implemented a 46-month climate change project since the beginning of 2018. The project aims to improve resilience to climate change of target communities and the eco-systems in the Punj-Amu River Basin and the sustainability of their use for the benefit of rural communities. This goal is followed by two key objectives as following:
- Conservation of biodiversity and increased eco-systems and community resilience through improved natural resources management and climate change adaptation measures in upper watersheds of the Punj-Amu River Basin.
- Sustainable rural development and protection of ecosystems and biodiversity through increased renewable energy generation and distribution.
Since the project had a complex model in design and has been implemented in a complex context where the data play a key role to identify the issues which were making the communities vulnerable, and inform the project management and planning, a comprehensive Monitoring and Evaluation plan was embedded in the project design. The Monitoring and Evaluation activities suggested are aligned with the project results’ indicators and for informing prioritization, planning and implementation of the project. The key MERL activities included are as following:
- Vulnerability Assessment: a vulnerability assessment was conducted at the beginning of project to assess all aspects of climate, natural hazards, ecosystem, hydrology, wildlife species, and local communities with the aim to identify the entry points for program and issues to be addressed by the project implementation. It also provides information to track the indicators in the project logframe.
- Develop and implement long-term monitoring program for climate change and integrate the monitoring system with the national environmental database.
- Routine monitoring for tracking project’s immediate results and progress against related outcomes and outputs indicators.
This assignment should focus on the impact level indicators and assess the immediate and intermediate changes as result of the project interventions.
OVERALL OBJECTIVE:
As part of ECCC M&E activities, AKF (Afg) team has already collected the output level data which is covering almost all indicators, except the one which measures the socio-economic impact of the project. Therefore, the end-line evaluation will investigate the available data, do analyse, and compare it with the available baseline relevant findings from the vulnerability assessment and collect the significant change stories to be reflected and synthesised in the end-line evaluation and completion report.
Therefore, the overall objective of this assignment is to use the available and collected data as well as to design the tools for the needed information (primary level data) and write the end-line evaluation report. This will contribute to the completion report of the project. More specifically the ECCC end-line evaluation will measure the following goal and objectives:
Goal:
- To conduct an end-line evaluation and measure the project’s indicators outlined in the project logframe and capture and document the learning, success and best practices.
Specific Objectives:
- Measure the project’s indicators outlined in the logframe and map the changes.
- In addition to the indicators in the logframe, the end-line evaluation should address the following evaluation questions:
- Effectiveness:
o To what extent has the project achieved its intended results (considering the project’s indicators in the logframe), so far focusing on achievements and whether the objectives have been met or not as part of effectiveness evaluation criteria.
o What are the learning from the projects outcomes, approaches and implementation processes at national and regional levels and how this can be improved in future programming?
- Efficiency:
o To what extent the project has efficiently been implemented, considering the cost, timeline, and resources?
o What are the factors which have affected project implementation including (internal factors such as technical, managerial, organizational, institutional, and external factors such as socio-economic and context)?
o Was the project Monitoring and Evaluation System/plan and reporting adequate/efficient to provide sufficient and reliable data, assessments, and analyses to report, learn, improve and adapt throughout the project?
- Relevancy:
o To what degree were the program activities in each target province and district relevant to the need of project beneficiaries, national and global policy and priorities?
o Does the project/ or any its individual interventions has the potential to be upscaled and/or continued to be piloted in other geographical locations based on real need?
o Have the project activities been implemented as planned?
- Sustainability
o To what extent the project’s interventions are sustainable beyond the project’s timeline?
o Have there been any localized mechanisms established to guarantee the project’s sustainability?
- Impact:
o What is the impact of project on the life of beneficiaries, capacity of institutions (particularly businesses) and policies at national and regional levels?
o Has the project contributed/attributed to the desired situations that was proposed in project’s results framework?
- Adaptation/ changes:
o What were the areas of adaptation or changes in the approaches and strategies of the project especially in relation to COVID-19 restrictions including change of regime?
o Why such adaptations have been considered?
3. METHODOLOGY:
Considering the overall objective of project, a baseline study was planned at the earlier stages of project to indicate the existing situations especially around climate change adaptation strategies, plans and approaches including the natural resource management strategies and practices of target communities. The baseline data was not fully collected at the proper time to provide initial data against all indicators. Therefore, the MERL team used meta-analysis by looking to the available and relevant data and for each indicator including the output level data the participatory rural appraisal (PRA) was conducted to establish some level of benchmarks which enabled comparison. This approach helped to understand the key constraints and issues and accordingly define the targets for the project.
Therefore, for the end-line evaluation an outcome harvesting approach to map the progress by tracking the outputs and immediate results according to beneficiaries’ perspective should be applied. In addition, the consultancy will need to review and collect the Stories of Significant Changes to reflect on the changes and the project impact on beneficiaries’ life. Analysis of information gathered through both routine monitoring and other existing available data/information should be used to inform the changes and map the outputs and outcomes.
The end-line evaluation report should reflect the findings of meta-analysis, outcome harvesting and most significant change stories (MSCs). It should focus on the indicators and discuss the challenges and limitations that the project faced during implementation.
4. MAIN DUTIES AND RESPONSIBILITIES
The successful candidate should work together with both the AKF (Afg)’s MERL and Agriculture & Food Security (AFS) teams in accordance with the specific duties assigned below for end-line evaluation:
- Review the available data and provide a content for the end-line evaluation report.
- Develop the tools/discussion guidelines where needed for further data collection based on project’s indicators.
- Conduct data verification through triangulation of various reports including, vulnerability assessment, routine data captured and documented in the MIS and progress reports.
- Facilitate the FGDs and KIIs with the contextual support of AKF (Afg)’s program and M&E team teams to understand the efficiency, relevancy, and sustainability of the project.
- Collect and analyse the quantitative information to inform and measure the relevant indicators.
- Prepare and present the first draft of end-line evaluation report for review by AKF (Afg) to check the factual basis of the review, and to discuss the draft findings, conclusions, and recommendations.
- Incorporate comments and submit final report on agreed date.
5. QUALIFICATIONS REQUIRED
The consultancy should have the following qualifications:
- Master or higher-level degree in climate related Sciences
o Very good technical understanding of climate change theories including the examples of some practical and applicable models related to previous experience in the same field,
o Experience of evaluating climate change adaptation projects.
- Extensive M&E, data management, MIS and research experience,
- Experience in preparing and writing professional and utilizable reports and other utilization focussed outputs.
- Experience working in the regions in which this study will take place, including knowledge of the local language(s) and context.
- An ability to work with tight timelines and deadlines.
- Education and/or research experience in development challenges; and
- Ability and willingness to work in remote, hard-to-reach parts of Afghanistan.
- Ability to prepare analyses, presentations, and reports using professional/academic English language write up skills.
6. WORK LOCATION(S) AND REPORTING LINE
The consultancy will need to be presented in Kabul for an initial series of orientations and kick-off meetings, but if necessary, the consultancy will need to travel to concerned regions to enhance understanding. The consultancy will mainly be reporting to the National Manager, M&E – MERL, while the National Lead AFS and National Lead MERL will also be part of reporting sources who are based in the National Programme Office in Kabul.
7. DELIVERABLES
The following deliverables are expected to be provided to AKF (Afg) as per this contract:
- A comprehensive work plan to be followed by the consultancy;
- End-line evaluation report contents as part of inception.
- Comprehensive study methodology and data collection tools
- Report from review of available data and analysis plan
- Progress monthly report the supervisors/MERL team
- Draft of end-line evaluation report
- Final report
- PowerPoint presentations on end-line evaluation report.
- Final progress report.
8. CONSULTANCY TIMELINE
This assignment needs to be started by mid-June and completed by mid-August 2023.
9. ADDITIONAL SIGNIFICANT INFORMATION
The consultancy is bound to all ethical norms with the consideration of confidentiality for all information provided by AKF (Afg), including record-keeping for the pre-defined/agreed timeline. Ignorance of any ethical norms will be considered a breach of mentioned norms and will lead to a follow up of AKDN/AKF-A’s policy and procedures including the relevant law of Afghanistan.
The Evaluator should be aware that this opportunity is offered on a quality output-based contract basis and that AKF (Afg) reserve the right to cancel this contract where this is not achieved.
10. APPLICATION DOCUMENTS REQUIRED
Please note that all the following documents are required to be submitted with the proposal and that failure to supply these will result in an automatic disqualification from the process:
- Detailed technical proposal - this should demonstrate the following :
o Understanding of the ToR, and ability to respond to all necessary requirements by being to the point,
o Very good technical understanding of climate change theories including the examples of some practical and applicable models related to previous experience in the same field,
o Good technical M&E knowledge and experience and the ability to apply best practices to this contract,
o A clear understanding of the subject matter,
o Good technical analysis skills including choice of methodologies, limitations, and bias, etc;
o An understanding of key principles such as beneficiary engagement and feedback.
The review panel will also be using these criteria to assess the bids.
- Detailed budget including a breakdown of costs per each category,
- CVs of the key team members who are currently working with the firm/consultancy, as individual consultants.
- Appropriate business registration documents of the firm,
- Anti-corruption and counter fraud policy of the firm,
- Clearly defined set of procedures for the employment of women and interviewing women beneficiaries in the light of existing regime’s rules – could be part of the proposed methodology as well.
Applicants should be aware that no communication should be attempted between an organization/staff and AKF (Afg) with respect to this opportunity outside those channels and times specified by AKF-A; any such contact will result in the removal of the relevant organization from the bidding process. AKF (Afg) will provide feedback for unsuccessful Evaluators if requested.
11. CONFIDENTIALITY:
All documents, data and reports under this assignment should not be used for any other purpose without prior written approval from AKF (Afg). Thus, the overall gathered data and information would need to be comprehensively handed over to AKF (Afg)’s MERL.
Evaluators may:
a. Register their interest to submit a proposal to jobs.afghanistan@akdn.org.
b. Put queries in writing via email to the HR department. All queries will be shared with Evaluators who have registered their interest in submitting a proposal, via email as needed.