Procurement Notices

Associate Evaluators for Case Study Development: Assessment of the Implementation of the Anti-Red Tape Act (Two [2] Individual Consultants)

1. Project Title

Using Strategic Monitoring & Evaluation (M&E) to Accelerate the Implementation of the Philippine Development Plan (PDP) 2017-2022 (Strategic M&E Project)

2. Background and Rationale

The National Economic and Development Authority (NEDA) and the United Nations Development Programme (UNDP) Philippine country office have embarked on a partnership to strengthen the conduct of evaluations of priority government programs under the Philippine Development Program (PDP). Financed by NEDA and implemented with full UNDP country office support, the Strategic Monitoring and Evaluation (M&E) Project will help strengthen the M&E capacities of NEDA and key government agencies to support the achievement of the PDP and the Sustainable Development Goals (SDGs) through evidence-based decision making.

A key component of the project is the commissioning of independent evaluations on key themes and programs relevant to the PDP and the SDGs. These studies will evaluate the relevance, efficiency, effectiveness, and sustainability of priority social and economic programs that have been implemented or are being implemented by the government. The results of the evaluation studies are envisaged to inform how policies and programs are designed and implemented to achieve the desired results of the PDP and contribute to strengthening the government’s M&E systems.

One of these evaluations is on the implementation of the Anti-Red Tape Act (ARTA). Passed in 2007 as Republic Act (R.A.) No. 9485, ARTA seeks to promote integrity and curb corruption through greater transparency about the public’s transactions with the government. The Act required the simplification of frontline service procedures, formulation of service standards to be observed in every transaction, and publication of these standards for clients’ information. ARTA applies to all government offices, national government agencies (NGAs), local government units (LGUs) as well as government-owned and controlled corporations (GOCCs) that provide frontline services. In adopting simplified and standardized procedures, the law sought to reduce red tape and thereby curb opportunities for graft and corruption. Recently, the Ease of Doing Business Act and Efficient Government Service Delivery Act (EODB Act, R.A. No. 11032), also known as the Expanded ARTA law, was passed to strengthen the service efficiency aspects of the 2007 law and to emphasize improving business competitiveness.

To assess agencies’ compliance with ARTA and clients’ satisfaction on frontline services, the Civil Service Commission (CSC) has been conducting the Report Card Survey (RCS). Introduced in 2010, the RCS is held annually to rate frontline service offices’ compliance with ARTA requirements such as the presence of Citizen’s Charter, manned Public Assistance and Complaints Desks, Anti-Fixer campaign materials, and observance of “No Noon Break” policy. Apart from checking on these requirements, the RCS also surveys a sample of clients to assess their satisfaction on the services rendered by the frontline service office. CSC also collects bulk data on complaints from citizens through the Contact Center ng Bayan (CCB)

An independent evaluation is necessary to gain an objective view of the implementation by NGAs, GOCCs, and LGUs of the ARTA; the results so far in terms of outcomes and, if feasible, impacts; and the mechanisms for monitoring and evaluation of the Expanded ARTA law.

NEDA, through the UNDP, will commission such evaluation through the hiring of a team of individual consultants, including the Two Associate Evaluators for Case Study Development. Working with the evaluation team that will be led by a Governance Specialist / Lead Investigator, the associate evaluators will develop at least six (6) case studies of agencies who have improved their performance and whose performance have stagnated or worsened. An Evaluation Reference Group (ERG) has also been convened, with NEDA, CSC, DTI / National Competitiveness Council (NCC), and UNDP as members, to provide direction and input to the design and conduct of the study. 

3. Objectives of the Study

The proposed study on the implementation of ARTA aims to develop its relationship with and impact on frontline services and provide inputs to the implementation of the new EoDB law. The study seeks to draw lessons from the implementation of ARTA over the last eight (8) years, particularly, analyzing trends on the implementation of ARTA, identifying the elements and practices that helped improve the efficiency of frontline services and those which constrained the effectiveness of anti-red tape interventions, as well as to surface the law’s unintended consequences. Furthermore, the study seeks to design a standard framework and methodology for the conduct of future impact evaluations on the implementation of the Expanded ARTA.

The study seeks to answer the following evaluation questions:

1. Efficiency –  
   1. How has ARTA been implemented by NGAs, GOCCs, and LGUs? 
   2. What can the RCS and other databases of CSC reveal about ARTA implementation?
   3. How do agencies monitor their compliance to the standards in their Citizens Charters and revise the standards when necessary?
2. Effectiveness –  
   1. How has the implementation of ARTA policies improved frontline services? What are the elements which helped achieve the goals of ARTA, and what are the constraining factors?
   2. Has the improvement in frontline services of key national and local agencies translated in improved perceptions on business-friendliness and corruption? Why or why not?
   3. What factors influence various stakeholders in the implementation of ARTA? How do they collaborate to meet (or curtail) the goals of ARTA?
   4. What are good practices from the most improved agencies (provide innovative examples)?
   5. What support can be extended to agencies in difficult situations or those who fail in the RCS?
   6. What are the facilitating factors and barriers to the public’s utilization of the Citizen’s Charter?
3. Relevance – To what extent are the various components of the ARTA program relevant to the implementation of the new Expanded ARTA?
4. Sustainability –
   1. Based on the findings, how can ARTA be better implemented in light of the new EODB law?
   2. Are there areas that need further improvement to sustain/maximize the benefits already achieved by implementing ARTA?
   3. How should a future impact evaluation study for the Expanded ARTA be designed?
   4. What methodologies can be applied to assess the costs and benefits associated with ARTA?
   5. What methodologies and tools can be applied to measure NGAs’ implementation of the standards (e.g. processes, steps, and time) in their Citizens’ Charters?
   6. How can data collected by CSC be used to analyze the causal impact on ARTA outcomes, i.e., reducing corruption and improving ease of doing business?

4. Scope of Services and Methodology

Under the overall guidance of the ERG of the ARTA study and reporting directly and regularly to the Strategic M&E Project Coordinator of UNDP, the consultants will:

1. Assess the efficiency and effectiveness of the implementation of entire ARTA program throughout the government and in selected NGAs in terms of:
   1. Processes and activities undertaken to comply with ARTA;
   2. Challenges and issues, including capacity constraints;
   3. Best practices, enabling factors, and lessons learned;
2. Provide insights as to the extent the ARTA program has achieved its desired results or objectives of improving frontline services, easing doing business, and curbing corruption as well as the program’s unintended consequences through perceptions from key sectors such as academe, civil society, and private sector, and provide recommendations on how to sustain the efficiencies already achieved as well as to address the gaps;
3. Design a standard framework and methodology for the conduct of future impact evaluations on implementation of the ARTA program, including recommendations on tools, techniques, and innovations that can be used to improve monitoring and data collection.
4. In collaboration with the ERG, define a strategy to communicate and disseminate the results and findings of the evaluation study, including a) mapping of stakeholders with interest in the evaluation; b) a one-page brief and a maximum five (5)-page executive summary to accompany the dissemination of the report.

In undertaking the evaluation, the evaluation team will undertake two methodological tracks. First, a quantitative track which will involve statistical testing and analysis of the existing data gathered by CSC through its RCS, inspection checklist, contact center complaints, and other means and tools to a) define the statistical reliability of the data; and b) draw trends and other insights from the data on how ARTA has been implemented by the agencies.

For the qualitative track, which will be led by the associate evaluators for case study development, at least (6) case studies of NGAs/GOCCs (at least two (2)) and LGUs (at least four (4)) with relatively comparable service profiles and with equal number of improved and stagnant performers will be undertaken to a) identify the drivers and other factors for ARTA performance; and b) identify change in key stakeholders’ perceptions of corruption and ease of doing business. Key informant interviews and focus group discussions (FGDs) with government and non-government stakeholders (e.g., civil society, business, academe) will be conducted at the central office and select service delivery offices. Available data from the subject NGAs/GOCCs and LGUs will be analyzed. The two (2) associate evaluators will share an equal number of NGAs/GOCCs and LGUs.

From the findings from these two tracks, the evaluation study will design a standard framework and methodology for the conduct of impact evaluations in the future on the implementation of the ARTA program in the context of the recent amendments. The framework may include the development of a case study tool for the regular in-depth assessment of the top ARTA performers.

To undertake such, the Associate Evaluators for Case Study Development to be engaged for the study shall collectively undertake the following:

1. Preparatory work, including:
   1. Prepare case study tool and guide questions for the conduct of the study; and
   2. Identify proposed case studies and key informants
2. Field Work, including:
   1. Gather data for the case studies through the conduct of key informant interviews, facilitation of focus group discussions, and other means;
3. Analytical Work, including:
   1. Review and synthesize relevant literature and documents;
4. Reporting, Presentation, and Dissemination of Results:
   1. Prepare and submit a final report, subject to review by the ERG for this study, including the raw and processed data used in building the report;
   2. The consultants will also be tasked to make a presentation of preliminary or final results of the evaluation in a public forum;
   3. The consultants will also i) condense the findings in a one-page summary and a maximum five (5)-page executive summary, and ii) recommend strategies on how to communicate and disseminate the evaluation study and its findings.

The two (2) associate evaluators will each develop an equal share of the total number of case studies to be agreed up on with the UNDP and Lead Investgator, which shall be a total of two (2) NGAs/GOCCs and four (4) LGUs or their equivalent. The lead investigator is expected to oversee work and assure the quality of the outputs of the associate evaluators. The associate evaluators are expected to contribute draft sections to the evaluation report as agreed upon with the Lead Investigator and the Strategic M&E Project Coordinator of UNDP.

5. Expected Outputs

The evaluation team is expected to deliver the following:

1. An Inception Report collectively submitted by the team, as compiled by the lead investigator, which spells out the analytical framework, detailed methodology, and work plan for the evaluation. It should include the list of proposed NGAs/GOCCs for case study, including targeted locations of frontline service delivery offices (e.g., branches) to be visited, and the division of assignments among the associate evaluators. The inception report should include as attachments the following from the associate evaluators for case study development:  
   1. Interview and discussion questionnaires;
   2. List of targeted key informants to be interviewed and/or included to focus group discussions (FGDs).
2. A Draft Evaluation Report, according to the template or outline agreed upon at the inception stage, that provides an initial consolidation of the findings and recommendations of the study based on the data gathered and analyzed so far. Moreover:
   1. The draft report should include as attachments at least half of the case studies produced by the associate evaluators, as reviewed by the lead investigator.
   2. After presentation to the ERG, the consultants shall prepare a matrix or documentation of inputs from the ERG, with feedback on the inputs to be accepted and those not to be accepted and reasons for such recommendations.
3. A Final Draft Evaluation Report which refines and completes the consolidation and synthesis of the findings and recommendations of the study based on all the data gathered and analyzed. The final draft evaluation shall also include:
   1. A proposed framework for evaluating the impact of the implementation of the Expanded ARTA and other recommendations to improve the collection of data to prepare for quality analyses;
   2. Proposed recommendations or action items for implementing agencies on improving ARTA implementation, presented in a matrix or other relevant format;
   3. All the case studies produced by the associate evaluators and reviewed by the lead investigator, as attachments to the report and with summary boxes/sidebars integrated into the report proper;
   4. After presentation to the ERG, the consultants shall prepare a matrix or documentation of inputs from the ERG, with feedback on the inputs to be accepted and those not to be accepted and reasons for such recommendations.
4. A Final Publication-Ready Evaluation Report which makes necessary refinements or adjustments to the report based on the management responses to be provided by the CSC and other implementing agencies. The consultants are expected to respond to each of the Management Responses, in consultation with UNDP and NEDA. The consultants shall also:
   1. Present the evaluation report to a public forum, which may be the annual M&E Network Summit, as decided upon by the ERG. In doing so, they shall prepare a presentation slide deck;
   2. Produce condensed versions of the report, namely: i) a one-page summary of the key findings of the study; ii) a maximum five (5)-page executive summary;
   3. Submit a proposed communication plan containing recommended strategies and actions for communicating the study and its results to key stakeholders and the general public.

The consultants are expected to deliver the said outputs according to the following schedule:

The Associate Evaluators for Case Study Development are also each expected to submit:

1. Monthly Status Reports, consisting of maximum two (2) pages, outlining a summary of activities undertaken and deliverables submitted, as well as issues encountered and lessons learned;
2. Raw and Processed Data used in the study, including but not limited to interview transcripts and/or notes, interview results/thematic matrices, and all other documentation made during the conduct of the case studies, together with each version of the Report submitted as well as when required by UNDP or NEDA (through UNDP). 

6. Key Performance Indicators and Service Level

1. Timely delivery of quality outputs for the evaluation according to the timetable.
2. Effective communication and collaboration with stakeholders of the study, including but not limited to the ERG and its members and implementing NGAs/GOCCs subject to the case study.
3. Effective use of quantitative and qualitative research methodologies that are appropriate to the evaluation study, including introduction of innovative tools and techniques as well as identification/use of non-traditional sources of data.
4. Usefulness and relevance of the study findings and recommendations to the needs of the NEDA, CSC, and other implementing agencies of the ARTA.

7. Governance and Accountability

1. The ERG of the ARTA study, which is composed of NEDA, CSC, NCC, and UNDP, shall provide overall direction, guidance, and input to the conduct of the study. The consultants shall be required to submit and present their Reports to the ERG, through the UNDP Project Coordinator; attend meetings convened by the ERG on the study; and consider the inputs of the ERG and its members as indicated in Section E of this contract.
2. The Consultants shall be directly supervised by the Project Coordinator of the NEDA-UNDP Strategic M&E Project, with whom all outputs shall be submitted and through whom all communications shall be coursed or copied. The Associate Evaluators shall likewise be supervised by the Governance Specialist / Lead Investigator;
3. When necessary, the Consultants, through the Lead Investigator, may be required to coordinate with the Senior Adviser/Economist, Management Support Unit Team Leader, and other officials of UNDP; and with the OIC-Director of the Monitoring and Evaluation Staff (MES) and the Director of the Governance Staff (GovStaff) and other officials of the NEDA.
4. The Consultants, through the Lead Investigator, shall report progress, provide updates, or raise issues to the Project Coordinator on a weekly basis. The Consultants are expected to be accessible to the UNDP Project Coordinator via phone, mobile, or internet, and may be asked to report physically to UNDP when warranted.
5. In conducting their research, data gathering, and fieldwork, the consultants shall coordinate with the NEDA, CSC and other relevant government agencies, directly or through the UNDP. The NEDA and UNDP shall provide the necessary endorsements, including endorsement letters and calls, to the implementing agencies to be covered by the case studies.
6. The Consultants shall also report to/coordinate with the ERG and their members and shall take note of and act on their recommendations and suggestions unless these are not feasible or are otherwise disapproved by NEDA and UNDP.

8. Expected Duration of the Contract

1. Each consultant will be hired for a period of five (5) months, in accordance with the timetable set forth in Section E above and Annex 2.
2. The target start of work date is 1 July 2018 and the maximum end date of the contract is 30 November 2018. The contract may be terminated earlier if all the outputs have been submitted satisfactorily and accepted.

9. Duty Station

1. The Consultants will be based in Manila but should be able to travel to locations of consultations within the country, as required by the evaluation studies assigned.
2. The Consultants will be required to report physically to UNDP on a fortnightly basis or as agreed during the inception report, and when physical participation in activities will be necessary.
3. The Consultants are expected to have their own work spaces, computers/laptops, and other facilities and equipment. UNDP may provide an intermittent working area in its premises only if available and if arranged in advance.

10. Professional Qualifications of the Successful Consultants

The Associate Evaluators shall each have the following minimum qualifications:

1. Filipino nationals or expatriates currently residing in the Philippines only;
2. At least two (2) years of work or consultancy experience in the monitoring and evaluation of development programs and projects, with preference to those with demonstrated specialization/experience in evaluations; 
3. At least four (4) years of overall work or consultancy experience in the government, international development organizations, non-profits, among others. Previous employment or consultancy experience with UN agencies is an asset;
4. At least an undergraduate degree in economics, political science, social science, public administration, management, or other relevant fields. A master’s degree is advantageous;
5. Having published research work in relevant policy/program areas and/or research output from consultancy projects is preferred. Research works may include applied research studies, e.g. evaluation, action research, policy papers, etc.;
6. Having satisfactorily completed projects with the public and/or development sector, including but not limited to government, international development organizations, international financial institutions, and non-government organizations is preferred;
7. Demonstrated knowledge of and experience in the application of various research methodologies, with demonstrated specialization in qualitative research. Demonstrated knowledge of quantitative research, in addition to the requirement, is advantageous;
8. Demonstrated specialization, through professional and academic qualification, in any of the following policy and program areas is advantageous:
   1. Democratic Governance – including but not limited to anti-corruption, government efficiency and organizational development, public finance management, results-based management, public administration, among others;
   2. Economic Development – including but not limited to economic and fiscal policy, transport and non-transport infrastructure development, competitiveness and entrepreneurship, agriculture policy, industrial policy, services and trade, among others.   
9. UNDP gives preference to consultants who demonstrate the following values:
   1. Gender Equality – achieving gender equality and gender mainstreaming are key principles and strategies of UN system agencies, therefore, UNDP encourages the deployment of staff and consultants (male/female) at a balanced ratio based on the requirements. As such, in forming the evaluation team, UNDP will strive to achieve a balanced composition of consultants.  
   2. Integrity – UNDP implements a policy of zero tolerance on proscribed practices, including fraud, corruption, collusion, unethical practices, and obstruction. UNDP is committed to preventing, identifying and addressing all acts of fraud and corrupt practices against UNDP as well as third parties involved in UNDP activities. UNDP requires all proposers to conduct themselves in a professional, objective and impartial manner. Proposers must strictly avoid conflicts with other assignments or their own interests and act without consideration for future work. UNDP reserves the right to reject bidders who are in an actual or potential conflict of interest or have been involved in unethical practices.

11. Scope of Price Proposal and Schedule of Payments

1. Financial proposals must be expressed as a lump sum “all inclusive” rate for the whole duration of the contract. The term “all inclusive” implies all costs (professional fees, allowances, travel costs within Manila, etc.). The lump sum price is fixed regardless of changes in the cost components. The financial proposals shall not include the cost of domestic travel and lodging for fieldwork, venue, and food & beverage for FGDs and out-of-town KIIs, which will be provided by the UNDP;
2. The Consultants shall receive payments based on the following schedule:

12. Recommended Presentation of the Offer

Applicants are requested to submit the following documents to procurement.ph@undp.org.

1. Duly accomplished Letter of Confirmation of Interest and Availability using the template provided by UNDP;
2. Personal CV or P11, indicating all past experience from similar projects or requirements, as well as the contact details (email and telephone number) of the Candidate and at least three (3) professional references;
3. Portfolio of Prior Research Work, with Evaluation Studies listed first followed by published and unpublished research outputs. A list with hyperlinks to publicly accessible websites or shared cloud storage, and a sample research output, is required at this point. Printouts of the research outputs are discouraged. Include in the list a) information on contract cost, duration, and key outputs/performance indicators; b) statements of satisfactory performance from the top clients in terms of contract value in the past and these clients’ contact information;
4. Portfolio of Prior Projects, indicating title and summary of completed projects, sponsoring and implementing organization, contract cost and duration, hyperlink to available information on the project, role and deliverables in the projects, results of the project, and statements of satisfactory performance from top clients in terms of contract value and these clients’ contact information;  
5. Brief description, separately or as part of the CV, of why the individual considers him/herself as the most suitable for the assignment, highlighting specialized background advantages for the scope of work and requirement.

13. Criteria for Selection of the Best Offer

The offer will be evaluated based on a Combined Scoring method where the qualifications and technical proposal will be weighted a maximum of 70%, and combined with the price offer which will be weighted a maximum of 30%. The criteria for the technical scoring will be the following:

Top shortlisted bidders for either band may be called to an interview to further assess their technical proposal and/or fitness for the role.