Other schemes:
Outcome Evaluation of the Cross Departmental Research Pool (CDRP)

New Zealand Institute for Economic Research, November 2005.

Scope:

The substantive evaluation of the CDRP was agreed between the Foundation? and the Ministry of Research Science and Technology as part of the full transfer of the scheme to the Foundation for 2004/5.

The evaluation sought information from Government departments and agencies on CDRP processes, outcomes and benefits attributable to completed CDRP projects.

The evaluation involved the following:

• A review of successful and unsuccessful CDRP applications.
• Obtaining generic views on CDRP effectiveness – i.e. a survey covering 31 government departments (68% response).
• Interviews on CDRP effectiveness and outcomes – i.e. with lead and collaborative agencies from 10 selected projects (funded between 2000 and 2003).
• Feedback from the departments on the findings and recommendations in the draft final report.

Summary

Allowing for the circumstances that impact CDRP projects and outcomes, the evaluation results do indicate some positive findings:

1. The fund is meeting a real need for cross-departmental research that would otherwise not be funded by the departments involved.

2. CDRP continues to have the confidence of government departments as a fund with the potential to support strategic cross-departmental objectives and contribute to outcomes related to government strategic policies.

NZIER note that ‘there was agreement, but not especially strong, that the process lends itself to undertaking of policy-relevant, high-quality and well-managed research’. This evaluation finding can be seen as a realistic assessment of the fund by those involved.

3. The fund has contributed to some departments working more closely in collaborative relationships. Existing relationships have been strengthened, however, rather than new relationships developed.

4. Evaluation participants were evenly split on whether their projects encouraged their agencies to invest in long-term, high-quality research. There was some suggestion that CDRP projects reinforced organisations’ commitments to long-term research or that organisations already have a long-term research focus.

5. The majority of participants reported that their projects had not provided the basis for significant government decisions.

Conclusions

CDRP is a small fund and the objectives of the fund are both wide-reaching and profuse. Within a public policy sector where there are variable processes in strategic policy development and cross-departmental interactions,

CDRP projects appear to have made a modest impact to date. In practical terms, the pathways from research to policy development and outcomes have been, and continue to be, complex. This is particularly the situation in social policy, which has been the predominant area of CDRP funding to date. These issues are not unique to CDRP, however.

The evaluation findings suggest there is unlikely to be any significant changes to the level of achievement of cross-departmental outcomes from the scheme under the current objectives and processes.

The CDRP processes were not the subject of major criticism. Proactive communication and consultation by the Foundation and the Ministry prior to a funding round would greatly assist in ensuring that the most strategically important areas for cross-departmental research are included the proposals submitted.

The question of the strategic direction of CDRP proved to be largely out of scope of the evaluation. On this question, a further review is needed to determine whether there is support among departments for a different funding strategy for CDRP (e.g. pre-defined cross-departmental strategic priorities for funding). The outcome evaluation itself does not support major changes to the current strategy that facilitates a wide-range of cross-departmental research.