Home > Results and review > Evaluation > Evaluation types

Results and review
Types of evaluations

We use a range of tools and methods to evaluate the investment schemes and processes, and outcomes from our investments.

Evaluations can be based on:

  • case studies of research programmes
  • surveys of research users, private firms, or stakeholders
  • outcome performance indicators
  • internal and external evidence of investment processes.

Many evaluations use more than one method. By combining qualitative and quantitative techniques for collecting information, we are able to achieve better coverage and depth in the evaluations - for example, an internet-based survey, together with a small number of face-to-face interviews and/or case studies.

Examples of evaluation types

Case studies

Case studies provide detailed information about the nature and impact of benefits arising from research. They also examine processes, such as developing of research capability, which affect the benefits achieved from research programmes.

Standard case studies

Standard case studies involve structured face-to-face interviews with researchers and examining secondary sources of information such as research organisations' websites. In some cases, we also carry out follow-up discussions with a selection of people who use the research results.

Cost benefit studies

Cost benefit studies are prepared in a similar way to standard case studies, except there is a strong focus on quantifying the net benefits to New Zealand from Research Science and Technology (RS&T).The studies also typically involve wide discussions with research teams, users of research results, and other stakeholders.

Surveys

Electronic surveys

Internet and email-based surveys provide us with information on outcomes from research we have invested in and on our investment processes.

Research-user surveys

Research-user surveys provide outcome information from users of Foundation?-funded research. The surveys cover the benefits attributed to Foundation-funded research and research teams, the uptake and application of Foundation-funded research by users, and issues that affect the implementation of research results.
We also look at how users were involved with Foundation-funded research teams and their levels of commitment to research programmes, along with any other issues, such as the impact of our processes and strategies.

Business surveys

Business surveys evaluate the performance of firms who have completed Foundation-funded technology development projects. For example, we have surveyed businesses 18 months after the completion of Technology for Business Growth (TBG?) contracts regarding outcomes from one TBG funded project.

These surveys typically examine firms’ performance in terms of impacts resulting from the Foundation-funded projects, such as new products or processes, intellectual property, revenue and exports.

Stakeholder surveys

Stakeholder surveys seek information from a wide range of people and groups on the overall effectiveness of the Foundation. Stakeholders include research organisations, private businesses, industry and sector representatives, government agencies (regional and national), and other interest groups.

The stakeholder surveys cover all of our research, science and technology funding schemes. They look at stakeholders’ perspectives on our investment strategies, investment signals, investment processes and procedures, communications, and effectiveness.

Outcome indicators

Research organisation with public good research contracts are asked to supply us with information about the success of their projects. We ask for measures of:

  • benefits (such as new or improved products, processes or services)
  • revenue (such as co-funding)
  • knowledge generation (such as patents)
  • knowledge transfer (such as journal articles)
  • sector relationships (such as user collaborations)
  • capability development (such as PhDs).

These measures show us how the outcomes resulting from our funding change over time.

Process evaluations

Evaluations of our investment processes enable us to enhance our operational efficiency. These studies are completed either internally or by an independent consultant and involve a variety of approaches, such as internal reviews and external surveys or consultations.