Evaluating the performance of innovation intermediaries

Evaluating the performance of innovation intermediaries: insights from the experience of Tuscany’s Innovation Poles

DEMB – Working paper series n. 62, submitted for publication to fteval – Journal for Research and Technology Policy Evaluation (journal issue NO.41 – “EVALUATION OF INTERMEDARY ORGANIZATIONS”

Margherita Russo* , Annalisa Caloffi^, Federica Rossi**, Valentina Fiordelmondo, Stefano Ghinoi

* Dipartimento di Economia Marco Biagi, University of Modena and Reggio Emilia, Italy; corresponding author: margherita.russo@unimore.it
^ Departmente of Economics and managemente, University of Padua, Italy, annalisa.caloffi@unipd.it
** Birkbeck Univerity, London, United Kingdom, f.rossi@bbk.ac.uk


Within knowledge-intensive economies, intermediary organizations that support firm-level and collaborative innovation have gained increasing prominence (Howells, 2006; Lazaric et al, 2008). These ‘innovation intermediaries’ provide a range of knowledge-intensive services which might include, among others, technology foresight and technology scouting, supplier selection, R&D partnership formation, technical assistance in the realization of R&D projects, dissemination and commercialization of results, and technology transfer. Innovation intermediaries can also contribute to the success of innovation policies (see e.g. Kauffeld-Monz and Fritsch, 2013). Their role is particularly important for policies targeting micro firms and SMEs, since the presence of intermediaries may facilitate the exchange of knowledge and competencies with other organizations (large firms, universities and research centres) that have different languages, organizational cultures, decision-making horizons, systems of incentives and objectives (Howells, 2006; Russo and Rossi, 2009; Caloffi et al, 2015). Policies aimed at promoting local, regional and national development increasingly involve the public funding of organizations that perform at least some innovation intermediary functions: examples are the regional competitiveness poles in France, the Innovation Networks in Denmark, the Strategic Centres for Science, Technology and Innovation in Finland, the Local Enterprise Partnerships and the Catapult Centres in the UK.

With the growing importance of policies sponsoring innovation intermediaries, a need has emerged for appropriate instruments to evaluate their performance. Current approaches to assessing the performance of such organizations often rely on the use of indicators aimed at capturing the immediate outputs produced by the intermediaries. However, approaches that focus only on immediate outputs while neglecting the intermediaries’ role in stimulating widespread, long term changes in the innovation system they are supporting, are perceived as being unsatisfactory by both policymakers and intermediaries themselves.

Our paper proposes a reflection on the limitations of current approaches to evaluating the performance of innovation intermediaries, and discusses some feasible avenues for improvement. The discussion builds upon our analysis of a case study of publicly-funded innovation intermediaries in the Italian region of Tuscany. In the programming period 2007-2013 (effectively starting from 2010), the regional government of Tuscany funded the creation of ‘innovation poles’, which are innovation intermediaries (organized to provide a range of services, including brokering and matchmaking) that bring together a number of universities and innovative service providers with potential end-users of these services. Similar intermediaries have been funded in different countries often with different labels (innovative clusters, poles, platforms), but their main goal is always to promote the development of university-industry linkages.

The intervention that we studied operated in two phases. In a first phase, the Tuscany Region (DGR 1040/2010) identified a set of 12 key technologies/applications, and it promoted the creation of an innovation pole for each of these. After having identified objectives and rules of operation of these poles (different rules for poles of different size), the Region selected the 12 poles according to proposals for a three-year period. The poles did not receive any initial funding; instead, they were funded after one year, after the Region had assessed their progress and had ascertained the achievement of a minimum performance. In a second phase, the policymaker provided several incentives for firms to acquire services from these poles.

The evaluation of such a complex architecture requires the mobilization of different tools and techniques. Based on the experience of the innovation poles (our analysis relies on a mix of semi-structured interviews, a survey and a focus group with the organizations managing the poles and providing services for them), we examine the problems that have emerged with the system of indicators used by the Region. That assessment system has generated a set of incentives for the agents involved, and we discuss whether and in which direction it has influenced their behavior.

Then, we try to propose a framework of indicators that are able to grasp the many facets of the intervention and the many dimensions of technology transfer. The main focus is not simply on the direct outputs of the activities of publicly-funded innovation intermediaries, which would not have been achieved in their absence (e.g. new innovation projects realized, new intellectual property generated and sold/licensed, new partnerships formed, new products and processes launched), but more importantly on the permanent behavioural changes (Georghiou, 1998; Hyvarinen and Rautiainen, 2007; Gok and Edler, 2012) that the innovation intermediaries have stimulated in the organizations, particularly SMEs, they have worked with: how they have changed their way of innovating, rather than just how many innovations they have realized.

This analysis can be useful in defining the system of evaluating the performance of other infrastructure for technology transfer purposes similar to the poles of innovation, such as technological districts.