University research performance: Influence of funding competition, policy steering and micro-level factors

Abstract

Rahoituskilpailu on tehoton keino parantaa yliopistotutkimuksen tuloksellisuutta

Yliopistojen tutkimuksen tuloksellisuus on noussut tärkeäksi tiede- ja korkeakoulupoliittiseksi tavoitteeksi 1980-luvulta lähtien monissa maissa. Yliopistoihin tehdään suuria taloudellisia panostuksia ja niiden odotetaan tuottavan korkeatasoista tutkimusta, jonka tulokset ovat myös yhteiskunnallisesti ja taloudellisesti hyödyllisiä. Useat valtiot ovat lisänneet kilpailua tutkimusrahoituksesta sekä ohjanneet yliopistoja tehostamaan ja kehittämään toimintaansa autonomisina ja samalla tilivelvollisina organisaatioina. Mutta parantaako tällainen politiikka yliopistotutkimuksen tuloksellisuutta?

Väitöstutkimuksessa tarkasteltiin rahoituskilpailun ja valtioiden harjoittaman yliopistojen ohjauksen vaikutusta yliopistotutkimuksen tuloksellisuuteen kahdeksassa maassa. Lisäksi tutkittiin rahoituskilpailun vaikutusta tuloksellisuuteen kolmessa suomalaisessa yliopistossa. Analyysien aikajänne on 1980-luvulta 2000-luvun puoliväliin. Näitä analyysejä täydennettiin pohjoismaisten yliopistojen sosiologian alan tutkijoiden julkaisutoiminnan ja tutkimustuloksellisuuden tarkastelulla.

Kansallisten yliopistojärjestelmien ja suomalaisten yliopistojen tuloksellisuuden analyysissä hyödynnettiin tieteellisiä julkaisuja, viittauksia ja tohtorintutkintoja kuvaavia indikaattoreita. Sosiologian tutkijoiden julkaisemista tarkasteltiin tilastollisilla monimuuttujamenetelmillä.

Väitöskirjan tulokset osoittavat, että rahoituskilpailu tai markkinamekanismeja painottava valtiollinen ohjaus eivät yksiselitteisesti johda yliopistojen tutkimustuloksellisuuden paranemiseen. Kansallisen tason analyysin perusteella voidaan myös todeta, että rahoituskilpailun lisäämistä tai markkinamekanismeihin perustuvaa valtio-ohjausta ei välttämättä tarvita tuloksellisuuden nostamiseen. Toisaalta analyysit viittaavat siihen, että rahoituskilpailu voi parantaa tuloksellisuutta lyhyellä aikavälillä sekä kansallisella että yliopistojen tasolla.

Pohjoismaisia sosiologian tutkijoita koskeva analyysi osoittaa kuinka pysyviä ovat eri tieteenaloilla vallitsevat julkaisemisen kulttuurit. Sosiologien julkaisukäyttäytymisessä korostuu edelleen tieteellisten monografioiden ja kirja-artikkeleiden julkaiseminen ja niihin viittaaminen. Myös tutkijoiden väliset erot tuloksellisuudessa ovat hyvin vakaita: vähemmistö tutkijoista on selvästi muita tuottavampaa julkaisutoiminnassa, ja heidän julkaisuihinsa viitataan suhteessa merkittävästi enemmän kuin tutkijoiden enemmistön julkaisuihin.

Väitöskirjan pääasiallinen johtopäätös on, että rahoituskilpailuun ja markkinaperustaiseen ohjaukseen tukeutuva politiikka vaikuttaa olevan verrattain tehoton keino parantaa yliopistojen tutkimustuloksellisuutta pitkällä aikavälillä. Tätä tukeva johtopäätös on, että tiedeyhteisöjen arvo- ja palkkiojärjestelmä voi heikentää tiede- ja korkeakoulupoliittisten toimien vaikutusta. Tutkimustyön ruohonjuuritasolla korkeatasoinen ja tuloksellinen tutkimustoiminta edellyttää sellaisia tekijöitä, joihin ei voi riittävästi vaikuttaa kilpailua lisäämällä. Lisäksi kilpailun lisäämisen sivuvaikutukset, kuten lyhytjänteisyys ja resurssien kasautuminen, saattavat itse asiassa heikentää tuloksellisen tutkimustoiminnan edellytyksiä.

This dissertation explores the influence of science and higher education policy and the micro-level factors of research environment on university research performance. The main objective is to analyze university research in a context where high research performance is regarded as an important goal in science and higher education policies and to ask if this performance-oriented science and higher education policy is beneficial for universities’ research performance. To achieve this objective, I first study universities’ research funding both at national and university level. I then analyze the policy models which governments have used to steer university sector. Finally, I present an analysis of university research performance at national and university level, and scrutinize the influences of competition for funding and steering models on research performance. These analyses are longitudinal, ranging from the 1980s or the early 1990s to the mid-2000s, and include comparisons of several, mainly European, countries and comparisons of Finnish universities.

The second objective of the dissertation is to study research performance at the micro-level of university systems. This part of the dissertation focuses on the field of sociology at the Nordic universities and asks: what are the patterns of research performance among Nordic sociologists and factors which influence it? The analysis of Nordic sociology is also aimed at testing an alternative data source for the analyses of research performance, namely the Google Scholar web search engine.

My study relates to the discussions about trends in science and higher education policies during the past 20-30 years in several developed Western countries. The nature and effects of these trends has been debated in higher education research and science policy studies. Many studies have shown that improving universities’ research performance has become a significant policy goal in the post-industrial societies. The ideology of New Public Management, emphasizing accountability, cost-effectiveness and competition in the public sector, has influenced policy-making so that universities are expected to be accountable for the large (financial) investments on them. Another point of reference for my study is the discussion on the position of universities and scientific research in the so called knowledge-based economy. Many scholars in innovation studies, science studies and higher education research argue that after the 1980s, knowledge-creating institutions such as universities and science have been of strategic importance for the post-industrial societies. Third, the dissertation stems from the research traditions of sociology of science and bibliometrics, where the patterns of research performance among scientists and factors affecting these patterns have been studied.

The major contribution of my dissertation to existing research is the analysis of connections between performance oriented policy and research performance, at both national and university level. With a few exceptions, previous research has largely concentrated on studying either policy trends and instruments (such as funding) or research performance. Furthermore, it has been customary to focus on a particular level of university system. Another contribution comes from the use of Google Scholar for analyzing publication productivity and citation visibility in sociology. The majority of studies on research performance focus on natural sciences and medicine and use citation databases such as the Web of Science as data sources. Google Scholar provides a more comprehensive perspective on scientific publishing and citation patterns than citation databases. Results of the analysis of the Nordic sociologists’ research performance also enable me to reflect upon the relationship between performance-oriented policy solutions and the norms and values of scientific communities.

I employ several conceptual resources in my study. The funding environment of university research is analyzed using an analytical framework that indicates the level of competition in the funding environment. The framework was developed in collaboration with Mika Nieminen. Gornitzka and Maassen’s idea of the four state steering models for higher education is applied for studying the governmental steering of universities. The principal-agent theory, which originates from economics, has later been used, for instance, to study the relationships between the state (principal) and universities (agents). I utilize the theory to frame the development of science and higher education policies from the 1980s to the 2000s. I approach the patterns of research performance and scientific publishing among the Nordic sociologists by using the concepts of disciplinary publication practices and cumulative advantage in science.

The comparisons of national level funding systems and research performance include a total of eight countries: Australia, Denmark, Finland, Germany, the Netherlands, Norway, Sweden and the United Kingdom. The state steering models have been analyzed in the cases of five countries: Australia, Finland, the Netherlands, Norway and the United Kingdom. University level analysis on research funding and performance includes three Finnish universities: the University of Helsinki, the University of Jyväskylä and the University of Kuopio. The data on research funding consists of statistics and documentary material from the OECD and from the national sources. Analysis of the state steering models is based on documents and accounts in research literature. The data for the national and university level studies of research performance is drawn from the Web of Science citation database and from the Finnish higher education database. The analysis of Nordic sociologists is based on the data from the Google Scholar and from the websites of the sociology departments.

I employ multiple indicators on publications, citations and doctoral degrees for analyzing the research performance at national and university level. The framework of funding environments is used for the analyses of research funding. In the case of the state steering models, the analysis is based on interpretation of documents and research literature. A multi-level regression model is used to study Nordic sociologists’ research performance in order to isolate the effects of individual and departmental factors on productivity and visibility.

The results of the dissertation indicate that, at the level of national university systems, improvements of research performance are not an automatic and direct result of the competitive funding environment or of a state steering model that relies on market mechanisms. Furthermore, the national level data shows that university systems like those in Sweden, Denmark and the Netherlands can be high-performing and/or improve their performance in conditions of a relatively low level of competition for funding or when the state steering is not based on a market-type steering model. However, short-term improvements in research performance can follow from increasing competition for funding, as happened in Norway and in the United Kingdom. The university-level analysis also points to the relative ineffectiveness of a competitive funding environment in improving research performance, except for the University of Jyväskylä. However, even in Jyväskylä the effect of competition was not very long-term.

The analysis of the patterns of publication productivity and citation visibility among Nordic sociologists shows the persistence of disciplinary cultures. The English-language, international scientific publications understandably attract more citations than the publications in Scandinavian languages, because of the larger audience of the English-language publications. Still, sociologists continue to publish and cite monographs and articles in books. In a similar vein, the differences of research performance among individual researchers persist. The minority of researchers are much more productive in publishing and are cited disproportionally more often than the majority.

The main conclusion of my dissertation is that policy measures based on competition for funding and market-type steering appear to be relatively ineffective instruments for improving research performance in universities in the long term. A related major conclusion from the findings of the micro-level analysis is that the reward and value system of science is potentially able to check the influence of policy instruments. I conclude the study by presenting potential explanations for the mixed success of performance oriented policy solutions, reflect some of the limitations of my analysis, and suggest themes and directions for further research on research performance.