Measuring digitalization at scale using web scraped data

Abstract

Measuring digitalization has been a central topic in academic discourse. While evaluating firms' efforts in increasing digitalization is crucial, quantifying it at scale, presents considerable challenges. This paper uses website information as a source of data to operationalize a measure of digitalization. Drawing on a sample of 60,942 firms, our approach proposes two distinct measures of digitalization: one at the product level and the other at the general organizational level. We substantiate these measures using a blend of qualitative and quantitative methods. The study validates the content of websites as a relevant source of innovation indicator data and verifies the indicators using multiple experiments. The developed digitalization indicators offer future research an empirical measure of digitalization that can be run at scale, across industries and regions through time.