Privacy-Preserving Machine Learning Based on Homomorphic Encryption : Evaluation of Activation Functions in Convolutional Neural Networks

Abstract

With an increased popularity of Machine Learning (ML) and Deep Learning (DL) companies have started to offer Machine Learning as a Service (MLaaS). These services are under threat due to vulnerabilities in privacy that potential attackers could take advantage. Privacy-preserving Machine Learning (PPML) guarantees that used data or ML model does not leak any information outside of the system.

The techniques of ML and DL perform well at image classification tasks. Convolutional Neural Network (CNN) is the ML model in this thesis and Homomorphic Encryption (HE) serves as the method to ensure privacy. The purpose of this thesis is to evaluate the time efficiency and accuracy of CNN model with different activation functions in PPML system by using the MNIST (Modified National Institute of Standards and Technology) dataset. Furthermore, the affects of encryption are examined. The MNIST dataset includes images of hand-written numbers that can be used to train and test ML model.

It can be seen from the results of experiments that the classification accuracy is consistent whether the data is encrypted or not. However, there is a significant difference on time efficiency if the data is encrypted. It takes over 100 seconds to classify one encrypted image regardless of chosen activation function in CNN. The same operation takes under one second on plaintext. The results show that low degree Chebyshev polynomial approximation of Rectified Linear Unit (ReLU) as the activation in CNN is a suitable option in the proposed PPML system.