Matrix factorization of large scale data using multistage matrix factorization

Abstract

Matrix Factorization (MF) is a resource intensive task that consumes significant memory and computational effort and is not scalable with the quantum of data. When the size of the input matrix and the latent feature matrices are higher than the available memory, both on a Central Processing Unit (CPU) as well as a Graphical Processing Unit (GPU), loading all the required matrices on to CPU/GPU memory may not be possible. Such scenarios call for alternative techniques that not only allow parallelism but also address memory limitations and plays a crucial role in industrial applications. In this paper we propose a divide and conquer technique based on a two stage factorization process. In the first step, we divide the data set into different groups and factorize each group. In the second step, we use factorization based learning model to combine the latent features derived in the first step. Our motivation is to develop a method that can achieve both parallelism and scalability as well as address factorization of incrementally growing data. Our contribution is a novel multi-stage matrix factorization (MsMF) approach. The experimental results demonstrate improvements in RMSE as well as computational efficiency.