In this work, we explore a way for analyzing a mammography exam holistically. We present the design and implementation of a deep learning model that takes as input the two mammography views (CC and MLO), as well as all detected breast masses and micro calcifications, and produces an output consisting of a three-class classification of the entire exam: negative (or normal), benign, or malignant findings. The registration of the CC and MLO views of a mammography exam is a difficult task due to the difficulty in estimating the non-rigid deformations that can align these two views, so we propose the deep learning model classification, with the hypothesis that the high-level nature of these features will reduce the need for a low-level matching of the input data.