Monitoring the working conditions of sucker rod pumping wells in a timely and accurate manner is important for oil production. With the development of smart oil fields, more and more sensors are installed on the well, and the monitored data are continuously transmitted to the data center to form big data. In this work, we aim to utilize the big data collected during oil well production and a deep learning technique to build a new generation of intelligent diagnosis model to monitor working condition of sucker rod pumping wells. More than 5×106 of well monitoring records, which covers information from about 1 year for more than 300 wells in an oilfield block, are collected and preprocessed. To show the dynamic changes of the working conditions for the wells, the overlay dynamometer card is proposed and plotted for each data record. The working conditions are divided into 30 types, and the corresponding data set is created. An intelligent diagnosis model using the convolutional neural network (CNN), one of the deep learning frameworks, is proposed. By the convolution and pooling operation, the CNN can extract features of an image implicitly without human effort and prior knowledge. That makes a CNN very suitable for the recognition of the overlay dynamometer cards. The architecture for a working condition diagnosis CNN model is designed. The CNN model consists of 14 layers with six convolutional layers, three pooling layers, and three fully connected layers. The total number of neurons is more than 1.7×106. The overlay dynamometer card data set is used to train and validate the CNN model. The accuracy and efficiency of the model are evaluated. Both the training and validation accuracies of the CNN model are greater than 99% after 10 training epochs. The average training elapsed time for an epoch is 8909.5 seconds, and the average time to diagnosis a sample is 1.3 milliseconds. Based on the trained CNN model, a working condition monitoring software for a sucker rod pumping well is developed. The software runs 7 × 24 hours to diagnosis the working conditions of wells and post a warning to users. It also has a feedback learning workflow to update the CNN model regularly to improve its performance. The on-site run shows that the actual accuracy of the CNN model is greater than 90%.