Aiming at the problems such as complex tea garden background, different scale of tea diseases, minimal disease spots and easy to miss and misdetect, an improved Faster RCNN model was proposed to identify tea leaf diseases.By comparing the feature extraction network VGG-16, mobilenetv2 and ResNet50 with optimized region suggestion frame, ResNet50 is selected as the backbone network with good effect;FPN network is integrated to improve the problem of missing detection of small targets and multi-scale problem of disease spots;Rank & Sort (RS) Loss function is used to replace the loss function in the original Faster RCNN to alleviate the impact of uneven sample distribution on model performance.The results show: The average precision mAP of the model proposed in this study was 88.06%, the detection speed was 19.1 frames /s, and the average accuracy of the identification of algal spot, white star disease, anthrax and soot disease was 75.54%, 86.84%, 90.42% and 99.45%, respectively. Compared with the original Faster RCNN model, the improvements were 40.98%, 44.16%, 13.9% and 2.43%, respectively.The results showed that this study could well detect and identify tea leaf diseases under the complex background of tea gardens, meet the requirements of tea leaf disease detection, provide reference for tea disease detection under natural environment, and have important research significance for tea disease prevention.