Determining soybean maturity and the proper soybean maturity zone a soybean belongs to has been a laborious task for plant breeding programs. The conventional method to estimate the maturity dates of breeding lines uses subjective visual ratings based on when 95% of pods have turned brown. In this work, we developed and validated a pipeline for high throughput phenotyping (HTP) of soybean maturity using RGB camera collected time-course images collected from a low-cost unmanned aerial vehicle (UAV). Maturity dates collected from five contrasting environments with 51 experimental trials (4422 total soybean breeding plots) were measured using visual rating scores from the beginning maturity stage (R7) to full maturity stage (R8), and during the same time but with different numbers of flights for each environment, aerial RGB images were also taken to explore the UAV performance. Three different index values from RGB bands and five extraction methods from each plot were investigated using three different methods: local regression, segmented regression, and Random Forest (a supervised machine learning model). We also illustrated a common breeding program’s challenges using the ground-truth data to investigate the UAV performance during practical applications of our proposed methods. Using the mean greenness leaf index from the pixel values of each plot and local regression, we achieved very high correlation across all trials and environments. Additionally, UAV heritability measures were greater in 33 of 51 experimental trials. Our results show that RGB data collected once a week from a UAV-based HTP platform will improve the efficiency of maturity recording in a modern soybean breeding program.