As a public transportation mode, rail transit systems are one of the most preferred modes to avoid traffic congestion, especially during the rush hours. This paper proposes a service quality evaluation outline to measure rail transit lines' performances via passenger satisfaction surveys. The proposed method combines statistical analysis, fuzzy trapezoidal numbers and TOPSIS to evaluate service quality levels for multi periods. In total 17,769 surveys that are conducted in Istanbul in 2012, 2013, and 2014 are considered to determine the factors need to be improved. We provide recommendations to enhance the operation for specific lines and guidelines for future investments.