This paper investigates the performance of a high temperature Polymer Electrolyte Membrane (PEM) electrolyzer integrated with concentrating solar power (CSP) plant and thermal energy storage (TES) to produce hydrogen and electricity, concurrently. A finite time -thermodynamic analysis is conducted to evaluate the performance of a PEM system integrated with a Rankine cycle based on the concept of exergy. The effects of solar intensity, electrolyzer current density and working temperature on the performance of the overall system are identified. A TES subsystem is utilized to facilitate continuous generation of hydrogen and electricity. The hydrogen and electricity generation efficiency and the exergy efficiency of the integrated system are 20.1% and 41.25%, respectively. When TES system supplies the required energy, the overall energy and exergy efficiencies decrease to 23.1% and 45%, respectively. The integration of PEM electrolyzer enhances the exergy efficiency of the Rankine cycle, considerably. However, it causes almost 5% exergy destruction in the integrated system due to conversion of electrical energy to hydrogen energy. Also, it is concluded that increase of working pressure and membrane thickness leads to higher cell voltage and lower electrolyzer efficiency. The results indicate that the integrated system is a promising technology to enhance the performance of concentrating solar power plants.