The optimisation of gearbox geometric design parameters to reduce gear-rattle noise in an automotive transmission based on a transmission error model is presented. Towards this aim, a four-degree-of-freedom torsional vibration model for the geared system is obtained. Differential equations of the pinion gear-wheel gear system are obtained. The state-space forms of the differential equations are obtained. The transmission error of the gear system is calculated via a state-space model. An empirical model is used for rattle noise calculation for the five speed gearbox. The transmission error is considered as the objective function, and bending stress, contact stress and the constant distance between gear centres are considered as constraint functions. By optimising the geometric parameters of the gearbox, such as the module, number of teeth, axial clearance, and backlash, it is possible to obtain a gear structure with high bending and contact strength and to minimise the torsional vibration, transmission error and gear-rattling noise. It is concluded through optimisation that minimising the transmission errors of the gearbox leads to reduced vibration and noise levels of the gearbox. It is determined that the optimised geometric design parameters reduce the calculated rattle noise level by 10% [dB] compared with the sample five-speed gearbox. Furthermore, a 95% reduction in transmission error results in a 12% decrease in rattle noise. All optimised geometric design parameters are significant for the required constraints.