We study the electronic and magnetic properties of the quaternary AuMnSn1-xSbx Heusler alloys using first principles calculations. We determine their magnetic phase diagram and show that they present a phase transition from a ferromagnetic (FM) to an antiferromagnetic (AFM) state with increasing Sb concentration. For large Sb concentrations the AFM superexchange coupling dominates over the FM RKKY-like exchange mechanism. This behaviour is similar to that demonstrated by the isovalent Ni1-xCuxMnSb alloy studied recently by the authors (Galanakis et al 2008 Phys. Rev. B 77 214417). Thus the variation of the concentration of the sp-electrons (Sn and Sb atoms) in the AuMnSn1-xSbx compound and the variation of the concentration of the non-magnetic 3d atoms (Cu) in Ni1-xCuxMnSb lead to a similar tuning of the magnetic properties of the two Heusler alloys. We show that the inclusion of correlation effects does not alter the phase diagram of the AuMnSn1-xSbx compound. The calculated results are in good agreement with the available experimental data.