This paper analyzes measured triaxial vibrations of road and rail traffic on and around a typical residential masonry building in Istanbul and its response to adjacent ground-born vibrations through numerical modeling. Therefore, to determine the dynamic behavior and resultant internal forces within the building, which stands between a road and a railway line, a numerical model was created, updated, and validated through the measured vibrations. The results show that train-induced vibrations caused the walls of the building to experience tensile stresses up to 23% of the masonry tensile strength. Additionally, the measured maximum vibration level on the building is well above the limit for complaints in residential environments and very close to the damage limit for historic and sensitive structures. Furthermore, a nearby vibration amplification zone on the ground surface suggests potential amplified loadings. Therefore, human responses to these prolonged vibrations should be the controlling factor, entailing site-specific analyses by the codes of practice in cities with dense traffic.