Ipswich had the better of the first half and deservedly went ahead on 43 minutes when Tommy Miller swept home a low left foot shot after collecting a Matt Richards pass in his stride.
Midway through the first period both sides had gone close to opening the scoring.
First, Horsfield's 35-yard shot flashed inches past the post and then Russell Hoult dived full length to tip away a low Jim Magilton drive.
Georges Santos sent a diving header wide, and Artim Sakiri would have levelled Miller's striker, but for his shot striking team mate Andy Johnson.
Kelvin Davis made a good save early in the second half to deny Johnson, before Horsfield was correctly flagged offside as he touched home a Lee Hughes cross.
Davis made another good save with his legs to deny Horsfield, before substitute Jason Koumas levelled on 71 minutes, curling a free-kick home from 28 yards, harshly given after Matt Elliott had been adjudged to have fouled Horsfield.
Two minutes later West Brom were ahead when another substitute, Lloyd Dyer, crashed a shot past Davis after good work by Horsfield.
Ipswich hit back when Martijn Reuser's cross saw Shefki Kuqi challenge Joe Murphy, a replacement for the injured Hoult. The ball broke loose and Darren Bent touched it home.
Dyer saw a cross-cum-shot drift just over and a header well saved by Davis as the visitors pressed for the winner.
They got it in the final moments when a quick break saw Andy Johnson find Lee Hughes and his low cross was touched in by the impressive Horsfield.
Ipswich: Davis, Wilnis, Elliott, Santos, Richards, Wright, Magilton, Miller, Westlake, Bent, Bowditch. Subs: Price, Kuqi, Mitchell, Diallo, Reuser.
West Brom: Hoult, Gaardsoe, Moore, Gregan, Chambers, Sakiri, Kinsella, Johnson, Robinson, Hughes, Horsfield. Subs: Murphy, Koumas, Dyer, Clement, Facey.
Referee: G Laws (Tyne & Wear)