Second-half strikes from Jason Koumas and Horsfield, in his fifth appearance for the club following last month's £1million move from Wigan, revived Albion's promotion push.
They had drawn their last five league games at The Hawthorns and had to ride their luck to see off the challenge of Walsall who had been seeking to complete a famous double after hammering Albion 4-1 on the opening day of the season.
Walsall, beaten for only the third time in 11 league games, should have been two up after just eight minutes but were frustrated by their own finishing and Albion goalkeeper Russell Hoult.
Winger Darren Wrack completely missed the ball from just four yards in only the third minute after good work from Paul Merson and Jorge Leitao had created the chance.
Striker Gary Birch did at least manage to hit the target five minutes later after a cross from Merson but this time it was the right hand of Hoult that denied Walsall.
Having survived the double let-off, Albion did their best to take advantage of their good fortune.
Scottish international striker Scott Dobie was inches away from converting a Horsfield cross in the 15th minute.
Horsfield then headed a Koumas cross wide on the half hour before referee Barry Knight found himself at the centre of controversy five minutes before half time.
Horsfield appeared to be fouled in the area by Walsall central defender Ian Roper but Knight immediately dismissed Albion's claims for a penalty and booked Horsfield for diving.
The burly striker's frustration was then increased a minute before the interval when he collected a misplaced pass by Roper and, faced with just James Walker to beat, fired his shot straight at the long-serving Walsall keeper.
Having ridden their luck in the opening stages, Albion then needed another huge slice of luck to make their growing pressure pay.
Horsfield was fouled on the edge of the area by Roper after 62 minutes and from the resulting free-kick Koumas saw his shot deflected into the net via Saddlers midfielder Vinny Samways.
Albion sealed their success ten minutes later when Horsfield was rewarded for an evening of hard work.
He bundled the ball home from close range after Dobie had nodded down a cross from substitute Lloyd Dyer to seal Albion's second win in seven games.