Caretaker boss James Shan admitted his disappointment as Albion folded at The Den in his first loss in charge of the Baggies.
Albion never looked at the races and it was no surprise when the Baggies fell behind as Sam Johnstone failed to stop Ryan Tunnicliffe's strike.
Jay Rodriguez then missed a glorious chance from the spot to pull Albion level but he pulled his shot wide.
Minutes later Albion were two behind when Ahmed Hegazi's attempted block found its way past Johnstone. The big Egytian then saw his marching orders for a second bookable offence.
Shan said: "We knew it was going to be difficult at Millwall and we are all really disappointed with the result.
“The important thing now is that we recover right before Tuesday’s game at Bristol City.
“There are things we need to improve on and deal better with.
“They took the game to us early on and built up momentum.
“The place erupts when they score and I was disappointed with the way we conceded the first goal and the timing of it.
“The second-half panned out very similar to the first-half.
“They came out quick again but we managed to gain a bit of control and we won the penalty.
“The penalty is one big momentum swing. Jay’s been fantastic from the spot this season but unfortunately it wasn’t to be today.
“If we score that the momentum swings in our favour but they go and add a second and it’s back in their favour.”