Albion have now won nine and drawn one of their last Division Two games after failing to win any of their first 11 matches.
The Blue Toon went into this match on a run of four successive league victories, but they started slowly and paid the price after just seven minutes. Chris Aitken drove the ball into the area following a short corner and Andy Graham teed it up for Dougie Wilson, who blasted a low shot past Paul Mathers.
Stirling failed to build on their early dominance and the Blue Toon should have equalised on 31 minutes when Keith Gibson headed a Martin Wood cross over the top with keeper Myles Hogarth stranded.
Albion substitute Steven Bell headed an Aitken cross on to the top of the bar soon after half-time, and the home side doubled their advantage just before the hour mark. Bell was fouled 25 yards from goal and Stewart Devine whipped the free-kick just inside the left-hand post.
That should have been it, but Peterhead dug in and Myles Hogarth made a marvellous save to deny Martin Wood at the back post before Jamie Buchan seized on some shoddy defending to rifle the ball home in the 81st minute.
Cue a frantic finish in which Albion hung on and might even have scored a third goal, but Chris Hay was thwarted by Mathers after the Peterhead keeper mis-hit a clearance straight to him.