The visitors led for much of the match but were eventually undone when on-loan Kilmarnock striker Paul Di Giacomo pounced with just seven minutes left.
Albion had conceded ten goals in their previous three games and were behind inside three minutes – thanks to a spectacular goal from an unlikely source.
Myles Hogarth conceded a corner with a tremendous save from Chris Templeman's header and, when the ball came over, giant defender Stewart Greacen found space to net with an overhead kick.
Stirling refused to buckle and Di Giacomo was denied an equaliser by keeper David McGurn after a superb run into the area before the Cappielow custodian thwarted David O'Brien.
Robert Dunn, back at Stirling after moving from Dumbarton, also saw a shot saved then he was stunned on the hour mark when McGurn somehow kept out his close-range effort from a Dougie Wilson cross.
Just when it looked like Morton might hold on for a vital win, substitute Martin Glancy turned the ball into the path of Di Giacomo, who swept it home from six yards to ensure a repeat of the result when the teams clashed at Forthbank in September.
Stirling will now travel with confidence to Stranraer on Saturday while Morton know they cannot afford to slip up when they entertain league leaders Brechin.