It was also Albion's first victory in Greenock since 1997 and they fully deserved all three points after a superb display.
Morton struggled to get going and they should have fallen behind on the half-hour mark when Stirling were awarded a penalty. David O'Brien was upended by keeper David McGurn but he then saved Stewart Devine's weak penalty.
However, the visitors got the goal they deserved in first-half injury time when Paddy Connolly fired home his sixth of the season from close range after a fine passing move.
Morton were rocked again after 54 minutes when Ross Forsyth got the break of the ball and thumped a shot high into the net.
The home side tried to lift the pace and they were handed a lifeline after 67 minutes when Peter Weatherson thundered a 20-yard free-kick into the corner of the net after substitute Jason Walker had been fouled.
Jim McAlistair struck an 18-yard effort inches wide as Morton pressed but that allowed Albion to threaten on the break and O'Brien was denied by McGurn late on as Stirling closed out a vital win.