Mikel Arteta made seven changes to his side and was without captain Pierre-Emerick Aubameyang for personal reasons as the Gunners were outplayed, losing their grip on the trophy following a 1-0 defeat at St Mary’s.
Southampton should have won by a bigger margin but now face a trip to Wolves in round five as Gabriel’s poor return to the Arsenal side was capped by accidentally deciding the tie in favour of the hosts.
Saints started lively and were unsettling the recalled Gabriel in the opening minute.
Danny Ings was then picked out free inside the box and his effort was deflected behind, with James Ward-Prowse hitting the crossbar direct from a corner as he looked to catch Bernd Leno off his line.
Arsenal at least started to get a foothold in the game as Mohamed Elneny’s fine pass ultimately led to nothing, with former Saints defender Cedric shooting over soon after.
But the visitors still looked frail at the back as Che Adams burst past Gabriel as if he was not there before forcing a smart stop out of Leno.
Gabriel Martinelli then passed up a fine opportunity to put the 14-time FA Cup winners in front, miscuing his shot after breaking the offside trap from a free-kick.
Saints took the lead just before the midway point of the first half as Elneny gave away possession on the edge of his own box.
The ball was worked wide to Kyle Walker-Peters, whose cross-cum-shot was deflected into his own goal by the unfortunate Gabriel, the first time Arsenal have conceded in 508 minutes.
The second half started in similar fashion, Arsenal shabby in possession as Southampton pressed the wobbly backline.
Arteta reacted by turning to his bench, introducing Thomas Partey and Bukayo Saka just before the hour.
But it was defender Rob Holding who spurned a great chance to equalise, sending an effort over the crossbar after the ball bobbled up for him in the Southampton box.
Former Arsenal player Theo Walcott then ballooned over a decent chance at the other end before Fraser Forster did well to prevent Eddie Nketiah from levelling.
Arteta’s last throw of the dice was to bring on Alexandre Lacazette as he sacrificed Hector Bellerin for a more attacking set-up.
The shift meant Arsenal enjoyed more possession in and around the Southampton box without ever really looking like finding the goal to take the tie into extra-time.