They recovered with a first half free-kick from Santi Cazorla, a second half equaliser from Laurent Koscileny and a brilliantly taken extra-time winner from Aaron Ramsey who scored with the outside of his right foot.
After the final whistle jubilant manager Arsene Wenger, in charge of Arsenal since 1996, confirmed he was staying at the club to end speculation over his future.
Wenger, showing rare emotion after he was doused with water by his celebrating players, replied "yeah, yeah, yeah" when asked if he was signing a new contract.
Not only was this Arsenal's first major honour since their FA Cup win in 2005, they equalled Manchester United's record of 11 FA Cup wins.
Wenger equalled former United manager Alex Ferguson's record of five FA Cup wins by a manager and Arsenal became the first team since Everton beat Sheffield Wednesday 3-2 in 1966 to win the FA Cup after coming from two goals down.
Frenchman Wenger, beaming with satisfaction, said; "We wanted to make history and win the game. It was the turning point in the life span of this team.
"If we hadn't have come back it would have been terrible. I praised the spirit many times of this team. Congratulations to Hull they played very well.
"I always said my future doesn't depend on this match but I wanted to wait to see how it all goes."
He might well have been saying "Non ! Non ! Non!" after the opening eight minutes when Hull, playing in the FA Cup final for the first time in their 110-year history and whose trophy drought has gone on ever since, blasted their way into a 2-0 lead with goals from two of their three centre-backs, James Chester and skipper Curtis Davies.
Chester scored when Arsenal failed to cut our a mis-hit pass from Tom Huddlestone with Chester planting the ball wide of Arsenal keeper Lukasz Fabianski to give his team a shock lead.
Worse was to follow for Arsenal four minutes later when they failed to clear from a set-piece and Davies scored from a narrow angle after Fabianski had palmed the ball on to a post.
In some ways, although Hull were delighted to be ahead, the goals came too early in the match for them.
Arsenal, better in every position, ended the league season in great form with five straight wins to secure fourth place while Hull staggered over the finish line and ended 16th.
Arsenal's superiority, with Cazorla the main midfield creator and Ramsey showing glimpses of the form he had at the start of the season before injury sidelined him for three months, complemented him superbly and helped Arsenal dominate possession for long periods.
Despite their early advantage, Hull could not build on it, but matters might have turned out very differenly if they had gone three up after 12 minutes when a dipping header from Alex Bruce was cleared from under the Arsenal crossbar by Kieran Gibbs.
Gibbs was guilty of a staggering miss after 78 minutes which could have haunted him if Arsenal had lost when, with just a touch needed to score what would probably have been the winner, he failed to find the target.
Despite going behind Arsenal showed great resolve in their fightback and got a lifeline when Cazorla scored with a brilliantly struck freekick from 25-metres out after 17 minutes.
His strike evoked memories of Paul Gascoigne's famous free-kick for Tottenham Hotspur against Arsenal in the 1991 FA Cup semi-final at Wembley as it soared into the upper left-hand corner of the goal.
Arsenal continued to press and attack from both left and right wings but with striker Olivier Giroud having minimal impact on the Hull defence, they had to wait until Koscielny benefited from two lucky deflections to turn and equalise from close range after 71 minutes.
Giroud had a great chance to settle it early in extra time when he thundered a header against McGregor's bar before Arsenal's sublime winner set up by Giroud's back-heel.
"It's such a great feeling right now, I'm quite emotional thinking about it," matchwinner Ramsey said.
"We have conceded early a few times this season but we showed great strength and character today.
"That's up there with the most important goals I've scored and hopefully this is the first of many to come."