Police were called to the glass skyscraper at 8.02 GMT, where a 39-year-old man was pronounced dead at the scene after hitting a lower 9th-floor roof. Witnesses said the body remained on the roof for several hours.
London police said no arrests had been made and the incident was being treated as non-suspicious at this early stage.
A source familiar with the matter confirmed the deceased was Gabriel Magee, a vice president with the JP Morgan's corporate and investment bank technology arm, who had been an employee since 2004.
"We are deeply saddened to have lost a member of the JP Morgan family at 25 Bank Street today," JP Morgan said in a statement. "Our thoughts and sympathy are with his family and his friends."
Workers in Canary Wharf, whose Manhattan-style skyscrapers form part of one of the world's major financial centres, took to Twitter to express their shock at the death.
"The 9th floor roof of JP Morgan is visible from my office window," tweeted Hetal V Patel. "For a long time the body was left cordoned and unattended. Weird. #Wharf."
The JP Morgan building has been the headquarters of the bank's Europe, Middle East and Africa operation since July 2012. It was previously occupied by Lehman Brothers, whose staff left with their belongings in cardboard boxes after the investment bank filed for bankruptcy on Sept. 15, 2008.
Home to Barclays, Citi, Credit Suisse, HSBC, JP Morgan, Morgan Stanley, State Street and Thomson Reuters, Canary Wharf, lies to the east of the City of London.
Though the details of Tuesday's incident are still unclear, occasional suicides by people working in London's big banks have provoked criticism of the demands placed on some financial services workers.
A Bank of America exchange manager jumped in front of a train and another man jumped from a seventh-floor restaurant, both in 2012. A German-born intern at Bank of America died of epilepsy last year in London.
On Tuesday, when asked about the death of William Broeksmit, a former senior manager at Deutsche Bank, London police said a 58-year-old man had been found hanging at a house in South Kensington on Sunday afternoon.