The crash happened at 7:20 a.m. (1220 GMT) about 100 yards (metres) north of Metro-North's Spuyten Duyvil station in the city's Bronx borough, said Metro-North spokesman Aaron Donovan.
Metropolitan Transit Authority police said two men and two women were killed and the MTA said 63 people were injured. A Fire Department spokesman said 11 people had been sent to the hospital in critical condition and six in serious condition with non-life threatening injuries.
The train, headed south toward Manhattan's Grand Central Terminal, was about half full at the time of the crash with about 150 passengers and was not scheduled to stop at the Spuyten Duyvil station, said the state's Metropolitan Transportation Authority (MTA), parent company of Metro-North.
"On a work day, fully occupied, it would have been a tremendous disaster," New York City Fire Commissioner Salvatore Joseph Cassano told reporters at the scene.
The derailment happened in a wooded area where the Hudson and Harlem rivers meet. At least one rail car was lying toppled near the water and others were lying on their sides.
There was no official word on possible causes of the accident.
"That is a dangerous area on the track just by design," Governor Andrew Cuomo told CNN after touring the site. "The trains are going about 70 miles per hour (112 kph) coming down the straight part of the track. They slow to about 30 miles per hour (48 kph) to make that sharp curve ... where the Hudson River meets the Harlem River and that is a difficult area of the track."
Cuomo said it appeared that all passengers had been accounted for.
He said recovery of the train's "black box" - a data-recording device similar to those on airplanes - would reveal more about the train's speed, possible mechanical issues and whether brakes were applied.
The National Transportation Safety Board said it would be on the scene investigating the accident for at least the next week and would focus on track conditions, signaling systems, mechanical equipment and the performance of the train crew.
Passenger Frank Tatulli told television station WABC he had been riding in the first car and the train had been traveling "a lot faster" than usual.
"The guy was going real fast on the turns and I just didn't know why because we were making good time. And all of a sudden we derailed on the turn," he said.
Joseph Bruno, who heads the city's Office of Emergency Management, told CNN it appeared that three of the four people killed had been ejected from the train. The MTA and the fire department both said that could not immediately be confirmed.
Michael Keaveney, 22, a security worker whose home overlooks the site, said he had heard a loud bang when the train derailed.
"It woke me up from my sleep," he said. "It looked like (the train) took out a lot of trees on its way over toward the water."
SERIES OF ACCIDENTS
New York police divers were seen in the water near the accident, and dozens of firefighters were helping pull people from the wreckage. None of the passengers were in the water, said Marjorie Anders of Metro-North.
The derailment was the latest in a string of problems this year for Metro-North, the second busiest U.S. commuter railroad in terms of monthly ridership. The MTA said details about how the accident would impact Monday morning's commute were not yet available.
In July, 10 cars of a CSX freight train carrying trash derailed in the same area, Anders said. Partial service was restored four days later but full service did not return for more than a week.
In May, a Metro-North passenger train struck a commuter train between Fairfield and Bridgeport, Connecticut, injuring more than 70 people and halting service on the line.
The MTA said Sunday's accident marked the first customer fatality in Metro-North's three-decade history and that it was a "black day" for the railroad.
Amtrak said its Empire Line service between New York City and Albany was being restored after being halted immediately after the crash. Amtrak's Northeast Corridor service between Boston and Washington was not affected.
Metro-North's Hudson Line service has been suspended between Tarrytown and Grand Central station, and bus service is being provided between White Plains and Tarrytown, the MTA said.
New York-Presbyterian Hospital said it was treating 17 patients from the accident, including four in critical condition. Jacobi Medical Center, which received 13 patients from the accident, said none have critical injuries and several had already been discharged.
President Barack Obama was briefed on the accident and a White House official said the president's thoughts and prayers were with the friends and families of those involved.