CHESTER, Vt. — While most eyes warily watched the shoreline during Hurricane Irene’s grinding ride up the East Coast, it was inland — sometimes hundreds of miles inland — where the most serious damage actually occurred. And the major culprit was not wind, but water.

As blue skies and temperate breezes returned on Monday, a clearer picture of the storm’s devastation emerged, with the gravest consequences stemming from river flooding in Vermont and upstate New York.

Here in southern Vermont, normally picturesque towns and villages were digging out from thick mud and piles of debris that Sunday’s floodwaters left behind. With roughly 250 roads and several bridges closed off, many residents remained stranded in their neighborhoods; others could not get to grocery stores, hospitals or work. It was unclear how many people had been displaced, though the Red Cross said more than 300 had stayed in its shelters on Sunday, and it expected the number to grow.

In upstate New York, houses were swept from their foundations, and a woman drowned on Sunday when an overflowing creek submerged the cottage where she was vacationing. Flash floods continued to be a concern into Monday afternoon. In the Catskills, where Gov. Andrew M. Cuomo led a helicopter tour of suffering towns, cars were submerged, crops ruined and roads washed out. In tiny, hard-hit Prattsville, what looked like a jumble of homes lay across a roadway, as if they had been tossed like Lego pieces.

“We were very lucky in the city, not quite as lucky on Long Island, but we were lucky on Long Island,” Mr. Cuomo said. “But Catskills, mid-Hudson, this is a different story and we paid a terrible price here, and many of these communities are communities that could least afford to pay this kind of price. So the state has its hands full.”

In Vermont, officials recovered the body of a man who was tending the municipal water system in Rutland during the storm. They said his son, who was with him at the time, was also feared dead. A 21-year-old woman died after being swept into the Deerfield River in Wilmington, a small town west of Brattleboro. And a man was found dead in Ludlow. As of Monday afternoon, the storm had caused at least 40 deaths in 11 states, according to The Associated Press.

“This is a really tough battle for us,” Gov. Peter Shumlin of Vermont said after surveying the damage across the state in a helicopter. “What you see is farms destroyed, crops destroyed, businesses underwater, houses eroded or swept away and widespread devastation.”

In the Catskills, state and local officials had, by Monday afternoon, carried out 191 rescues since the storm began, often plucking people from cars or homes as water rose. State officials confirmed six people had died in connection with the storm: five drowned and one was electrocuted.

Gov. Chris Christie of New Jersey said his state was facing some of the worst inland flooding it had seen in years. Many small streams are now at flood level and some larger rivers — including the Ramapo, Passaic and Delaware — were peaking on Monday or expected to peak over the next 24 hours, reaching record or near-record levels. Almost 200 New Jersey roads were either partially or fully closed. About 110 people were forced to leave their homes Monday morning in Vineland and 60 people in Millville because of possible breach of two dams in the area.

In Connecticut, officials were grappling with damage from the storm surge on Long Island Sound, which punished shore communities like East Haven and Milford, as well as rising rivers and streams inland. Colleen Flanagan, a spokesman for the governor’s office, said some waterways, like the Connecticut River, were not expected to crest until Wednesday.

Hundreds of miles to the south, in North Carolina, where Hurricane Irene first made landfall, state-operated ferries began on Monday to move personnel and supplies to Hatteras Island on the Outer Banks, where an estimated 2,500 residents remained cut off from the mainland by damage to the main highway.