A "crumb bun" is something you eat. A "crumb-bum" is a worthless fellow. Like the word "bum" itself (see the Brooklyn Dodgers' "bums"), "crumb-bum" was used as a term of endearment.

Toots Shor, who operated a saloon frequented by sports figures, popularized the term.

(Historical Dictionary of American Slang)
Crumb-bum n. a filthy, worthless, or contemptible person.
1934 Appel Brain Guy 138: None of the leaders were there, only a dozen or so of the crumbums and chisellers.
1944 Brooks Brick Foxhole 79: "Now listen to me, you crumb-bums," said the mess sergeant.
1946 H.A. Smith Rhubarb 95: "That crumb bum!" he said.
1946 J. Adams Gags 18: Toots Shor has called me a crum-bum and many other names.

24 December 1944, Washington Post, pg. M7:
There's a joint in New York named Toots Shor's which had tradition the day it opened its doors, for the sports mob showed up the first night the place's beer bung was sprung. And the mob has never broken the habit.

"Those crumb-bums made me what I am today," Toots says affectionately.

24 January 1977, Los Angeles Times, pg. A1:
He always said he ran his watering holes for his pals and didn't care that people he didn't like -- he called them "crumb bums" --found him somewhat abrupt.

25 January 1977, Los Angeles Times, pg. B5:
Toots Shor is gone, and it's a time of mourning for the crumb-bums.

To those who had drunk and debated over the last 40 years with the old saloon bouncer and restaurateur who died late Sunday at 73, "crumb-bum" was a term of endearment.

Commenting is not available in this channel entry.