Harvey Allen grew up poor in Philadelphia. After taking singing lessons at the Artie Singer Vocal School, he got jobs singing in night clubs up and down the east coast. He also took acting classes in New York and performed at the Copa Club. In the 1950s, he moved to Las Vegas, Nevada and auditioned for Jack Entratter at the Sands Hotel and Casino where he performed with numerous entertainers including Lena Home, Louis Armstrong, Tallulah Bankhead and Robert Merrill.