Isadore Washington was born in Tallulah, Louisiana, the son of Isadore Washington, Sr., and Destelle Washington. His father came out to Las Vegas, Nevada in the 1930s to work on Hoover Dam, and the family followed in 1942 when Isadore was eight years old. His mother worked at the El Rancho and Last Frontier Hotels, the Shaw Apartments and Harrison boarding house on the Westside where black entertainers were expected to stay. Washington recalls playing with Wayne Newton when they were children and life on the Westside where he attended the Westside School.