The BBC has a pretty powerful description of the segregation in place at the White House up through the 1950's.  It's the story of the first black presidential aide, E Frederic Morrow, appointed to the Eisenhower administration.  Here are some highlights, see the link at bottom for the full article.
I know that Obama often plays down the historic nature of his candidacy a little, but the fact that he even has a chance at winning is amazing given this history:

Up until [Morrow's] appointment in the mid-50s, black White House employees appeared either with pristine white towels draped over their arms or cleaning mops in their hands.  Morrow, a successful PR man, had arrived ostensibly to help shape policy, not that his boss, President Dwight D Eisenhower, much valued his counsel.

The former general wanted to attract black support in key northern battleground states - after all, the Republicans were the party of Abraham Lincoln - and Morrow was recruited for mainly ornamental purposes.  [...] White secretaries refused to work with him; and he was prohibited from being alone in the same room with any female employee, lest he sexually molest them. [...]

Before the civil rights era, Washington DC was notoriously unwelcoming to its black residents and visitors.  For black passengers travelling from north to south, Union Station in Washington was the point of transfer at which they moved from an integrated carriage to one that was strictly segregated.
African diplomats looked on the city as a hardship posting, because of the difficulty in finding landlords who would allow them to rent suitable apartments.

Even black congressmen, Mr Obama's forerunners, suffered many of the same indignities.  Under an unwritten code of conduct, black lawmakers - there were just two in 1945 compared with 41 today - were barred from using Congress's barber shops and swimming pool.
Black reporters were barred even from the press gallery. No wonder Capitol Hill was often called the "Old Southern Home", and widely viewed as the only place in the US where the south had not lost the civil war.