By RACHEL L. SWARNS

 
Most of the time, the prominent men hovered in different orbits and different cities. Yet for years now, their lives have converged here on this resort island of white beaches and rippling sea.

There was William Hairston, a local builder whose wife is active in Republican circles here. There was Michael R. Hollis, an Atlanta lawyer, entrepreneur and presidential history buff who vacations here.

And there was President Bush's housing secretary, Alphonso R. Jackson, who golfed and socialized here and led the federal agency that gave hundreds of thousands of dollars in business to friends and acquaintances, including Mr. Hairston and Mr. Hollis.

One such friend, an Atlanta developer, received a $127 million contract last year as part of a joint venture to rebuild a New Orleans public housing project. That developer's company has paid Mr. Jackson more than $250,000 in fees since Mr. Jackson joined the Bush administration in 2001, for work done before Mr. Jackson joined government, the developer's lawyer said.

Mr. Jackson, who announced his resignation in March, leaves office on Friday as federal authorities continue to investigate whether he enriched himself and friends with lucrative contracts. The inquiry has also laid bare the connections between Mr. Jackson, who was determined to expand opportunities for minority contractors, and the ambitious men who benefited from those opportunities.

It is the story of a small circle of black businessmen linked by their financial interests in the revitalization of troubled public housing and, in most cases, a shared affinity for conservative politics, and how those connections may have helped force the housing secretary from public life.

In 2003, the year before Mr. Jackson was named secretary, 14 percent -- or $134 million -- of the Department of Housing and Urban Development's contracts went to black-owned firms, officials say. By 2007, black-owned businesses were receiving 25 percent of the department's contracts, or $195.6 million.

Mr. Jackson has proudly promoted such statistics, saying that ''a good bottom line with small and minority businesses helps to build a stronger America.''

Indeed, some of Mr. Jackson's supporters deride the scrutiny of his casual friendships as a racist effort to undermine a prominent black official and several respected black businessmen, noting that no one has been charged with a crime.

Representative James E. Clyburn of South Carolina, the No. 3 Democrat in the House, said he believed the investigation was fueled by officials determined to derail Mr. Jackson's efforts to expand affirmative action.

''Is there something wrong with trying to make sure African-Americans participate in the contracting program with the American government?'' asked Mr. Clyburn, who vacations here regularly and knows Mr. Jackson, Mr. Hairston and Mr. Hollis.

But over time, concerns have grown -- first among some housing officials and later among federal investigators -- as it became clear several men who interacted with and had business deals with Mr. Jackson became beneficiaries of his efforts to further integrate the contracting corps.

Mr. Hairston, who golfed with Mr. Jackson here, received at least $610,000 in contracts from the New Orleans housing authority, which HUD took over in 2002, for reconstruction work on public housing complexes that were battered by Hurricane Katrina, officials say. (Mr. Hairston did not shy from talking up his personal ties to Mr. Jackson, according to housing officials who worked with him. And Mr. Jackson rebuked department employees who challenged Mr. Hairston's contracts and authority, the officials said.)

Mr. Hollis, an acquaintance of the housing secretary, owns a law firm that was paid at least $1 million by HUD for running the Virgin Islands housing authority, government contracting records show. (Maynard H. Jackson Jr., the former mayor of Atlanta who died in 2003, introduced Mr. Hollis to Alphonso Jackson more than a decade ago.)

The Atlanta developer, Noel Khalil, who occasionally dined with Mr. Jackson in Atlanta and in Washington, runs Columbia Residential, a development company that received the $127 million contract from the New Orleans housing authority last year as part of a joint venture hired to redevelop the St. Bernard housing project.

Mr. Khalil, who does not vacation on Hilton Head, hired Mr. Jackson as a partner in 1998 for development deals in Texas, before Mr. Jackson joined HUD as deputy secretary in 2001. (The two men met in 1994, also via an introduction from former Mayor Jackson, when Mr. Jackson was running the housing authority in Dallas.)

From 2001 to 2007, Columbia Residential paid Mr. Jackson over $250,000 in developer fees on three housing complexes for work that he completed before he entered government, said Mr. Khalil's lawyer, Buddy Parker.

Mr. Jackson listed only one payment -- of $35,000 -- from Columbia Residential in the financial disclosure forms he filed for 2001 to 2006. Investigators have been looking into whether Mr. Jackson steered contracts to Mr. Khalil to ensure that Mr. Khalil could make those payments.

Mr. Jackson declined to comment on his ties to the three men, citing the ongoing investigation. Mr. Hairston did not respond to repeated requests for comment.

Pressuring subordinates to award contracts to specific firms could be a crime, according to government officials briefed on the inquiry. The officials said investigators were also trying to determine if Mr. Jackson received payments in exchange for any help he gave friends. But the officials, who spoke on condition of anonymity, said investigators had not found evidence of such an arrangement.

Mr. Hollis and Mr. Khalil denied that they asked for or were offered any special treatment because they were friendly with Mr. Jackson. They say they believe federal investigators know that their contracts were legitimate.

Mr. Parker said he had turned over documents and spoken with investigators. He said a Justice Department official told him that his client was considered a witness in the investigation.

''When you pay money that's clearly traceable to a sitting secretary of the cabinet,'' Mr. Parker said, ''it's not a shocking idea that you're going to be investigated.''

''But the fact is that he has nothing to hide,'' he said. ''I feel comfortable in saying that they've checked our facts out.''

Mr. Hollis said federal investigators had not contacted him about his contracts to manage the Virgin Islands housing authority, which extended from February 2006 to May 2007. The authorities have subpoenaed records from Smith Real Estate Services, an Atlanta firm that retained him as a special adviser for a Virgin Islands contract with the department in 2004.

Pamela Smith, president of Smith Real Estate Services, declined to comment. Her lawyer, Ralph Caccia, said she cooperated fully with the authorities.

But Mr. Hollis said he had improved the troubled Virgin Islands housing authority, imposing financial accountability, rehabbing 300 public housing units, negotiating for efficient and cost-effective water service and removing hundreds of abandoned cars from the properties, among other steps.

Carmen Valenti, a HUD official who oversaw Mr. Hollis's work, called him ''dedicated, very conscientious and really hard-working.'' Mr. Valenti said Mr. Hollis's contract required several approvals and was extended several times by HUD officials.

''I'm very proud of what we did,'' Mr. Hollis said. ''We pulled together a team that improved the housing conditions for nearly 15 percent of the people who live in the Virgin Islands, as well as the working conditions of nearly 300 public housing employees.''

''When our engagement started, V.I.H.A. was a highly troubled agency,'' he said, referring to the Virgin Islands agency. ''When our engagement was over, V.I.H.A. was a much stronger agency and poised for economic recovery.''

Senior Democrats in Congress, who urged Mr. Jackson to resign, say the deals smell of cronyism. Mr. Parker said there was nothing nefarious in the fact that several of the businessmen were acquainted with each other.

''There are a substantial number of successful African-Americans who know each other through business and politics,'' Mr. Parker said. ''That's how Noel Khalil knows Michael Hollis and Alphonso Jackson.''

Some African-Americans here bristle at the notion that Mr. Jackson's casual friendships with the black professionals who flock to this resort town have become part of a federal investigation.

''You get an African-American in a position where he can help black folks, and people just don't like it,'' said Larry Holman, president of the Beaufort County Black Chamber of Commerce, who knows Mr. Jackson and Mr. Hairston.

''It's unfortunate,'' Mr. Holman said. ''We have a lot of respect for Secretary Jackson here.''

Property records show that Mr. Jackson bought a house in an exclusive gated community here in 2004. Since then, local residents in this town of 33,000 people have watched his comings and goings with interest.

Mr. Jackson hobnobs with local businessmen, golfs, dines with friends and chats with neighbors who live alongside his vacation home, a cream-colored colonial with columns. He socialized with Mr. Hairston, who had been looking for work beyond South Carolina after his stucco business withered in the face of competition from Hispanic-owned companies here, according to people who know Mr. Hairston.

And he would occasionally bump into Mr. Hollis at parties or gatherings hosted by mutual friends.

Clifford Bush, a local lawyer, said Mr. Jackson made a point of mingling with black businessmen, even stopping by an event organized by the county's black chamber of commerce.

As a prominent black conservative, Mr. Jackson certainly stood out. Mr. Hairston and Mr. Khalil also share an affinity for the Republican Party. Mr. Hairston's wife, Starletta, is running for a seat in the South Carolina House, on the Republican ticket.

Mr. Hollis said he still admired Mr. Jackson, despite the housing secretary's troubles, because he climbed out of poverty to become a lawyer and a member of President Bush's cabinet. ''He pulled himself up by his bootstraps,'' he said.

Mr. Khalil said through his lawyer that he ''regrets the circumstances that Alphonso Jackson finds himself in.''

As for Mr. Jackson, he is planning on ''a few months of rest and relaxation'' after stepping down from office, said a HUD spokesman, Stephen C. O'Halloran.

Mr. O'Halloran said Mr. Jackson planned to continue vacationing here in Hilton Head.