When I was clerking for Justice Harry Blackmun, I remember the Justice taking scrupulous care to recuse himself from cases where it might be thought that he had a financial interest in the outcome, as, for example, when he held shares of stock in a company that was a party to a case. I admired Blackmun for his zeal on the subject. He thought even the slightest appearance of impropriety should be avoided.
But I also remember thinking that, as a practical matter, any concern regarding conflicts of interest was, for the most part, overblown. It seemed absurd to me that anyone could believe that, in the absence of really significant stock holdings, Justice Blackmun or any of the Justices would be swayed, even subconsciously, by the indirect prospect of some at best incremental financial gain or loss depending on how the Court decided a case. If Supreme Court Justices were truly this susceptible to corrupt thought, we were facing bigger problems than just recusal from cases!
On the other side of the coin, moreover, the price of recusal at the Supreme Court, in particular, is potentially very high. At the trial court or court of appeals level, one judge can be replaced by another, usually with a minimum of disruption. At the Supreme Court, by contrast, there is no one to sub in when one of the Justices recuses himself or herself.