Pine River Capital Management, a $15 billion hedge-fund firm that made big profits off the rebound in government-backed mortgage bonds after the financial crisis, has returned external money to investors from a fund dedicated to such securities, according to a person familiar with the decision.

The firm concluded the opportunities for further large-scale profits in such bonds were limited, the person said.