The monster goes to the graveyard of its "owner", which is the player whose deck originally contained it. Your cards can never end up in your opponent's graveyard or remove from play zone, even if your opponent "controlled" the card when it was destroyed.
Player A activates Monster Reborn and revives Sangan from Player B's grave.
Sangan is destroyed later and returns to Player B's grave.
Player B gets to use Sangan's effect.
Another example (common from older decks)
Player A summons Pyramid Turtle and uses Creature Swap to switch it with the opponent's monster.
Player A destroys Pyramid Turtle in battle and it is sent to Player A's graveyard.
Player A gets to activate Pyramid Turtle's effect to summon another monster from his deck.
