I cast Rescue on a transformed persistent nightmare, and it returned to my hand, but since the flipped side has no casting cost, it couldn’t gain the eight mana from Rescue. It stayed in my hand through the end of the turn, despite having a zero casting cost and an open creature slot on the battlefield.
The following turn, I matched enough Mana to charge the card in front of it, and partially charge the card behind it in my hand. The PN then cast, and hit the board as its transformed form. The following turn, it attacked, and bounced back to my hand naturally, and had full mana when it did.
I haven’t tested this with other transform cards, but as none of them have a natural bounce mechanic, I can’t say that the results would be similar.