The point of this card was not necessarily to give something to those who did not have the powerful speed cards, but to give an option to play something other than speed. It is definitely not enough on its own, but it is an important piece of the puzzle. If you do not take actions associated with speed, then you get a reward. Also, it is very easily countered by common cards like Nazareth, so that combined with the inability to use speed strategies really limits the card in terms of power. I don't expect we'll see an issue with it.
On your other points, this card already turns off if exchanges are done. Exchanges include a search of the location they go to, if they are hand, deck, or discard. If
Nazareth is in play, it only protects from search, but that still shuts down exchange. Same with
Hezekiah's Signet Ring. Both shut down exchange to the locations, even though they say search.
If you exchange to those locations, you also search them, and thus are not 'patient' enough to get this reward.
Lastly, I see your point on play, and like that idea, but you do already have that card you are playing, so it isn't 'true speed'. However, I truly doubt any changes will be made at this point, especially if the card does not break the game.