The idea of unusable enhancements in battle has been brought up before (i.e. for Grapes of Wrath), and the ruling has always been as browarod stated. If there are no characters of matching brigade, then the enhancement is discarded immediately.
PAB's ability is unusual, and is obviously clarifying text. However, lately "clarifying text" is being disregarded and we have switched to more literal interpretations of cards. If we are going to go there at all, then we need to go there consistently, therefore I would agree that any evil enhancements played before PAB would have to remain in battle, even if there are no matching characters to use them on.