Yeah, things that are considered on or in something else follow that thing wherever it goes. If a character is removed from the game while holding a weapon, that weapon will also be removed from the game; if a fortress is discarded while holding enhancements and/or characters (Musicians' Chambers, Storehouse, The Darkness, etc.), all of those will be discarded also.