Honestly, the best way to speed up your defense is usually by using a faster offense. Looking at your decklist, I believe that to be the case here. In my experience, Silver + NT Prophets is slow. There just isn't much speed to be had.
I'm guessing Silas is in this deck to make it easier to play Gam's Speech. In my experience, using a character with a weak ability for identifiers only (brigade, etc.) or to play enhancements of certain brigades is a bad trade. I love Gam's Speech, but if you have to include Silas to do it, it might be best to drop both and try 2 different cards. You want your characters to have strong abilities, because they can be reused.
Masquerading isn't worth the card slot just to give your one demon Cannot Be Ignored.
You have a LOT of EEs that won't win you a battle outright. You've got some stall blocks in that defense, but the deck probably won't be fast enough (or last long enough) to get all your combos set up before the opponent has rescued 5. Obviously getting AiG and Stoic in play would help a lot, but that's going to be hit-and-miss.
To expand on that, I'm worried you're trying to do too many things on defense. If you find that the defense is inconsistent, I'd recommend focusing the defensive list to be smaller and more potent.
Ultimately, the best way to know how your deck will do is play games with it! I recommend building this deck on Lackey and playing against a good player using a strong offense.
