I have had a decent amount of tournaments with only three people. Many of those being myself, my wife and either a friend or cousin, or Jonathan. My last few tournaments have been around 5 people, with his kids in addition.
There was at least one time back in Kentucky that we played a tourney with Mark, myself and my wife, either at his place or mine.
If you can take the hit in tournament cost, at least you win big when you end up playing in every category.