the best way to design a balanced rolemadness game is to not actually want to design a rolemadness game and instead just slap unbelievably shitty abilities on VTs for button purposes
the only role madness setup I’ve designed (17’er) had 1. Cop 2. Mason Communal Jailkeepers 3. 2-Shot Dayvig + fake poisoner 4. A bunch of abilities that were either entirely useless (Town 1-Shot Governor, Town Lawyer, Town Doctor but only heals attacks from 3ps, of which none are in the game) or related to some game-mechanics (like a Coroner to counter the Wolf Deathmiller that could only make players flip 3p, or a role that could send messages to every currently poisoned player)
basically what Icibalus already said
I usually start with the desired core mechanics with a balanced PR mix and a healthy amount of VTs against Goons, and then slap the funniest and least useful abilities on the vanilla roles afterward
In my experience, once you have a balanced 7p or 9p setup with a healthy amount of VTs, then that works
but actually finding a balanced setup there is hard, because everything is really finnicky
you don’t have the buffer for error that you do in larger games (1-2 townies on D2 can make a huge difference in faction win chances), and roles are inherently swingier in smaller games, so expect a lot of questioning yourself on exactly which faction the setup is biased towards. If you find a way to make it not swingy but still have the mechanics/roles for you want, good on you.