Your next step should be increasing your personal Levies. Coincidentally some of these also happen to be the cheapest options which make them even better choices for the early game. When fighting a defence battle these roll modifiers can turn defeat into victory, but if you were to only choose one, the Bailey offers much better benefits and remains viable well into late game. A good starting point may be to build a few Defender Advantage buildings such as the Outpost and Bailey. What you choose will be dependent on what you need most at the time, and all buildings can be replaced at a later date, although you do lose the initial investment.