Expansion of your colony is critical but also comes at a price, many in fact. The larger your colony, the larger the area you need to build defenses for. Unlike other games, there is an additional price to pay when your buildings get overrun by the enemy. If you expand too quickly, you could unintentionally turn your expansion into a horde of zombies that end up kicking your ass.
Expanding slowly comes with two benefits. The first is that it allows you to build a layered defense, something I have discussed in the guide to creating a good defense. A layered defense will allow you to withstand a wall breach if a horde manages to break into your base. If nothing else, this reason alone is enough of a reason to take expansion slowly.
The second reason I suggest is because expansions leave you vulnerable. Expansion comes at an economical cost and a military one. You need to send soldiers out into the new area to clear out any infected. This could trigger a horde. If not, you must still patrol the area while you wait for walls, towers and other defensive structures to be completed. This leaves you at risk of attack. You will be spending lots of resources to do this too. Resources that could fall short if you find yourself having to react urgently to a sudden attack from the enemy.
Rather than take on a huge area of land to expand do with several choke points to monitor, take over a much smaller area. Yes you will lose some of the total expansion space building walls and towers that you will eventually expand beyond, but these small expansions ensure you stay safe while you do it and that you have a kick ass defense to fall back on when things go south.
Do not expand toward an infected village
Even if you feel like you have the resources and firepower to clear an infected village, do not expand toward it until the threat is over. If you expand toward it and your defenses are still being constructed, you could end up with a massive horde within the outer ring of your settlement which will increase in size and potentially break in further.
Until you have cleared an infected village from the map, expand in the other direction. There is a guide in this wiki that will explain how to clear these infected villages quite easily. I would suggest you follow this first. There is a good reason to make it a priority to get rid of them anyway.
Build out before you build up
This benefits the economical side of expansion. Upgrading a house will double the colonists and workers that it provides but it costs far more than double to upgrade. You can build more than two tents for the price of one wooden cottage. Use up all of the available space before you start upgrading buildings. This will ensure you expand as quickly as possible without hurting your resource generation too much.
With this being said, never expand beyond the area you are capable of protecting. If you expand into a new area and start building lots of tents but do not defend the area well enough. A single zombie could turn into a large horde and overrun your command center.
Pausing resource buildings
If you have multiple sawmills or a quarry and your wood or stone supply has reached the maximum, you will not be able to earn any more resources, but the buildings will continue to function. Aim to build a market as soon as possible, but this requires you to have 200 colonists. Until you have one, any resources you generate when the resource cap is hit, are gone. Pausing the building will free up the workers and power so that you can spend it on other buildings that are needed. Once you have the market you can turn it back on and it will generate some handy gold for you. In the early days, all of the workers and power you get will be badly needed.