The starting neutral fleets are random, I've seen 15 point fleets over Sullust and 45's, so that's something you can't control, other than a restart. But once you have 2 tectors and say a dozen 1 point carries none of the neutral fleets can stop you.
I would recommend not building any space stations or hvg's, you simply can't spare the money, and later when you have the money then you're already winning.
Don't underestimate how powerful the 1 point carrier is, those tie hunters have ion cannons and they can kill the shields on capitals very quickly for your tectors to then easily kill them. I like to have at least 12 in a fleet, those tie hunters hold up well against other fighters while you ignore them and attack the enemies capitals, imo the tector is the best unit the EA has, i just find ssds annoying personally and the Praetor is good, but you can have 2 tectors for each one. I will hyperspace in and let the enemy come to me. While they're on their way to you, use your tie hunters to take down the shields and engines on a capital and then immediately move your hunters to the next capital. Usually by the time the enemy ships get to my capitals none of them have shields and because you took out engines too then they get to your units peace meal as well.
Also the neutral fleets will never go on the offensive, so as soon as you have a link to an enemy planet go hard after that enemy and ignore the neutrals until you have to go through one to get to more enemy planets. When I play FTGU i will end up with lots of neutral planets still left to mop up after the other factions have been eliminated.
If you would prefer to turtle more then don't station a fleet in a system next you your enemy, the enemy will always send a fleet they think is capable of destroying what you have stationed there, so if you have a 60 point sitting at your planet the enemy will send 90 or more to beat it. If you leave nothing stationed there then the enemy tends to send 20 point fleets, I prefer to leave level one space station planets undefended because it's cheaper to just rebuild that level one after the enemy shows up. I keep reaction fleets near but not at my borders and they can be as small as 1 tector and some support ships, i rarely see the enemy send anything substantial into one of your undefended systems.
Also if you've just won a battle over an enemy planet, leave a storm trooper in orbit and move right on to the next planet, you can stop the enemy from building a new station and if they come at you all you lost was a stormtrooper.