Capacitors are used to increase the maximum power output of the Power Station they are surrounding. If the facility consumes more power than the Power Station can supply, it will shut down along with the capacitors (overdraw).
A maximum of 12 capacitors can be around any one power station, forming a 5x5 square around it when fully upgraded. However, capacitors added to the corners of the power station will not increase power supply.
The first Capacitor is provided for free at the start of the game (in the supply truck), presuming it was successfully delivered.
If the power station short-circuits, each capacitor around the power station also turns off and may be turned on again individually, however turning a Power Station on will turn on all surrounding Capacitors.