These days, no matter where you live there are plenty of home heating options. Your location plays some part in the choice of course - really severe winters will certainly help you decide to go for pretty powerful heating solutions.
If you do have deep winters in your part of the world then electrical heating is unlikely to be the best solution - even though this technology has improved a lot, it is likely to work out as expensive if you have to fight off a lot of cold. It is a more suitable solution for smaller homes or warmer climates.
More than likely you will want to have some sort of forced water system - there are baseboard, underfloor and wall hanging options. For this you will need a home heating boilers, and this means choosing your fuel source.
Not all fuels make sense in every area. Some people do not have natural gas supplies in their area. It has some clear advantages if you have the option, but if not gas central heating boilers may not be the best option. (Of course they can be fueled with liquid propane.)
Oil fired home boilers are often a smart option, though as any user will know, you are open to the whims of the market when you have to by your fuel - it can get costly in winter for individual deliveries, but overall if is a reasonably economical system.
For some, home wood fired boilers are a serious consideration. Plenty of effort has been put into developing this technology since, if fueled from sustainable forest sources, it is probably the most environmentally friendly fuel aside from wind, solar and wave.