Go to your control panel, select Add\Remove programs (or Programs and Features in Vista, Windows 7, and Windows 8) and uninstall anything you no longer use. Occasionally you will find a toolbar or other item you were unaware was there!

Cleaning your hard drive of unneeded files:
One of the biggest problems is hard drive usage. Your hard drive has more moving parts then any other components in your computer and is constantly adding, deleting and reorganizing files. Many of the files are temporary files used to installed programs that are no longer needed and are still on the hard drive. You may want to clean your hard drive of these temporary files. A great tool that can help is CCleaner: http://www.piriform.com/ccleaner. 

Startup items:
If you are wondering why you have to wait a bit to use your computer after startup, it is because there may be programs being loading after Windows has started. They also use up memory just sitting there. You can prevent these programs from starting up that you do not use or frequently use. Do this should greatly improve performance. 

Remove invalid registry entries:
The registry is generally overrated as a speed tweak. But it does get large and often contain many invalid references. The registry is not an area for even the average computer user to be playing around in. It recommended to locate known and reliable registry cleaning tools or get professional support. If the registry is compromised with any wrong doing, the operating system will crash and the computer will fail.

 
Defragment your hard drive
Adding/removing programs and saving/deleing data files tend to leave random and unsorted empty spaces on the hard drive. This is known as fragmentation. Excessive fragmentation generally will degrade your system performance. Reading and writing data on a heavily fragmented hard drive will slowed down because the time for the hard drive read/write heads to move between fragments on the disk surface can be substantial. Microsoft includess a disk defragmentor. In general, this tool will do. However, when you do excessive adding and removing programs and files, it would be suggested to seek a dedicated defrag tools.