Computers have two types of storage system: primary storage and secondary storage. Primary storage or memory is directly connected to the motherboard and is the place where programs and data are kept while being used. Secondary storage is the place where software and data are kept when not in use. Some individuals (and references) use the term memory to refer to both memory and storage but we will use different terms on this Web site.
Memory is volatile, meaning that when the power is off the memory cells are empty. Secondary storage is non-volatile or "permanent", meaning that when the power is off that its contents are still there. Consequently storage is used for long-term storage of programs and data.
The most common technologies for storage devices are magnetic, optical, and integrated circuit.
Magnetic Storage - The most common form of magnetic storage is the hard disk or fixed disk. Almost every computer has at least one hard disk. External hard disks are also available. They can be plugged into a port of your computer. Each hard disk consists of multiple platters which contain two surfaces. Each surface is composed of many tracks (concentric circles) and many sectors (pie shaped wedges. The intersection of a track and sector is referred to as a disk block. All blocks on a disk have the same capacity. When data is read from a disk or written to a disk it is processed in collections of blocks called clusters. The larger the clusters are the faster the computer can read and write data to and from the disk. But by using smaller clusters less space on the disk is wasted. Typically the operating system determines the size of a cluster although the user may have some options.
Optical Storage - Optical storage consists of CDs and DVDs. Unlike magnetic disks in which the tracks are concentric circles, on an optical disk there is a single track which spirals into the center (similar to the grove on a phonograph record). This difference makes computer CDs and DVDs compatible with standard CDs and DVDs.
Integrated Circuit Storage - IC storage has the advantage of having no moving parts and therefore data can be transferred more quickly than with optical or magnetic storage. However IC storage tends to break down after being used a great number of times (typically in the thousands or hundreds of thousands of times).
Wikipedia article Computer Data Storage
Wikipedia article Hard disk drive
Wikipedia article Optical Disk Drive
Kingston Ultimate Memory Guide
howSTUFFworks How Hard Disks Work
howSTUFFworks Optical Media Explained 15 minute video