We looked at early digital computer memory, see History of the Computer – Core Memory and mentioned that the current RAM (random access memory) standard is chip memory. This is consistent with the oft-cited application of Moore’s Law (Gordon Moore is one of the founders of Intel). It states that the component density of integrated circuits, which can be paraphrased as performance per unit cost, doubles every 18 months. Early main memory had cycle times in microseconds, today we are talking about nanoseconds.
You may be familiar with the term cache as applied to personal computers. This is one of the performance features mentioned when we talk about the latest processor or hard drive. You can have L1 or L2 CPU cache and disk cache of different sizes. Some programs also have a cache, also known as a buffer, for example when writing data to a CD burner. Early CD burning programs had “overflow”. The end result of these was a good supply of pads!
Mainframe systems have used cache for many years. The concept became popular in the 1970s as a way to speed up memory access times. This was the time when main memory was being phased out and replaced with integrated circuits or chips. Although the chips were much more efficient in terms of physical space, they had other problems with reliability and heat generation. Chips of a certain design were faster, hotter, and more expensive than chips of another design, which were cheaper but slower. Speed has always been one of the most important factors in computer sales, and design engineers have always been on the lookout for ways to improve performance.
The concept of cache memory is based on the fact that a computer is inherently a sequential processing machine. Of course, one of the great advantages of a computer program is that it can “branch” or “jump” out of sequence – the subject of another article in this series. However, there are still enough times when one instruction follows another to make a buffer or cache a useful addition to a computer.
The main idea of the cache is to predict what data is needed from memory to be processed in the processor. Consider a program that is composed of a series of instructions, each stored in a memory location, say from address 100 upwards. The instruction at location 100 is read from memory and executed by the processor, then the next instruction is read from location 101 and executed, then 102, 103, and so on.
If the memory in question is main memory, it will take maybe 1 microsecond to read an instruction. If the processor takes, say, 100 nanoseconds to execute the instruction, then it must wait 900 nanoseconds for the next instruction (1 microsecond = 1000 nanoseconds). The CPU’s effective iteration rate is 1 microsecond.. (Times and speeds quoted are typical, but do not refer to specific hardware, but simply illustrate the principles involved).