What is Threading?
As with a number of things, threading has a number of meanings depending on how it is used when referencing computing. In this case threading can mean something slightly different when referencing software vs hardware.
In software a thread is generally termed to be a sequence of instructions that can be scheduled for a processor to execute. A software process can consist of many threads of execution that can be running simultaneously allowing the process to be doing multiple things at the same time. Unlike forking, which generates a new and independent process, the threads remain part of the same process which means that the separate execution threads all have access to the same variables and memory space. Traditionally threads would be handled in a CPU by time slicing, a method by which each thread would be allocated time for it’s code to be executed by the CPU. These allocations would generally be handled by the operating system in order to maximize the efficient use of the CPU’s resources.
Software can be single-threaded or multi-threaded. The downside of a single-threaded program is that as it can only do one thing at a time the program will be unresponsive to the user while it is doing something. By running multiple threads, the user interface can remain responsive running in one thread, while other threads carry out the work being done behind the scenes. By splitting the program into more threads it allows the software to get more done by spending less time waiting on sections of code execution to finish before starting another. Another advantage to multi-threaded programs is that the software’s execution can be split over multiple processors where they are available to speed up execution.
In the early 2000’s, Intel launched their Pentium 4 Processors with Hyper-Threading, which makes use of a multi-threading concept in hardware. The idea was to make more efficient use of a CPU by making use of resources that may be sat idle while a thread is being run. The CPU has the section for the architectural state duplicated but still contains a single execution core. This allows the CPU to queue up two threads simultaneously although they are still executed one at a time. The operating system can then make better use of the CPU’s resources, as time that may be spent idle while a thread that may be waiting on data or have had a cache miss. Rather than sitting idle and waiting to continue with this thread, the CPU can switch to processing the other thread. These are termed “logical cores” in the CPU as it appears to the operating system that it is a second core of the CPU despite only being part of one.
The performance benefit to hyper-threading is somewhat dependent on the software being run and the situation in which the CPU finds itself. In subsequent years, multiple core CPUs have been released which take this idea a step further and fully duplicate the execution core of the CPU enabling multiple threads to be executed simultaneously on a single CPU.
When dealing with threads one of the most important links in the chain is the operating system on the computer. This makes important decisions with regards to scheduling the processing of the software threads, when to run them and which CPU core to execute them on (in the case of multiple-core systems) – or whether to queue the thread in a logical core (in the case of hyper-threading systems). How well this works can have a big impact on the performance benefits of being able to run in multiple threading environments.