A Graphics Processing Unit, or GPU, is a chip dedicated (mostly) to graphics rendering and video processing. A GPU is capable of 3D computer graphics rendering much faster than a traditional CPU due to their parallel structure.
A GPU can be present on a video card with dedicated high speed memory, or as is the case in most PCs and laptops shipped nowadays, it could be integrated and reliant on the RAM in the system for memory use. The latter is much less powerful when it comes to high-performance rendering and will be inadequate for many gaming titles.
A GPU on a card is generally also responsible for driving the monitor(s) for the computer. GPUs also have other specialist features such as video decoding abilities.
In recent years, more emphasis has been put on using high-performance GPUs for more common tasks, such as within a web browser to drive web graphics and applications, or with resource intensive tasks such as photo or video editing.
An example of how a GPU can be exploited to carry out other tasks is Folding@Home. The project simulates protein folding in a distributing computing system. Both ATI and NVIDIA graphics chips can be used to do the simulations too, and do so at a much faster rate than traditional CPUs, thanks to their number crunching abilities.
A dedicated graphics card will generally plug into a PCI Express or Advanced Graphics Port on a motherboard.