Garbage collection is a background process, which cleans up objects that are no longer in use by your program. Garbage collectors were originally designed for symbolic languages, such as LISP. Smalltalk and Java also come with built-in garbage collectors.
Garbage collection is not free (TANSTAAFL applies here as everywhere). Garbage collection brings significant run-time overhead, and C++ does not provide garbage collection for this reason. I might argue that garbage collection should be a compiler option, but that would be beside the point. Understanding how garbage collection works can provide insight into the cost, into how other languages do implement it, and into what it would take to create your own.
One of the most common algorithms for garbage collection is stop and copy. In stop and copy you divide your memory in half. The first half is designated as working memory, where all of the "real" objects live, and the second half is designated as free memory. When working memory fills up, you begin garbage collection. This is a process of locating all of the useful objects in working memory and transferring them to free memory. You compact as you go, disposing of unneeded objects. The result is that free memory contains the useful objects and none of the garbage. You now make free memory into the new working memory, and free the old working memory to be the new free memory.
Another algorithm used for garbage collection is called mark and sweep. In mark and sweep you start at the globally available objects (e.g. static objects) and use them as the center of a pattern that fans out from them, following the pointers in these objects. Any objects reached are "marked". All objects are then reviewed. Those that are not marked are discarded as garbage. A new mark value is used for each garbage collection pass to ensure that the marking phase is done correctly.
Traditional garbage collectors ran a complete mark and sweep or stop and copy cycle. This meant that the user might be forced to wait while the system "did nothing". In some cases, this could take several seconds, or even several minutes. More modern garbage collectors employ incremental techniques, cleaning up a little bit here, a little bit there, so that there is no obvious delay in the system's processing due to the garbage collection activities.