Emulator

you cannot simply decide to recompile a mono-core program into a myriad of small independent programs to run through a GPGPU due to their different nature and goal. It would be totally inefficient because of the overhead implied by calling GPU kernel functions one after one in a CPU function since it's the way a mono-core program is running.

I never heard of this way of accessing the GPU, is it even practical? The closest thing I know for direct GPU programming is libraries like Physx, that must be of course supported by the GFX driver.

The efficiency of multi-tasked program is another challenge because it is not an easy job to find the best way to run smoothly all the threads with efficient synchronization (for instance, abusing mutexes tends to be counter-productive; that is the reason why I mostly avoid them). Now try to be portable in several platforms, your will lose your hair.

Some things can be multithreaded relatively easily, tough. PCSX2, for example, has a multithreaded recompiler for its VU chip, which is kinda simple since the VU itself is a multicore chip. You just create a thread for each core, and since the real hardware is made of separate cores, they don't need synchronization. The sync problem comes when you try to split one emulation task in two or more threads.