My desktop is running on XP (32bit) SP3, with Realtek HD audio (ALC 888 chipset). I had the same problem with missing stereo mix. The realtek HD audio manager (under the mixer tab) had the stereo mix, microphone & line volume controls greyed out. Audacity would not show stereo mix but only line in. CamStudio could not record audio being played on the speakers.
To solve the problem, I did the following:
1. unzipped & copied the extracted file rtkhdaud.dat file into c:\windows\system32\drivers folder on my system (the folder already contained rtkhdaud.sys file; I retained that file).
2. Rebooted the system
3. RTK HD audio manager now had functioning buttons & sliders that were greyed out earlier.
Source of the file: Realtek tech support.
Download file from link below: