From the ALSA wiki
The Realtek ALC650 is a 5-channel audio card with SPDIF output. It uses the via8233 chipset. I purchased a Shuttle PC  with one, installed Redhat 8 and the sound card was the only thing that didn't function.
Realtek lists a Linux driver on it's site.  This driver is actually the alsa driver with a sample modules.conf. This driver did NOT WORK with my system, and didn't include amixer (which is useful for SPDIF support).
- Download alsa-driver alsa-lib and alsa-utils 
- Untar and place the directories in
./configure make install ./snddevices
./configure make install
./configure make install
Run alsaconf and follow the onscreen instructions. It should generate a section in your
/etc/modules.conf that looks like this:
# --- BEGIN: Generated by ALSACONF, do not edit. --- # --- ALSACONF verion 0.9.0 --- alias char-major-116 snd alias snd-card-0 snd-via82xx alias char-major-14 soundcore alias sound-slot-0 snd-card-0 alias sound-service-0-0 snd-mixer-oss alias sound-service-0-1 snd-seq-oss alias sound-service-0-3 snd-pcm-oss alias sound-service-0-8 snd-seq-oss alias sound-service-0-12 snd-pcm-oss options snd major=116 cards_limit=1 device_mode=0666 options snd-via82xx index=0 # --- END: Generated by ALSACONF, do not edit. ---
It took me a sec to figure this one out, and I don't know if it's just my setup. SPDIF support is controlled by the sound control element called 'IEC958 Playback AC97-SPSA'. The gnome mixer program does NOT control the SPDIF output (the 'digital' output is something different). You can control it through alsamixer or amixer in the console (or you could use the alsa mixer for gnome ). The other trick is that the 'percentage' for output needs to be zero for the output.
Putting this in your
/etc/rc.d/rc.local should fix this:
amixer set 'IEC958 Playback AC97-SPSA' 0
from the via8233 page, this is expalined as follows...
If the playback device indicates a link is present, but makes no sound, check the mixer 'IEC958 Playback AC97-SPSA'. The S/PDIF always outputs at full volume - even muting the master or PCM mixers does not affect the S/PDIF volume. The meanings of the settings for 'IEC958 Playback AC97-SPSA' are
0. PCM1 1. PCM2,PCM1 (rear) 2. Centre and LFE 3. PCM3,Modem,Dedicated S/PDIF
Most users will want it set to 0 (PCM1). Some of the 82xx chips have a dedicated S/PDIF port, which is (I assume) accessed by setting it to 3 (Dedicated S/PDIF). The default appears to be 3.
The driver's been rock solid for me for a week! Happy listening! Please note that I only use the SPDIF support, although I have confirmed working analog output, too. Your mileage may vary.