S-VHS
S-Video was developed as an interface for the Super VHS (S-VHS) consumer videotape format. S-VHS utilizes tapes and a signal with higher Bandwidth than standard VHS. With the addition of S-Video it's possible to take advantage of the fact that in both VHS and S-VHS recordings the luminance (brightness) and chrominance (color) information is separated, and can be sent to a TV without the artifacts caused by interference between the two. In standard VHS decks, where only composite video connections are available, the luminance and chrominance must be combined into one signal, producing those artifacts. For this reason, as well as the generally higher quality of S-VHS decks, it's recommended to use them for standard VHS playback and digital Capture to retain maximum quality.
Digital Video
Since S-Video connections are the highest quality option for connecting a standard NTSC or PAL analog TV, they're the standard choice for most Standard Definition (SD) digital video sources, including DVD players, DV camcorders, and various types of TV set-top boxes. However, the quality of S-Video on digital sources is still inferior to Component Video because of the two combined signals that comprise the Chrominance portion of the signal. Since Component Video sends either Y (Luminance), U (Blue Chrominance Component), and V (Red Chrominance Component) or Red, Green, and Blue (RGB) channels alread converted from the video's native YUV format. This limitation is due to S-Video's origin in analog videotape, where the Chroma is already combined on the tape so there's no advantage to separating it before it reaches the TV.
High Definition Video
Although some HD video devices have S-Video outputs, these output SD signals, and aren't generally considered suitable for connecting to a HDTV when HD capable connections are available. Likewise, some SD devices, like DVD players, include both S-Video and Component Video outputs. Once again for connections to a HDTV you should use Component Video rather than S-Video.
S-Video to Composite Video
Among video card manufacturers it's common to see additional pins added to the design of the mini-DIN S-Video connector. This is done to add Composite Video support through the same output. These pins are only used by S-Video to Composite Video adapters, generally provided with the video card. When the adapter is plugged into the S-Video port on the graphics card it receives the combined Luminance and Chrominance signal required for Composite Video from the extra pins.
Video Capture
When capturing analog video it's best to use a S-Video connection whenever possible. If you're capturing from videotape this will mean using a S-VHS deck, as standard VHS VCRs are only equipped with Composite Video. Set-top boxes for various digital TV services, from terrestrial broadcasts to satellite and Cable Television, typically offer S-Video connections as well.
Related Guides
Digital Video Fundamentals - Color Formats
Digital Video Fundamentals - Frames & Framerates
Digital Video Fundamentals - Lossy Compression