We present measurements of dust reddening using the colors of stars with spectra in the Sloan Digital Sky Survey. We measure reddening as the difference between the measured and predicted colors of a star, as derived from stellar parameters from the SEGUE Stellar Parameter Pipeline (Lee et al. 2008a). We achieve uncertainties of 56, 34, 25, and 29 mmag in the colors u-g, g-r, r-i, and i-z, per star, though the uncertainty varies depending on the stellar type and the magnitude of the star. The spectrum-based reddening measurements confirm our earlier "blue tip" reddening measurements (Schlafly et al. 2010, S10), finding reddening coefficients different by -3%, 1%, 1%, and 2% in u-g, g-r, r-i, and i-z from those found by the blue tip method, after removing a 4% normalization difference. These results prefer an R_V=3.1 Fitzpatrick (1999) reddening law to O'Donnell (1994) or Cardelli et al. (1989) reddening laws. We provide a table of conversion coefficients from the Schlegel et al. (1998) maps of E(B-V) to extinction in 88 bandpasses for 4 values of R_V, using this reddening law and the 14% recalibration of SFD first reported by S10 and confirmed in this work.