The digital library and streaming services are great for checking out new stuff, singles, and the mighty, might playlist, but the only real option worth pursuing if you're interested in building a music library is one of recording's oldest formats: the vinyl record.
Albums, LPs, and 7" are even more available than they were ten or twenty years ago, and the deep back catalogs makes it a great option to get ahold of the classics. Of course, if you're going to play them, you'll need some sort of amplifier and speaker system, perhaps a preamp, and, of course, a turntable.
I like this roundup from Cool Material, which shares options from under $100 to $2500. (Oh, and $650,000+ but that one's not really available). Of course, if you're just getting started, you'll be mostly interested in the first couple options at the bottom end of the spectrum, any of which should suit you just fine.
The Best Turntable for Every Budget [CoolMaterial.com]