The Symphonies
Beethoven's 9 Symphonies are all classic works, and any good collection should have a complete set, but there are some Symphonies that are more 'basic' than others. Compiled are recommendations of complete sets and individual CDs so you can choose from a variety of recordings. As usual, this is not a definitive list at all, just my personal preferences.
THE COMPLETE SETS
The grandaddy of all recordings goes to Toscanini and the NBC Orchestra. RCA/BMG has finally compiled all of his Symphonies in a nice Box Set at an incredible price. The CDs are mono, but the sound is acceptable, and you just cannot overlook the way Toscanini works these pieces for all they are worth. The power and energy that Toscanini brings to these works is beyond compare. You quickly overlook the sonic shortcomings when you hear these recordings. To be honest, the sound quality is surprisingly good, and for under $30, this is a great way to get all the symphonies, or just fill in some works missing in your collection. Do not let the word MONO stop you from discovering this great set. This is not necessary good as an only set of symphonies, but a great way to fill in gaps, and you really should have some Tosc in your collection. Also available as an MP3 Album
Karajan recorded 4 complete sets, but his DG Issue from 1963 seems to be the best of the lot. Symphony 6 'Pastoral' is just too rushed in this set [see below], but the rest of the symphonies get a great interpretation. An older recording, but remastered with decent sound.
Klemperer on EMI is of similar vintage, but also comes with the 5 Piano Concertos and a few Overtures at a great price. 
Another good consistent set is from Bohm, also on DG, broken into several 2CD sets. These are solid performances, not too flashy, but a great set to have, and nice as fillers to complete your collection.
Vol. 1 Symph 1-2-4-5 / Vol 2 Symph 6-7-8 / Vol. 3 Symph 3-9
The Szell set with the Cleveland Orchestra is currently out of print, but 4 CD's are still part of the Sony Essential Classics series, and well worth getting. Basically Symphony 4 and 7 are not available [was released but now out of print, may still be around used], the other sets are 1&6, 2&5, 3&8 and Symph 9 with Fidelio Overture. All are available as MP3 albums, so should stay around for a while. There was a great complete set released from France with all Nine Symphonies plus the Five Piano Concertos, but looks to be unavailable now. A shame, these recordings should really be put back in a nice box set domestically and kept in print. The recorded sound is good for the age, Szell has a clean and sharp view of these works.
Update: The Beethoven 4 and 7 CD has been released by ArchivMusic through their ArchivCD program. Not at a bargain price, but good if you want to get the complete set. Link is below.
Available as CD and MP3:
Out Of Print Selections:
MP3 Version of Symphony 9 from older series:
INDIVIDUAL SYMPHONIES
Beethoven's most famous Symphony is the 5th, and the best CD is by Carlos Kleiber with the Vienna Philharmonic Orchestra, and includes the 7th Symphony. Also on MP3 Album.
Symphony 9 'Choral' is wonderful in the digital recording by Solti and the Chicago Symphony Orchestra from 1986. For a performance of the 'Choral' on MP3, try Ormandy and the Philadelphia Orchestra.
The only weak link in Karajan's 1963 set is the 6th 'Pastoral' but his recording in the 1970's is excellent [also on MP3 Album]. This is not a peaceful interpretation, not 'Pastoral' at all, but the sound is great, and the performance makes you stand up and notice. Critics are mixed about Karajan's 6ths, but I for one think it is a forceful interpretation, although maybe not what Beethoven had in mind. For a more mellow and open 6th, try Walter and the Columbia Symphony, coupled with Symphony No. 4, also on MP3 Album.