When this thread was started even though I'd joined the forum I'd only just started to get into Neil Young so I didn't contribute at the time. Being a lot more familiar with his career now - now to mention becoming a big fan of his - I will now offer some thoughts of mine.
Overall I think Rust Never Sleeps is Neil's best album. The first side showcases Neil at the top of his acoustic game with five excellent acoustic compositions - I must say I particularly love Pocahontas, which has some really inventive and original lyrics combined with a very memorable tune, while the latter side showcases Neil & Crazy Horse at the top of their electric game. Powderfinger is an exceptional song, it really rocks but also has terrific lyrics. There is also the small matter of the album opening with the acoustic My My, Hey Hey (Out Of The Blue) and ending with the absolutely filthy sounding guitar riffs of its electric brother Hey Hey, My My (Into The Black).
After The Gold Rush is my favourite of his mostly acoustic albums as I think it is a very cohesive record. My two favourites are Tell Me Why and Southern Man although I also love the title track and think that Don't Let It Bring You Down is another great song. Thinking about Harvest reminds me of the number of great songs it contains (Out On The Weekend, Heart Of Gold, The Needle And The Damage Done, Old Man etc) but I think that Gold Rush edges it out due to its greater consistency. I am fine with the orchestral numbers on Harvest and actually like A Man Needs A Maid quite a lot but I have to be in the mood to enjoy There's A World, whereas I always enjoy all of the tracks from ATGR.
Live Rust, the live companion to Rust Never Sleeps, is my favourite Neil Young live album. Like Rust Never Sleeps Neil is great in both acoustic and electric mode - I dearly love the performance of After The Gold Rush from the opening acoustic segment while I consider the electric Crazy Horse performance of The Loner to be the definitive one. The other songs (Like A Hurricane, Cinnamon Girl, The Needle And The Damage Done and so on) are all great as well although the one slight disappointment is that on the CD version the performance of Cortez The Killer - a big personal favourite of mine - is substantially cut, losing most of its intro. As much as I love Live Rust I think Weld is an essential purchase if you love electric Neil Young & Crazy Horse, where there is overlap between Live Rust and Weld I find the latter offers the superior version of the song in question, and this is bolstered by great performances of later songs like Crime In The City and F*!#in' Up. However, it is exclusively electric and therefore Live Rust is my favourite as it also has some acoustic songs.
Having recently added On The Beach to my collection I think that is a great album too, See The Sky About To Rain, Revolution Blues, the title track and Ambulance Blues are all fantastic songs. I probably prefer it to Tonight's The Night which while undoubtedly very raw and real in its feel is a bit difficult to listen to if I'm not feeling a bit down, by contrast with On The Beach I think a song like Walk On balances out the bleakness of the title track and the anger and/or fear in Revolution Blues.