I never thought to go back to putting the FLAC on my Phatbox, now that I've got the space. I'll give it a try!
If your sound system and your ears are good enough then you will certainly notice the difference (and you don't need hi-fi "golden ears" - just the mark one originals in good working order). It's a bit like having the "sparkle" put back into your music. You almost certainly won't have noticed that it wasn't there, but when you can hear the difference, you won't want to go back to how it was before.