I have had the last 3 50mm 1.4s (AF, AFD, AFS) and the AFD 1.8. I had the 1.4 afs since before the new 1.8 came out so I haven't seen any reason to try that one. I honestly don't get the "slowness" claims some make of the new AFS. It is barely slower than the 1.8 AFD - but when it focuses, it is spot on more than the older 1.4s. Some really forget that at 1.4, the focus is razor thin and the older 1.4 missed focus quite a bit. Sharpness & contrast wise, I never saw much difference - all the 50s are tack sharp in my eye.
That said, I personally don't think it all that different from the 1.8 other than the bokeh highlights are rounded rather than hexagonal. I think the bokeh is a bit smoother and that means more to me personally than many others. Other than that and being able to MF override, the difference is really small. For me, the Bokeh and MF override sold me to upgrade.
Is it worth dumping two lenses for one? I love the 85mm 1.8afd and ask the same question you did but with the 85 1.4. If you love shooting with the 50mm and do so all the time and never use the 85, it might be worth it. I have basically the same setup (minus the 17-55 & plus the 70-200) and although I often think about moving one direction to another, month - in and month out, there is rarely a low light situation that really a 35,50,& 85mm 1.8 can't handle when I run into it.