I was under impression that font-smoothing is another major performance killer (especially if there's a lot of font visible, e.g. when opening quest journal window).
But then again I didn't properly test it (aside from the fact that opening my horizontal quest journal serioulsy kills my framerate) -- it might be attributed to tileing or something else.
I tried to switch font-smoothing off and the game looked disgusting
Perhaps I could've worked around it by changing a lot of fonts, etc., but personally I prefer to customize as little as possible of my UI (makes life easier when updates happen).