"We've been friends a while." This is agreement, not explanation; it's perfectly possible for Trowa to know somebody for years and not trust them.

It's also worth noting, though, that the number of people Trowa refers to as 'friend' is vanishingly small. Even if this is telling Quatre nothing he doesn't already know.

"A lot of people have a hard time understanding her."

Which baffles Trowa slightly -- it's not as if it's hard, so long as there aren't too many abstract concepts involved -- but he's known for a long time that most people are astoundingly unobservant. So.