I have also heard people say they will happily free fly in far worse conditions than they would ever attempt with a motor. But so far I haven't really heard why, other than comments like "it feels a lot less pleasant with a motor". I'd love to understand why.
Personally I get quite nervous when it starts to get bumpy, and I don't see why that would feel significantly different without a motor.