First off, thanks for all the great replies, makes me feel a lot better (I was starting to think I should sell my PPG for a powered hang glider)....
I am flying a Fresh Breeze Simonini, and a Powerplay Sting wing (DULV 1, DHV 1/1-2) and I have a total of 5 hours experience. I also have 12-15 hours hang gliding experience.
I do fly actively, I think, I keep about 5 lbs brake pressure (number 2 according to the PPG Bible) and I check my surges. I just don't have the experience to feel if I am flying properly.
As Dan the Man said, it is probably just turbulence. The more I think about it and the field I flew from it just makes more sense than thermals 30 minutes before sunset. It is also the strongest wind I have flown in with my ppg. There was a small 'ridge' where I was feeling the lift the most, only about 30 feet high and I was around 150' up so I am not sure if that is what I was feeling but it makes sense.
Also the only reason I went over the barn, which I knew would be bumpy, was because there was a field on the other side more suitable for landing and I wanted DOWN. LOL.
Simon thanks for the great link, learned lots of stuff
As for flying higher, that is what I wanted to do but I figured I would end up going backwards.
All added up, it is obviously Idiot Pilot Error for even taking off. I am just glad I was safely able to learn something from it. Thanks again everyone and happy flying.