So I got my strut rod bushings replaced, hoping that would finally solve my issue. It doesn't seem to do it all the time, but when I brake especially going downhill, the car seems to pull to the left side. The harder I brake, the harder it pulls. The tires, brake pads, hoses, and rotors are new so there shouldn't be any inconsistencies with that.
What's the next thing to check? Everything else under the front end seemed to be fairly tight, the only thing I noticed any movement in is the ball end of the pitman arm and it's not much at all.