Yes, I've had one go bad on my 4.o in the MJ. It didn't vibrate much, but it would squeak. The rubber had failed between the hub and the pulley part, and the two could move a bit independently, caused the serpentine belt to be out of alignment with the rest of the system, so it would chirp. New balacer solved the problem.
Also replaced the balancer on my FSJ's 360, and it smoothed it out a bit. The rubber was swelled due to engine oil leaking on it. Replaced crankshaft seal at same time with balancer on both the FSJ and the MJ. No more leaks at the front of the engine.