I’m a little confused by your description (specifically if the servo is already at 5500, the maximum position setting, how can it turn more left?) but I think I understand your problem.
In general, a servomotor has no real speed control. It only knows it’s current position and the position command it is getting from the servo controller. Even when you are not moving the servo, each servo is getting a position command from the servo controller 20 times each second.
If the servo is at the commanded position it holds itself there. If the servo is not at the commanded position, or gets a new position command, it always moves to the new position as fast as it can. If the servo gets no signal, it does not power the motor, and you can move it by hand.
The Pololu servo controller reduces the servo speed by sending several position commands. For example, if the servo is in position 500, and you send it to position 5500, but you have a slow speed set, the Pololu controller will actually send out several position commands over time (1000, 1500, 2000, 2500, 3000…) The servo is still moving as fast as it can between these positions, but the overall speed appears slow and smooth.
When you turn off the servo using CMD 0, you are only turning off the command signal to that servo. When you turn the servo on again, the Pololu controller has no way of knowing if the servo has been moved, and will send the same position signal. Since the servo has no real speed control, if it has moved while it was not getting a signal, it will move as fast as it can back to the commanded position.
Could this be your problem?
If you want to let the joystick go without moving the servo you will need to handle it in your program, and keep sending the servo a position.
I hope this helps.
-Adam
P.S. I assume you are using analog servo motors, which you can move by hand when they are not receiving a signal. Most (all?) digital servo motors will hold a position after they stop receiving a signal. If you use Hitec brand digital servo motors and a Hitec Digital Servo Programmer:
http://www.shopatron.com/product/product_id=RCD44410/110
you can program the servos themselves to turn more slowly. This is a much more expensive solution to your problem though (the programmer alone is $180).