In general, the relationship between PWM duty cycle and motor speed can depend on a number of factors, including the PWM frequency, the details of how the driver operates (e.g. fast decay or slow decay), and even the motor itself. This old forum thread gives some examples:
What motors are you using? How are you measuring the speed? I expect what you are seeing is normal for your particular setup, but if you post a plot that shows the speed you are observing as a function of duty cycle, I can see if anything looks obviously amiss.