In this case, I am assuming you have verified the firewall on either side, and the network conditions are not an issue (eg. no packet loss, no crazy jitter and normal latency).
In some phones such as Grandstream or on your PBX, make sure you do NOT use the "adaptive jitter buffer" or similar settings. If possible disable that or change it to "fixed".
If you do, you may wonder why when you first start a call the user cannot hear you or the voice seems very choppy and roboticly slow/fast, this is because of the "adaptive jitter buffer" breaking things.
If you have any such setting in your PBX, ATA or phone firmware, disable that as it is often the cause of weird voice quality issues.
stuttering, audio, voip, audioin, verified, firewall, eg, packet, jitter, latency, grandstream, pbx, quot, adaptive, buffer, settings, disable, user, choppy, roboticly, ata, firmware,