that's because you told it to go fast and then to turn, you gotta take it slower
if you double click its sides it will go fast, if you lean forward then it will go even faster... (you are now warned)
try just giving it a light click and holding the reins so they are not tight, but you're able to steer the horse by pulling lightly on them