First of all, that's not very impressive then.
Second, morality doesn't physically exist. It only exists as an abstract idea in the same ways chess rules exist as an abstract idea. People don't gravitate towards the golden rule because the golden rule is "true", it's because the golden rule arises naturally from human nature.
Lastly, morality isn't unchanging. It's something that changes all the time. Some morals do remain largely untouched such as morals stating murder as wrong - but it's not something that exists in some sort of physical state.
There's this idea that morality exists in a platonic manner.
This video goes over absolute truth, and it touches up on mathematical platonism. It's relevant because the same arguments against mathematical platonism also works against moral platonism.
https://www.youtube.com/watch?v=1YnlW59--JE&index=2&list=PL3IOkNR8_9gpQa5teO1xQANB-3MiY17uk