What is the meaning of the word "downs" in the name of most horse racing tracks, i.e. Churchill Downs?

my understanding is that 'downs' are a geographical feature, like plains and valleys etc....

Correct, in Britain, in a contradictory way, downs are ranges of hills. It's actually related to the word 'dune', meaning an undulating feature of land, and is not related to the directional meaning of down.
Not sure why so many tracks in the US are called downs though, unless all named after Epsom Downs over here which is definitely up in the hills and has an incredible view.