Is there a historical reason why Kazakhstan and Mongolia don't have a common border yet are so close to each other?

Nothing remarkable seems to have driven this oddity. Triple point borders or tripoints are fairly common throughout the world and although it does seem unusual to have two of them in such close proximity, the current layout is just the result of typical border defining events (natural obstacles, conquest/expansion, treaties and concessions, etc.)
Also bear in mind what is now the (nearly) four way meeting point only involved three entities during much of the critical periods where the borders were defined. What is currently Kazakhstan, was under the control of the Russian Empire and then, the Soviet Union. At the time the current border between what was then the Kazakh SSR and Soviet Russia was delineated, there was nothing noteworthy in Kazakh-Mongol relations so it seems doubtful the placement of that internal Soviet state border was altered specifically to avoid contact between the two.

https://en.m.wikipedia.org/wiki/Kazakhstan%E2%80%93Russia_border
https://en.m.wikipedia.org/wiki/China%E2%80%93Mongolia_border
https://en.m.wikipedia.org/wiki/China%E2%80%93Kazakhstan_border

Here's one explanation:

Does Kazakhstan Border Mongolia? No, Kazakhstan does not share a border with Mongolia. The two countries are separated by a stretch of land where the borders of Russia and China meet. The closest points between Kazakhstan and Mongolia are about 55 kilometers (approximately 34 miles) apart, with Russian and Chinese territories in between.
Why Kazakhstan and Mongolia don't share a border? During various historical periods, the geopolitical dynamics among regional powers significantly influenced the borders in Central Asia. The modern state borders were further solidified during the 20th century, especially during and after the Soviet era, as the USSR had a significant influence over the region, it was in USSR / Russia's best interest to keep the borders separate. Through various back-door agreements, China and Russia dictated the terms for most of the borders in Central and Inner Asia, thus for geopolitical and profit related reasons they agreed to keep Mongolia's and Kazakhstan's border separate.

https://explorekazakhstan.com/does-kazakhstan-border-mongolia/