The fundamental issue is that hot things get cold. There is no arguing about that. Your forgotten cup of tea didn’t remain piping hot and instead became a frozen tundra. But why does this happen?
The culprit, as always, is physics. Specifically, kinetic probability.
As you may remember from science lessons at school, as something heats up its particles within it begin to vibrate and move about more. The more heat, the more those particles move; they gain kinetic energy. It is how solids turn to liquids and how liquids turn to gases. And of course, it works the same way in reverse: the colder something is, the less the particles move; they lose kinetic energy.
As these particles move about, whether it be slowly or quickly, they will collide with one another. As they collide, kinetic energy will be transferred to the particle that has been hit. Hence, if a particle with high kinetic energy were to hit a particle with low kinetic energy, the former would lose some kinetic energy and the latter would gain some kinetic energy, spreading it between the two particles. This process keeps happening with the higher kinetic energy particles gradually losing more and more energy with each collision.
This is what is happening with your cup of tea. The hot tea particles (correct terminology) are bouncing around with high kinetic energy, hitting each other and anything they can. As they collide with lower kinetic energy particles, like the mug or milk, the net kinetic energy will decrease as a result of the transfer of energy to those new elements, which in turn will get slightly warmer.
But then if a high-energy particle hits a low-energy particle causing a transfer of energy from high to low, the reverse should also happen from low to high. A low kinetic energy particle, as slow-moving as it may be, could hit another particle and transfer what little kinetic energy it has. Then theoretically, in a cold environment, your cup of tea could warm up slightly. So why doesn't that happen?
This is where kinetic probability comes into play. Sure, a high kinetic energy particle can hit a low kinetic energy particle and a low kinetic energy particle can hit a high kinetic energy particle but it is the former that is more likely to occur. The reason is that higher kinetic energy particles move more. Something which moves more has a greater probability of hitting something else, whereas something that doesn’t move is less likely to crash into something else.
To summarise, hot things become cold rather than the other way around because hotter particles (with more kinetic energy) are more likely to hit something and lose energy.
P.S. Americans stop making tea in the microwave.