Make sure you have the latest kernel, there was a problem with buster and rtl-sdr devices:
sudo apt update
sudo apt dist-upgrade
Apart from that the power supply is the most likely problem.
Also make sure you have the location set precisely (at least 4 decimal digits).
You can script automatic reboots when the clock is unstable, but i’d check for the other possibilities first.
If you want some inspiration in regards to scripting:
Dump1090-fa stops sending messages after some time - #12 by wiedehopf
It checks for the stick giving up completely though as you can see with the grep command.