It's also entirely possible that the save state battery is dead. They're getting to be 30 and 40 years old now.
I haven't tried personally, I prefer to run my games on original hardware. Not that I judge anybody for using a retron or similar clone console. I just see so many retron's that have a variety of issues including now power, shitty edge connectors, some won't read one type of cartridge, etc. We've had a few customers at our retail stores that had Retron5's that were brand new out of box take a dump after only a day or two, made us have to deal with Hyperkin directly. They're QC seems to need a bit of tweaking.