Yes, that's possible. That's what we do. Only for 2-3 hours max. but we have often not needed the water and kept it warm all night.

I sent the warmer back because I didn't like it. This is how I solved the problem of keeping boiled water warm for bottles. Kettle from Solis with thermo function and temperature selection as well as thermos jug. This way you always have water for shopping that is not too hot or that you can heat up quickly.