Temp gauge rises slightly then falls - thermostat?

My '93 Accord usually handles these deep freeze winters well. But for the
last few (very cold) days, the temperature gauge hardly rises even after a
long period of driving. When I am at a stop and the car is idling at low
engine speed, the temp gauge sometimes rises slightly. But when I take off
and accelerate again, it falls back to its lowest point.

Is it safe to assume that the thermostat is the culprit here? I am guessing
that the thermostat the car has now is the original (I've had the car for
only 5 years and have not had it replaced since I've owned it). It is also
taking longer than usual to warm up on the inside.

Am I assuming correctly?

 

Yes.

 

Ditto.

Replace with OEM. It's cheap and much better-made than anything you'll get
aftermarket.

 

Ditto.

 

Thanks. And yes, I will definitely get an OEM replacement.

 

Replace the t-stat with a Honda brand name only or the new one will not
last long