Taking about 1mm off a door

I’ve made the attached wardrobe, but the doors are perhaps too snug. Having hung them, they rub on both sides when closing.

Any way I can take literally around 1-2mm off the side? My circular saw would be a pain at that degree of precision and I don’t fancy sanding it as could lose the straight edge.

A hand lathe perhaps (I don’t have one but could if needs be)