Comes from the same origins as removing your shoes at the door, I’d imagine. Considered proper guest etiquette or some other Victorian era nonsense like that.