It all depends on the amount of purchase, price of the house post purchase, credit ratings, current market in interest rate etc.
If you put down more than 20% down, you are probably not paying PMI, and maybe (!) Your closing costs were low enough. However, if you didn't put down 20%, and the house appreciated enough for you to save on PMI, and you can find a low cost refinance institution, then you do it.
My current house was refinanced a year after we bought it, because interest rates dropped low enough, the house appreciated high enough, the refi was cheap enough that it was worth it.
11 years later or house was paid off in full.