Refinancing house mortgage

Thinking of refinancing mortgage. We previously worked with a lender when we bought our house five years ago. This time he gave us a higher % deal (3.625) with a high closing cost (~$5000). Is it better to contact a bank directly or is it time to look for another lender? Or those are the todays’s rates? My husband and I have quite high credit score and good income. So this high number doesn’t make any sense. First time doing sometime like this, so feeling clueless. Thank you!