Walk away from him & how to really do this?

Your problem is that he is a serial philanderer and by talking to him every night you show that you accept his phone calls and he knows you are going to pick up and you are always available to him. You need to show him that you mean business and you are not going to engage in these silly conversations. Put your phone on voice mail, change your telephone number and get on with your life. You will need to go cold turkey but the alternative is just more of the same.