I think it is time to end this?

I think u deserve better. You need to talk to him and let him know that you are done with his ish. He seems like he doesn't care and just wants out. Why prolong it? He deserves to be alone.