I need some helpful advice on this situation?

My ex bf & I broke up a little over a month ago due to us fighting & him getting angry at times, at which I didn't deserve. After we broke up, I went out with friends a lot, and even though I kept in touch with, I was being in a way aloof because I didn't want to jump back into things like he wanted to. So I was giving us time. But he begged & wanted me back, BUT I would never ignore him & I never said I wouldn't get back with him nor did I ever tell him I was confused but that we had to take a little break. Like I said, during that time of the break I was living the life & didn't let myself get sad, even though he said he was sad all of the time. Well now the tables have turned & he is behaving now like I was and now I'm the sad one. So when I want to hang out with him he says he doesn't want to as often because he says he doesn't want us to fight & he needs to work on himself before we get back together. But it's making me have the WORST anxiety because of the "not knowing" and I miss him. So I don't think it's fair that he's now being the same way, since it's just prolonging us possibly getting back together. How do I handle this? Because it's making me upset & I don't know what to do.