The situation is not uncommon... We have all been there with the roles reversed. Or done it ourselves.
As a guy I know more or less what the guy is thinking. He got what he wanted, he feels safe. He came over for the physical pleasure. There is a difference between love and sex. Or emotional attachment and pleasure if you like.
So sadly, it appears that he is not as much into you as you thought. He likes you but he doesnt wanna have any strings attached. I would say dump him and go for the next.
I had a similar experience, I was out with a girl, one thing led to sex...
And she became needy. She kept texting me and so on for like a week. Even though I liked her she was too on. That was a huge turnoff. To send a text message the day after is cool but dont milk it. If he feels like he is avoiding to answer or if he just sends you short replies its a sign that you need to take a step back and relax it a bit. He feels pressured. Not good! Its like you expect stuff that he is not willing to give... yet...So advice for the future, read his signals. Try to understand him as well. Men actually have feelings too. Ok, the most important ones being horniness, sleepiness and hunger but we do have others.
You didnt reply because he annoyed you, reverse that thinking, maybe that was his reason too.
Ezo