Is he just confused because it's his first time....?

He's in love with you! If you don't feel the same or if you are not interested on having something more with him, you need to walk away. Being his first time its not a problem, he's doing all this so you can fall for him. He might have accepted to be your f*** buddy just to have the opportunity to win you over. If its not your first time you should have recognize that its time to draw out.