Why does he do this??

i have started dating an older man.he has a failed marriage, and a daughter from it. sometimes when i am in he will bring friends around, but he will never leave the room even to put his daughter to bed when i am sitting with them.his friends will tell me, he is infatuated with me,tho'he denies it.if they are there he would rather i go home than spend time talking with them.what does this mean??