Haha, i got the similar problem as you but is when i am speaking to more people rather than one on one. You can read up on more books to practice your speaking skills and increase your vocal so you can speak freely when you have more vocab plus you are able to know more information from books which will increase your knowledge on more things and you have more things to share.
If what you are lacking is self confident i will suggest you increase your overall image to gain more confidence.