Sex without a condom, but using contraception...?

I have been on the pill for a little over 4 years for various period cramping problems, and not for the "birth control" portion of it. My husband and I began having sex after being married and I am still on the pill for the period cramping problems. When having sex we always use a condom, but it would be great once in awhile to not use a condom. I am always very concerned with becoming pregnant, and for some reason do not feel safe against that only taking the pill.

Some of my concerns/thoughts (and I know this is very personal)
1) What exactly is it going to feel like compared to having sex with a condom?
2) I'm not so sure about everything that is usually caught in the condom being inside of me :( For some reason it seems to be a bit uneasing to me.
3) We are married and if a pregnancy were to occur we would be more than happy. Although, if possible we would like to be a little more prepared, planned, etc. Is it usual to be completely afraid in not using a condom even though I am on the pill?

I very much appreciate your help. It's always nice to hear from a variety of people and what they have experienced.

1. Sex without a condom feels so much better. Sex with a condom feels like plastic to me. Since you guys are married you should go for it. I never, ever have sex with a condom since being in a exclusively monogamous relationship.

2. Considering that you are on the pill, you don't have to worry about that.

3. The main reason I got on the pill is so my bf and i could have sex without a condom. It's been a year and I haven't gotten pregnant yet, so don't worry. Back-up contraceptives are nice to do when you're on the pill but it's not necessary.

I would like to hear the same answer my dear,

I've only heard that it is safe without a condom when you are on the pill, but personally I would feel worried, I'm a young women so I am in no place ready for a child...

But I hope to hear some valid answers as well..
Good Luck . :)