TTC with One Testicle

My fiance and I started TTC at the beginning of this year, 5 months in and no luck we decided to wait for me to see the doctor, but im worried the problem isnt me!?! My fiance 3 years ago had testicular cancer and had to remove his left testicle. We havent gotten him tested yet because he does not have medical insurance through his work. My doctor advised me to start BBT and a ovulation test kit, at a 3 month rate of trying she said we are bound to get pregnant. It is our second month and no luck! I was almost certain this month we had conceived but it was unsuccessful, being late 2 days on AF and high bbt after ovulation made us feel we achieved pregnancy, but i started my period today :-,( Has anyone known anyone who had difficult conceiving, when partner only has one testicle? Any tips on how to improve our chances? Any advice would be helpful, we are starting to feel there is no hope!