TTC and becoming VERY FRUSTRATED!!!!!

ok ladies here we go! i am 24 and my husband is 27 so you would think that ttc would be soooo easy for us right? well its not back in march 2009 my husband found out he had a tumor in his petuitary gland(brain) so he had surgery and had it removed. After the surgery we found out that the petuitary gland controls your hormones and everything so after have some blood work done we found out his sperm count and testosterone were very low :( so he has been on medicine since then. We thought everything was back to normal b/c we were doing th BD but every month we werent getting pregnant. recently we went to the doctor to find his count was still pretty low so he has been on injections. to me it seems like things are improving drastically, but still every month is another upset. we both are begining to become very frustrated and at times i want to just give up. and it really doesnt help that all of my friends and cousins my age are working on baby #2 and here i am struggling with #1. Im trying not to think too hard when ovulation time comes around but i cant help it. i dont want to push too hard because that takes all the fun out of ttc. Maybe there is something we are not doing right.......I NEED HELP!!!!!!