Does H4 visa stamping required before applying for extension?

I was in USA on F1 visa when I got my I-797 approved three years ago for the H4 visa. I did not travel out of USA in past three years (therefore, I do not have H4 stamping on my passport). Now, my husband is going to get an extension/renewal for his H1 (he already has H1 stamp) and we are wondering if I need to have H4 stamp done on my passport before applying for my visa extension/renewal. Please advise.

If you are inside then just H-4 extension is fine along w/ his H-1 extension. You will need to get H-4 visa stamp anytime you leave US and want to return on H-4.