Is H4 Extension Required with valid I-94 ? H4 visa stamp expired in passport

i need you valuable suggestion on H4 extension. we traveled back to USA recently and visa stamped until 31-Dec-2020 only as per my old I797. before go to india applied H1 extension alone(not applied H4 extension for dependents ) and its got approved until 31-Dec-2021 one day before travel. So we traveled with old I797 only and showed old I797 along with new receipt number at POE. officer checked with receipt number in there system and as per latest 797A given I94 until 31-Dec-2021 for me and dependents also and said not required to apply extension. So for dependents visa stamped until 31-Dec-2020 and I 94 valid next year end. do i need to apply H4 extension for dependents or not required? (they don’t have valid 797 after 31-dec 2020)

If your spouse has valid H4 visa stamp on her passport until Dec 2021, then you may not need to. Otherwise, you would very likely need to apply for H4 extension.
The simple reason is that, you need some sort of document for Status in the country. Just I-94 is not sufficient. One option is to have US Visa stamped until that date or second option is to have H4 extension approval until that date.