I94 did not change on I797A- F1 to H1B

Recently my H1b was approved, but my I94# mentioned on the approval notice is the same as it was when i was on F1. I did not travel outside United states.

Yes, it should NOT change as you have not traveled out of US. That is the correct info.
You will only get a new I-94 number, when you exit and re-enter. So, nothing to worry.