H1B Stamping required for COS?

I am currenlty on F1 visa. I applied for H1B in April. Traveled to India on 10th May.

H1B got approved last week and I have my I-797A with me. My old student visa is still active, so I can enter the United States after my vacation.

Should I come back to India for visa stamping in September?

Thanks in advance.

If you left while the H1-B was pending but before it got approved, you abandoned the COS, so you will either need to get an H1-B visa stamp and enter on that to change to H1-B, or if you are in the US on F1 status you can ask your employer to file another change of status (should be easy since you already have the H1-B).