Don't feel guilty. It is your life. You decide what you want.
When I decided to only have 1 child. Everyone fired like machine gun. The usual nonsense. But I just snapped back, "if you guys want 1 more, go give birth yourself." I won't get pregnant again, even if I were to want another child, I...