In life we always have a choice. You pushing your husband away is your choice, him having an affair is entirely his choice. My husband pushed me away a lot, too... I don't start having a fling with every dude that is available, right? It is a choice. And it is integrity - Integrity to me means not cheating even when others may not find out.
Having lived in different countries and met with people from all walks of life, I believe there are different "kinds" of marriage. There are those happy family kind with love, sex, and children - one most of us wants. Then there are people who are married in later part of their lives (or older men with younger wife), for companionship. Then there are those of convenience and financial security. Ultimately, it is what you want? The most important part about any marriage is to be HAPPY. Would staying married makes you happier? If not, that is a warning sign. Whatever "FAMILY" image your child grow up in, it is very important that his parents are happy. When you exudes positivity and happiness, your child sees it and becomes a strong and happy person.
You need to think about what makes you happy. If staying married means heartbreaks, tears, and insecurity in relationship, I do not think that is what you want your son to see. Also, by staying in a broken relationship also means allowing your child to grow up witnessing it, and might make him feel that "it is okay"... He might end up breaking someone's heart the same way yours is broken. The few men I knew that had affairs had fathers that had affairs (and mothers that endured it).