I stayed in Excelsior in Causeway Bay with my 2 year old - just her and me, and it was extremely conveniently located. MTR nearby, shopping centres nearby, close taxi ride to the airport express train, local eateries near by for breakfast. But the rooms are small so I think 4 pax in a room might be a squeeze.
Another hotel I've stayed in is Hotel Nikko Hong Kong in Tsim Tsa Tsui East. Not as convenient as Excelsior in terms of location but the room size makes up for it so 4 pax will be comfortable in one room. The hotel runs a free bus shuttle to various locations in Tsim Tsa Tsui and Star Ferry - bus runs in a loop every half hour. If you miss that, then a taxi to Star Ferry is about 5 minutes away and between HK$18 and HK$22. I get to HK via the Star Ferry as my daughter likes to ride the ferry across. But the hotel shuttle bus will drop you off a the MTR station too (or you can walk there - I've walked to the Star Ferry with her in a stroller too). It was also just my daughter and me at this hotel so the locations, etc is totally manageable. The free airport express bus shuttle also stops at this hotel which was convenient for us to get to the airport.