I think there is no loss for you to withdraw from the school as the school does not seem to be very good from what you describe. However, the school has no right to ask you to withdraw. They should instead work with you to see how they can work with your boy.