Don't panic, this is totally normal. Maybe you don't really produce enough milk and it is to exhausting for the baby - this could happen when you are under a lot of pressure. Maybe he just prefers the bottle :)
Also try to bump your milk and use formula in addition this should be a very balanced...