Hulls of boats generally don't just go missing or left behind. And, the fiberglass doesn't cause as much damage as high surface area fishing nets. I remember reading that the paint can be an issue though, as there are added particles to get the aesthetic glimmer that will get into the water over time
Anti-fouling paints applied to hulls contain organometallic chemicals that are, by design, toxic to marine life. This is to prevent algae and molusk to attach to the hull, slowing the boat over time and requiring extensive cleanup. You can imagine the effect these paints have when they chip off.
