It's just as well, many of the native birds we care about (whose population is generally also declining along with many of the native insects) need a diet that is an overwhelming majority of insects (Often over 90%, ending up numbering multiple thousand caterpillar larva consumed) in order to successfully raise young. Bird seed doesn't cut it, it's high fat and nowhere near enough protein. Protein as a percentage of dry weight in many insects can exceed that of beef.

In fact birdseed can become a sort of "trap", (much like milkweed being available at the wrong times of year for monarchs) where it tricks their biology into thinking it's a food rich area that's good for breeding, but what they need actually isn't there resulting in high mortality rate of the young they were trying to raise.

So having a bunch of garden plants getting shredded by native caterpillars is a good thing, one way or another.