I've read that in some online newspapers that are pretty politically far right, so they may not be exactly objective, but I've found some other web pages that make the claim so it looks a bit more plausible.
Here are some of the places that support the claim:
http://hungarytoday.hu/news/mirror-migration-germans-moving-hungary-fear-migrants-20317
Well, the last one doesn't specifically talk about families rather about one family, but it helps to support the claim anyway.
There are also some other web sites (not an insane amount but some) that support the claim.
How true is the claim really? What I want to learn from asking "how true it is" is a good estimation of the number of families that have moved because of that. I mean if 3 families have moved it doesn't deserve much consideration, but if there are e.g 200, it deserves some at least.