he thing that solved it for me was to change the Team settings in BACS from Smart Loadbalancing, which seems to be the default when a Team is created to hold vlans, to Generic Trunking.
Before the change the ARP entries in my router was all from the MAC address of the physical interface.
After, they were replaced with the MAC addresses of the virtual nics in the vm-servers.
And everything just started to work