As suggestion of the whitepapper I tried to move the GPU in different slots but the issue was always the same, I don't have onboard VGA just single Quadro 4000.
This is the problem here, by default the ESXi Console will grab the first adapter to display its output and since its now owned by the ESXi Console the driver will not be able to load.
So to get this working you need to add another adapter to have the console or use the workaround that is listed in the whitepaper to remove the ownership of the adapter from the console. (The last workaround you will need to do at every reboot and then manually load the driver again.)
// Linjo
