How to “cure” screen flickering in VirtualBox on Ubuntu with a Windows 10 guest and 3D support enabled
TL;DR: The goal of this article is to explain why VirtualBox machines on Linux host flicker if your host has an nVidia video card and the VM has 3D support enabled. Also, for 64 bit Linux, a patched library file is provided which can be installed “on top” of an existing VirtualBox installation.
The patches themselves are moved to a separate page: https://techblog.devlat.eu/virtualbox-patches/
The patched library should be installed in the
/usr/lib/virtualbox/ directory, as explained below.
Seems that VirtualBox has a strange problem. No, really. If you use an nVidia graphics card on your Ubuntu machine and wish to use a Windows (or, as far as I have read, basically any) virtual machine on VirtualBox for tasks that Ubuntu is not (yet) good enough and wish to enable the “experimental” 3D acceleration support, you will be unpleasantly surprised to find out that this 3D support is virtually unusable, flickering and blinking at will.
As it turns out, this problem was noticed several years ago in this forum. Basically, it’s a bug report with some folks offering a solution which involves changing a single line in the OpenGL rendering code. Unfortunately, the VirtualBox folks refused to patch this, as this breaks some functionality for an unnamed “paying customer” of theirs. This means that those affected by the problem have several options left:
a) Use a non-nVidia card, sending nVidia folks where Linus Torvalds once sent them (in this video).
b) Use the non-proprietary, free driver called nouveau with all the performance hit this implicates. Seriously, who needs 3D acceleration when it’s close to 0 (although I hope the situation will improve over time)?
c) Compile VirtualBox from source by applying the patch that can be found here.
I went through the hassle of doing that oxymoron thing of installing a VirtualBox VM on my VirtualBox to recompile a patched VirtualBox so I could patch my VirtualBox installation and use VirtualBox with 3D support and no flickering on an nVidia graphics card with the proprietary driver. 🙂
Basically, you need one file from the patched and recompiled VirtualBox, namely
Should you happen to run VirtualBox 5.1.8 (STRICTLY THAT VERSION) on an x64 CPU (that is, basically, 64-bit VirtualBox version 5.1.8) running on Ubuntu, you may download the patched library in its binary, ready-to-use form here.
Then, assuming you saved it in your
Downloads folder, execute these commands in terminal:
sudo mv /usr/lib/virtualbox/VBoxOGLrenderspu.so VBoxOGLrenderspu.so.original
cp VBoxOGLrenderspu.so /usr/lib/virtualbox/
Then just enable 3D acceleration on your virtual machine and launch it, enjoying flicker-free graphics.
Should you need me to recompile VirtualBox with this patch for any other VirtualBox version that might come out later, just let me know. Or, you might wish to learn it yourself (note: this requires some knowledge of command line and Linux/Debian package management) by reading the official building guide here.