0. Uninstall Intel Bluetooth Driver.
Bluetooth Audio Device does not work properly under Intel Bluetooth Driver when Windows Server is installed.
1. Install latest Toshiba Bluetooth Stack.
https://dynabook.com/assistpc/download/modify/soft/btswt/index_j.htm
2. Set the driver manually.
During installation, Setup would fail to detect Bluetooth Device.
Then, right click on Unknown Device → Update Driver → Browse my computer for driver software → Let me pick from a list of device drivers on my computer → Choose USB Bluetooth Driver-2 (V2.1+EDR)
3. All done.
Audio output will come through the paired Bluetooth device.