By default this is very annoying, as the window keeps popping up as soon as the cursor hovers over the top, here is how you can customise the hover time just for the taskbar:

1. Click on the Start Button and type in Regedit and hit Enter.
2. Navigate through HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced.
3. Right click on Advanced and select New DWORD and name the new value "ExtendedUIHoverTime".
4. Right click on ExtendedUIHoverTime and select Modify.
5. Switch to Decimal base and enter in your new number.  The default is 400 milliseconds. So set it to something like 9000, which is 9 seconds delay, up to you.
6. Once changed click OK and restart the explorer process or restart your computer.