Gridcoin GPU mining (6): Obtaining the maximum performance out of your GPUs

Welcome to the sixth installment of Gridcoin GPU mining series, continuing our exploration into the world of computational science, done through BOINC network and rewarded through Gridcoin - a cryptocurrency which is rewarding BOINC computations, on top of Proof-of-Stake.

Although BOINC is a volunteer effort, it has a very competitive community. Every BOINC project maintains a list of Top Participants, Top Teams and Top Hosts. Many BOINC crunchers are carefully studying those lists, comparing performance of their hardware to the performance of other crunchers. Of course, with Gridcoin there is also a monetary incentive to maximize the performance of your hardware: more successfully completed BOINC tasks also mean a larger Gridcoin income. So, I am going to provide all obvious and less obvious hints on how to squeeze every bit of performance out of your GPUs, maximizing your BOINC output and, in turn, your Gridcoin earnings. I am currently mostly involved with MilkyWay@home BOINC project and some hints will be relevant only for that particular project, but other hints will improve your GPU performance across all BOINC GPU applications (and maybe even your Proof-of-Work gigahashes, if that's your thing).

Laurent Domisse, top SETI@home user, has 107 machines with numerous high-end GPUs crunching for science. Yes, the competition is tough.

1. Choose your GPU and BOINC project carefully

I've already written about this in one of my previous articles, but it's worth repeating here. Unlike simple hashing which is mostly dealing with integer numbers, BOINC (and computational science in general) is dealing with decimal numbers and floating point operations. The majority of BOINC projects use FP32 computations (so called single-precision), but MilkyWay@home (and some other BOINC projects) require FP64 or double-precision. So yes, your shiny new GTX 1080 Ti is able to achieve 10.8 TFLOPS in FP32, but only 0.34 TFLOPS in FP64, so it's a waste of resources to use it for MilkyWay@home and it will perform terribly there. Use it for FP32 BOINC projects instead (there are plenty of them). In fact, if you check MilkyWay's Top hosts list, you will see that it's populated mostly with AMD 7970s and R9 280X GPUs which are pretty much outdated, but still renowned for their high FP64 performance at affordable prices. So choose your GPUs and BOINC projects carefully, if you want maximum performance. Don't bring a knife to a gunfight.

Nvidia GeForce GTX 1080 Ti. So awesome. Except in FP64.

2. Free up some CPU cores

Typical newbie mistake: load all CPU and GPU cores with BOINC tasks. CPU utilization at 100% all the time, that's surely the maximum performance I can get out of my BOINC machine, right? Well, not quite. With your CPU busy all the time, GPU tasks will get stalled often, severely reducing your overall BOINC output. To put it simply, there are no pure GPU workloads - the CPU is often needed to provide some 'assistance' with BOINC computations. And if your CPU is 100% busy, then your GPU will have to wait for its turn. And that's bad, you don't want your GPU tasks waiting and stalling, you want them running at full throttle all the time. So free up some CPU cores now. There is an option in your BOINC Manager just for that:

Here it is, under Options->Computing Preferences, use at most 75% of the cores (or any other number you choose).

How many cores to free up? How can you be sure that you are doing this right? Open your Task Manager and monitor your CPU utilization. Are there any flat lines at 100%? Free up more cores. This is the CPU utilization history for my machine (ID 439806) and you will probably want your chart to look approximately the same (few spikes are OK). 

Yes, following these instructions, your CPU output will be certainly reduced in the end. But this article is about GPU performance, first and foremost. And GPU tasks bring much more BOINC credits. You want your machine to be at the top and to strike stunning disbelief into the hearts of other BOINC crunchers? Then free up some CPU cores and let your GPU tasks fly. When you hit record numbers, no one will say "but one of your CPU cores is underutilized".

3. Run multiple BOINC tasks per GPU

Modern GPUs are becoming extremely powerful. GTX 1080 Ti, mentioned before, is equipped with 11 GB of memory and 10.8 TFLOPS of FP32 computing power (typical performance of fastest supercomputers only 15-16 years ago). To put it simply, many BOINC tasks aren't that demanding yet, to utilize efficiently such a large computing resource. However, there is a solution: run multiple GPU tasks simultaneously. Your average runtimes will increase of course, but so will your overall BOINC output.

This is especially important for short, non-memory intensive tasks, which are usually completed in a minute or less (MilkyWay@home is notorious for its short running tasks). With such a workload, your GPU has to switch tasks every minute, finishing a previous task and loading new one, with some idling (which is inevitable in such transitions) in-between. But if multiple tasks are running in concurrence, your GPU will never go fully idle, since concurrent tasks are almost never finished at the same time and a balanced load is created between the tasks which are being finished (and idling) and other tasks which are running at full throttle.

 I have 4 GPUs each running 4 MilkyWay@home tasks (16 tasks in total). This is how they look in my BOINC Manager - no two progress bars are the same. A mix of ending, running and starting tasks ensures that no GPU ever goes idle.

So the question is: how many tasks should I run on a single GPU, to obtain maximum BOINC output? Unfortunately, there is no universal answer here, since it depends on a lot of factors. Generally, for very long and demanding tasks (like for example PrimeGrid Genefer World Record workunits, which run for 72 hours or more) one task per GPU is usually more than enough. For short tasks (like MilkyWay@home) optimum is somewhere between 3 and 8 per GPU, I think. Some experimenting is needed to find out what works best for your hardware. If your system becomes slow and unresponsive, decrease the number of running tasks.

Artefacts? Corrupted frames? Video stutter? General unresponsiveness? Many BOINC tasks running simultaneously can hammer the GPU pretty hard. Revert back to one task per GPU and things will normalize again (i.e. there can be no permanent damage).

By default, BOINC is configured to run only one task per GPU, but that can be changed by creating and editing some XML configuration files. Only Notepad is required and it's a simple process - you need to create a file named app_config.xml in the project's directory. BOINC Manager will automatically detect it and load it upon start. 

An example of app_config.xml for MilkyWay@home can be found on Gridcoin Cryptocointalk. I wanted to paste it here, but it's messing up the editor.

More details about BOINC XML configuration can be found here.

No GPUs were harmed in the process

By now, we have more or less covered "conventional" stuff (meaning that your GPU warranty is still 100% valid). In the next article, we are going to look into increasing your GPU performance even further, no holds barred: overclocking, overvolting, modding the BIOS, voiding the warranty and turning your GPU into a fiery furnace, bent on the absolute maximum BOINC performance possible, power consumption and heat output be damned. You have been warned :)