Monday, December 23, 2013

Building a Linux Litecoin / Scrypt Miner

I've had a few people ask me about this, and while there are other walkthroughs of the process (I owe a serious debt of gratitude to Cryptobadger), many of those are a bit out of date and don't say much about the new R9 290/290X GPUs. So, here's my take on building a Linux system, as of December 24, 2013 -- Merry Christmas Eve!

Fundamentally, getting scrypt mining going under Linux requires a similar process to Windows:
  1. Install Linux (duh)
  2. Install AMD Radeon drivers
  3. Install cgminer or bfgminer (I have used cgminer 3.7.2)
  4. Configure cgminer and launch
  5. Create a startup script so that cgminer automatically launches (i.e. for occasions when the system crashes and restarts, or there's a power outage).
I might need to create a more detailed set of instructions, but if you're mostly familiar with Linux here's the short summary. Before you begin, set your PC (in the BIOS) to automatically turn on when power is restored. How you do this varies by motherboard, but usually press DELETE to get into the BIOS/UEFI screens and look in the Advanced settings. (Email me or post a comment with your specific motherboard model if you need help.) From there...

First, grab the Linux distro you want to use. I've used 64-bit Xubuntu 13.10 (Saucy Salamandar) as it's pretty simple, but others should work fine as well. (You're on your own for installing other distros!) Short summary: create a bootable device with the image (either a CD/DVD or a USB stick), then launch the installer and set things up. You can use a second USB stick as your OS device if you want, but they can really be slow so I recommend an inexpensive 60GB SSD or even a cheap 2.5"/3.5" HDD over a $5-$30 USB stick. (Seriously, you're spending $500 per GPU or whatever; go nuts and spend $50 on a decent boot device! You'll thank me in the long run, as every time you screw up and crash the OS and need to reboot can feel like an eternity...speaking from experience here.)

The setup process is pretty straightforward. About the only thing to note is that you'll want to automatically log in when the OS starts. (This is a total security no-no, but we're not worried about files on the PC -- we just want it to mine!) Once the OS is installed and you log in, you have a couple options. Either never worry about getting updates, or grab them all right now is my advice. (If something goes wrong, you want it to happen now rather than after everything is mining!) Now we're ready to install the AMD drivers and cgminer.

Open a terminal window. Now remove any old drivers:
sudo apt-get -y remove fglrx-updates fglrx-amdcccle-updates fglrx-updates-dev
(sudo will prompt you for the super user password the first time you run it in a terminal windows.)

Download the latest AMD Linux drivers and unpack to your user directory. Then from a terminal window:
sudo chmod +x ~/amd-catalyst*.run
sudo ~/amd-catalyst*.run
(Reboot)
sudo aticonfig --lsa
sudo rm -f /etc/x11/xorg.conf ~/.Xauthority
sudo aticonfig --adapter=all --initial
sudo reboot
(Wait for reboot again...)
At that point you should have all of the necessary AMD drivers installed. Now get cgminer (nothing after 3.8 supports GPUs, so get the last 3.7.2 build), then extract:
wget http://ck.kolivas.org/apps/cgminer/3.7/cgminer-3.7.2-x86_64-built.tar.bz2
tar jxvf cgminer-3.7.2-x86_64-built.tar.bz2
You're almost ready. Now set some items for future use so that you can mine properly. The sudo ln line is to create a link so cgminer knows where to find libudev. (This has been necessary on the latest 13.10 Xubuntu for me, at least.)
export DISPLAY=:0
export GPU_MAX_ALLOC_PERCENT=100
export GPU_USE_SYNC_OBJECTS=1
sudo ln -s /lib/x86_64-linux-gnu/libudev.so.0 /lib/x86_64-linux-gnu/libudev.so.1
exit
(Note that you may have to close and restart a new Terminal session to have the above stick, I think? In Windows, I know you need to open a new command prompt after setx; doesn't hurt to be safe.)

At this point, try running the following just to see if you can mine at all. (You can change to your own pool, or just test this and throw a few uLTC my way in the process):
export DISPLAY=:0
export GPU_MAX_ALLOC_PERCENT=100
export GPU_USE_SYNC_OBJECTS=1
cd ~/cgminer-3.7.2-x86_64-built
./cgminer --scrypt -I 20 --thread-concurrency 24000 -w 256 -g 1 --temp-target 85 --temp-overheat 95 --gpu-fan 40-95 --auto-fan --gpu-engine 875 --gpu-memclock 1425 --gpu-powertune 30 -o stratum01.hashco.ws:8888 -u trogdorjw73.tester -p tester
Assuming that all works okay, you should get around 725-850KHash/sec per GPU, depending on whether you're running R9 290 or R9 290X, as well as some "luck of the draw" on each card. You'll find that some GPUs will do better than others, and that's normal. From here you need to tune the clocks and thread-concurrency to get optimal performance. Note that if you want to exit cgminer, unless you can find a recompiled version to fix the problem, you do NOT want to press "Q" to quit -- it will crash your system and require a hard reset in my experience. Instead, close the terminal window manually to kill off cgminer.

For tuning, generally speaking go for max memclock first, then play with the core clock to find a good value -- often you'll find that if you push the GPU above 900MHz, it will actually be slower than at lower clocks, as AMD's hardware will throttle if it detects too much power or something going through the card(s). When you've got good RAM and GPU clocks, I tune the thread-concurrency in steps of 64. I haven't noticed a huge difference with changing TC, but you might get an extra 2% out of a GPU with the right value.

Of course, the tuning needs to be done for each GPU as well. One thing to try is once cgminer is running, press G (GPU Settings), the C (Change Settings), and from there you can tune the RAM clock and core clock in real-time, on each GPU. So start on GPU 0, and bump the RAM clock up in 25MHz increments from 1425MHz or so. You'll probably eventually cause the system to crash, so back the RAM clock down 25MHz and you're done on that card. Repeat for the other GPUs.

On the engine clocks, start probably closer to 800MHz and let cgminer run for maybe 5-10 minutes to stabilize. Look at the average hash rate for the GPU, and bump the core clock up 25MHz. Let things stabilize again for 5-10 minutes. If performance didn't go up, try the next step, and you'll probably see performance drop, which means you have reached the limits of that particular GPU for RAM and GPU clocks. Move on to the next GPU and repeat the process.

Tuning thread-concurrency unfortunately requires restarting cgminer each time, so I'd find good clocks first and then tune TC as a group. Start at 24000 or so, run cgminer for 5-10 minutes and note the averages for each card. Close out and bump TC up by 64 and run again. Repeat this for probably 10 iterations (20 at most) and you'll usually find a pattern where every five or so steps of 64 you'll see a peak and/or valley in hashing performance on each GPU.

Now just use the best result for each GPU in your cgminer.conf file and do a test run for a few hours. You may find that after the first 5-15 minutes performance will drop some and then stabilize. This is normal, and if you want you can see if you can tweak the settings a bit more to improve performance. (You might need to back off the RAM a bit more as well -- redlining any part of the GPU may end up in lowered performance over time, not to mention the potential for killing the hardware after six months or so.)

Another idea: having a large fan blow air across the GPUs may help out, though be careful with the potential for dust buildup if you do this. When everything is running well in cgminer, generate a conf file for future use. Press S (Settings) and then W (Write config file) and give your file a name (or use the default); I like to use something other than the default so that cgminer doesn't automatically load a config file when I don't expect it, so for my purposes I use cgminer-current.conf.

When everything is running smoothly, you'll also want to set Linux up to auto-launch cgminer on startup. Crytobadger has some scripts that work well enough, which I'll copy here with minor tweaking. First we make a script:
sudo nano mine_scrypt.sh
Paste the following into nano:
#!/bin/sh
export DISPLAY=:0
export GPU_MAX_ALLOC_PERCENT=100
export GPU_USE_SYNC_OBJECTS=1
cd /home/[LINUX USER NAME]/cgminer-3.7.2-x86_64-built
./cgminer --scrypt --auto-fan --failover-only --config cgminer-current.conf
Save and exit (CTRL+O, CTRL+X), then make the script executable:
sudo chmod +x mine_scrypt.sh
If you open a terminal and type "./mine_scrypt.sh" you'll start mining. (Note you can press [TAB] to auto-complete file and directory names, so typing "./mine[tab]" is usually sufficient.) First we need to make sure we have "screen" installed:
sudo apt-get install screen
Now we need to create a second script to start mining when Linux boots:
sudo nano start_mining.sh
Paste the following into nano:
#!/bin/bash
DEFAULT_DELAY=0
if [ "x$1" = "x" -o "x$1" = "xnone" ]; then
   DELAY=$DEFAULT_DELAY
else
   DELAY=$1
fi
sleep $DELAY
su [YOUR LINUX USERNAME] -c "screen -dmS cgm /home/[YOUR LINUX USERNAME]/mine_scrypt.sh"
Save and exit again (CTRL+O, CTRL+X), and make the script (scrypt? Hahaha) executable:
sudo chmod +x start_mining.sh
Now we edit the system config file to automatically call our script:
sudo nano /etc/rc.local
Add the following line just above the "exit 0" line; the 15 is a delay in seconds, so you can use more or less as you see fit:
/home/[YOUR LINUX USERNAME]/start_mining.sh 15 &
Save and exit (CTRL+O, CTRL+X) and you're done. The only catch is that now when you boot Linux, cgminer is running in the background and you can't see it. To bring it to the foreground, open a terminal window and type:
screen -x cgm
You can also make an alias (command shortcut) to open the cgminer window by editing your own .bashrc file ("sudo nano .bashrc") and add the following near the bottom:
alias cgm='screen -x cgm'
When you close the window showing cgminer now, however, cgminer will continue running. To actually kill and restart cgminer, type "ps -A" find the "cgminer" process, and then type "kill -9 [Process ID]". Now you can just type ./mine_scrypt.sh to restart cgminer if you want (e.g. after you tweak the config file).

Is Linux any faster at mining than Windows? I haven't noticed any major difference, so the main draw is that Linux is free. Personally, I continue to use Windows on my PCs, because I'm more familiar with the programs and things like MSI Afterburner are handy. Besides, again we're talking about $89 for Windows 7 64-bit (or $96 for Windows 8.1 64-bit) on a system that has $2000 worth of other hardware. It's less than 5% of the total cost and increases the resale value if you ever decide to stop mining. Anyway, YMMV.

LTC tips: LfCLyykrNFftzpdWejR73hf478ZtBzQ9jE
BTC tips: 1GGJUb1gFpydygpeKzd6oFoShLRUSyThV7

11 comments:

  1. Hi Jarred,

    to kill cgminer one could use "ps -A|grep cgminer" to avoid searching for the cgminer thread. In the end a "killall -9 cgminer" would be easier. The "-9" ensures a forced kill to prevent cgminer from crashing the whole system.

    ReplyDelete
    Replies
    1. Yes, don't forget the -9 or you crash! Hahaha. I need to try the recompiled binaries some have put together.

      Delete
  2. Great article! I've been looking for an "up to date" how-to on linux... Something a lot of us could probably figure out given time (and Google), but nice to have it all in one place for sure! Totally agree on the SSD vs USB, I can't even count how many times I've rebooted Windows when I was trying to get all 6 of my cards working with different PCI-e port configurations

    ReplyDelete
    Replies
    1. I'll be honest: this post was as much for me as for others. I wanted a single place I could go for a the full list of steps. :-)

      Delete
  3. Thanks sharing your knowledge!

    Have you ever tried the auto TC process outlined in the cgminer "scrypt readme". The one that uses .bin files. Does that work or is it better to tune it by hand?

    Hint: it's in the section titled: "TUNING AN AMD RADEON 7970"

    ReplyDelete
    Replies
    1. cgminer basically stopped worrying about GPUs several months back, so it never really has been tuned to work optimally on R9 290 hardware. The SCRYPT-Readme.txt file is woefully out of date in most areas, and the part about trying to find the highest TC value you can start at is mostly useless on R9 290 hardware -- I've started cgminer with TC as high as 56000 or so, and having auto-tested everything from 20K to 60K in 64 point increments, it seems like there's usually a bunch of settings that can work equally well.

      Also, 1900 MHz on the memory for an HD 7970 is nuts -- most cards won't do more than 1650 reliably. Likewise, overvolting is NOT the only thing that can kill your GPU, and considering many 7970 cards come with 1.250V stock you're already pushing them really hard.

      In short, Con Kolivas did some great work with cgminer, but he is after all only one person. Maybe AMD sent him a cherry 7970, but his advice on tuning performance on 7970 is full of potential problems. It's why I created the batch files. :-)

      Delete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Hi,
    13.10 Xubuntu didnt work for me, since AMD driver wont support the new Kernel Ubuntu is using. How did you manage to install AMD driver into your setup?
    Another question: R9 290 or R9 290x, which is faster? I have 2 R9 290x with the following output:
    cgminer version 3.7.2a - Started: [2014-01-07 19:38:19]
    --------->>>>>>>>---------------------------------------------------------------
    (5s):1.860M (avg):1.859Mh/s | A:37264 R:384 HW:0 WU:1731.6/m
    ST: 2 SS: 0 NB: 4 LW: 143 GF: 0 RF: 0
    Connected to ftc.give-me-coins.com diff 640 with stratum as user xxx
    Block: 5cc8d4d7... Diff:200 Started: [19:51:27] Best share: 133K
    ---------------------------------------------------->>>>>>>>--------------------
    [P]ool management [G]PU management [S]ettings [D]isplay options [Q]uit
    GPU 0: 59.0C | 930.6K/933.4Kh/s | R:0.0% HW:0 WU: 882.7/m T:1 I:20
    GPU 1: 62.0C | 931.4K/933.4Kh/s | R:1.9% HW:0 WU: 868.0/m T:1 I:20
    ---------------------------------------------------------------->>>>>>>>--------

    is 930Kh/s good?

    Brgds
    C.

    ReplyDelete
    Replies
    1. I just fired up a 4x R9 290 rig today with Ubuntu 13.10, the OS is a bit beefy compared to Xubuntu but I picked up 16gig USB sticks for under $10 so I have plenty of space for everything the rig needs.

      Everything I have read says that the R9 290's can push upwards of 900Kh/s and the R9 290X's upwards of maybe 950. Kh/s. Assuming optimal turning on an optimal R9 290/290x.

      1 R9 290 x4 rig built ... 1 more to build before the end of February!

      Delete
  6. Hi. Thank you for this guide. It works but I am not sure why.

    Why don't I need to install the AMD APP SDK and ADL SDK? I was let to think that they were important steps that were integral. Are they already included in the driver and cgminer?




    ReplyDelete
  7. Man oh man, I sure wish I had built a bitcoin or litecoin miner back when you posted this. Or, well, about 6 months before you posted this. I'm pretty sure this is right around when the bubble popped, haha.

    ReplyDelete