Anatomy of a blog hack

So, last weekend I found out that my blogs had been hacked.

Actually, it wasn’t just my blogs, nothing personal involved or anything like that: the shared server space my sites were hosted on was compromised, and a good number of other sites and files were hacked as well. Based on what I can piece together, here’s what happened:

There were a number of sites on this hosting space that were running out-of-date versions of WordPress, and some that also had various other PHP code installed (NetOffice, Gallery 2, a few others). Any software that is outdated is potentially at risk to known exploits, but more worryingly, I found an old bit of PHP code on the server that was set up to run arbitrary PHP code for (I presume) some back-end admin processing, and ultimately I think this was what had been exploited.

And until I had found and killed this code, the exploit happened at least 3 times even as I was cleaning up the server.

The exploit itself, once I knew what to look for, was fairly simple:

  • In PHP files that were writable to the Apache webserver process, the code was altered so that any line containing an opening PHP tag (which tells the server to start executing the code after it as dynamic PHP until the closing tag is reached) looked something like this:
    From <?php .....
    To: <?php     eval(base64_decode('malicious code encoded here')); .........
  • When I copied this code to a sandboxed PHP environment and decoded it, it contained fairly simple instructions:
    • If the visitor to the site was coming from a Referrer—in other words, if they had clicked on a link from another site like Google search results, Facebook, someone else’s blog—they were redirected instead to a completely different site that presumably contained spam, or malware, or whatever.
    • If the visitor was coming to the site directly—they had typed the URL directly into the browser’s Location bar, or clicked on a bookmark—then they were passed on through to the site.
Because I normally type in URLs to my blogs directly, or click the “recently visited” link in Chrome’s list, I didn’t see the exploit at first. But as I was writing a blog post on The Brew Site on Friday the 20th, I was searching out a link to a previous blog post (gotta love Google for that) and when I clicked that link to pull up the earlier post, I was redirected to some site in Poland (or at least, with a Polish country code for the top-level domain).

Fortunately, I don’t believe this hack was in place for long, since I often search out links in this manner and would have noticed sooner: Sometime in the wee hours of the morning of January 19th was when the files were first modified is the earliest I can determine.

It took me a bit of time to figure out the exploit (at first I was thinking it was the Google 302 hijacking exploit), but once I did I was cleaning up files on my blogs by Saturday morning. I hadn’t yet had the chance to address the (many) other files and old sites on the server hosting space, so unfortunately my blogs got re-infected at least once more before I was able to kill the old files and update others. Most of my weekend (and part of the following week) was spent updating, fixing permissions, cleaning, and deleting files and sites.

For reference, a handy pattern for detecting this code in grep is:

grep -R -l 'eval(base64_decode(' *

(This should always work because you should never have similar PHP running in your legitimate code…)

Now, I keep my WordPress blog software (and installed plugins) up-to-date pretty religiously, and I try to keep permissions set appropriately. But a good number of files in each blog were infected even so—how? It turns out, even though a fair number of the core files that were originally installed (manually) had the correct Unix group (“<account>:users”) and permissions of 644 (rw- r– r–) and were untouched, I was also making liberal use of WordPress’s built-in auto-updating feature, along with automatic plugin installation, and at some point the files that WordPress were updating got set to the “nobody:users” group—the Apache webserver process. It was these files that were exploitable to the “nobody” Apache process that was being exploited by the other code on the server. (Along with the few files I had set to group-writeable as well.)

So, lesson learned. I’ve battened down the hatches, fixed the permissions on all the files in my sites, and have decided to forgo WordPress’s auto-installing and update features for now for good measure. And, I’ve finished up a (long overdue) move of my blogs to a new webhost with none of the legacy code possibilities that were extant on the original server. (Nothing against the original web hosting provider, I just needed a clean break with an affordable price.)

Of course, you all let me know if you still run into any problems, okay?