www.apache.org Defaced.

Yesterday, due to system-level misconfigurations, www.apache.org was defaced after a root-level breakin. Those responsible for finding the holes and the ASF have been in cordial contact, and the holes have been plugged. In the process of doing that, FTP and other services on www.apache.org have been stopped. A mirror of the defaced site can be found on the Attrition.org mirror site.

Brian Behlendorf sent the following to various Apache mailing lists:

Hi. We have been made aware (thanks to a very humorous banner ad for Microsoft Back Office on the front of www.apache.org!) that our particular configuration on www.apache.org of ftpd and bugzilla opened a security hole that allowed someone from the outside to get a shell account, and then get root. We have been in contact with those who found the hole, and have closed up the misconfigurations that allowed this.

It is important to note that this is *not* a hole in the Apache web server or related software products. I would encourage double-checking the PGP signatures of Apache releases for the immediate future.

However, I do not believe we are out of the woods yet. Bugzilla has not been thoroughly audited, and while I am not worried about ftpd, simply having another deamon that can write files to the web server whose purpose has been completely superceded by others suggests that taking it down for good is the right idea.

So I am taking down FTP - something that should have been done long ago. If there are FTP links on any of our pages (or on places like freshmeat) they should be change to HTTP. There are enough high-quality text-mode HTTP clients that there is no point to having it up, save for mirroring, and we allow rsync and cvsup for that. I will be contacting the mirror site admins list to communicate this.

Also, I have taken down all installations of bugzilla on apache.org until it can be audited. I will be performing a first pass tonight over it, but anyone else familiar with perl and willing to deal with rather ugly code is welcome to do so as well. I will set it back up once I'm comfortable there's been at least one reasonable pass over the whole codebase and any obvious holes have been plugged. This is only life-support though; I really don't think we should be using bugzilla once a suitable replacement is found.

Finally, I think it can be said that this compromise was mostly due to a lack of discipline on the part of those who had root and set up services without considering the ramifications of the way they were installed. I don't want to point fingers, since I'm probably at least as to blame as others, but I do feel that the policy of giving root access to a larger number of people than usual was probably a mistake. Along those lines, I've changed the root password and removed everyone from group wheel but myself - sorry to be fascist about this but I kinda feel like at the end of the day it's my responsibility. We'll come up with a strategy soon about granting sudo access to particular people for particular binaries so that I don't become a bottleneck again.

The details will soon be posted to bugtraq. Thanks.