Amazon EC2 ’steals’ from you

As we implement more systems in the EC2 architecture, we are noticing a not so insignificant amount cpu cycles ’stolen’. What is a ’steal’ time? It is CPU time that is taken by the Xen hypervisor for something else other than your processes- from what I have read, other people’s processes. What we need to understand is how this affects performance. Does it truly matter? We have one virtual system that consistently has steal time of between 6-12%. That would mean that 6-12% of the CPU time we pay for is being used for instances other than our own. We will have to research this more to see what the true impact is on our systems, and if there is a way to mitigate it.

Busy holidays, and back to blogging

It’s been a long, busy holiday- and now time to resume blogging. My latest interest is testing our mysql servers using mysqlslap. While a great tool, it is unfortunately only distributed with MySQL 5.1.4 and above. However, many of our servers are in the 5.0 release, with some in the 4.x release. What I did was grab the latest 5.1.x version of mysql, compile it statically and test it out on a MYSQL 5.0.45 version, and it worked just fine. I have yet to test it on another machine, or against a version other than 5.0.45. I’ll update when I do.

Here are the steps I took:

1) wget “http://dev.mysql.com/get/Downloads/MySQL-5.1/mysql-5.1.30.tar.gz/from/http://mysql.llarian.net/”

2) tar -xf mysql-5.1.30.tar.gz&&cd mysql-5.1.30

3) ./configure –without-server –disable-shared

4) make&&cp -i ./client/mysqlslap /usr/local/bin/

Finding space hogs

I know this is a common post out there, as a google search will bring this up multiple times, but here it is anyway. A great alias to set up on your system to aid in finding space hogs in a given filesystem or directory is as follows:

alias ducks=’du -cks * |sort -rn |head -11′

Add the aliases to whichever file you keep them in (.bashrc, .aliases, etc)  and then run ‘ducks’ for output similar to this: Continue reading “Finding space hogs”

Mysql table opens, caching and iowait times

One issue that seems to be very common when running a MySQL server is high iowait. As we have delved into our servers, one of the main causes we have found is that our table_cache was set too low, causing an ever increasing number of table opens

One issue that seems to be very common when running a MySQL server is high iowait. As we have delved into our servers, one of the main causes we have found is that our table_cache was set too low, causing an ever increasing number of table opens. As we increased the table_cache, and watched for higher hit percentages, we were stumped regarding how many table opens in a given time period were acceptable. A number which I use as a guideline was given to me during a MySQL training course this year. We use a guideline of no more than 10 table opens per hour. More than that- and we need to increase table_cache.

Building a kernel RPM from source

While it is reasonably easy to build a kernel RPM on a Redhat or CentOS system using the kernel src rpm, another way is to do a standard kernel build (download your kernel from kernel.org), configure as usual. However, instead of just doing a standard “make” or “make bzImage”, do “make rpm“, and voila! You’ll be left with your kernel rpm in /usr/src/redhat/RPMS/{arch}, and your very own src rpm in /usr/src/redhat/SRPMS. Easy, right?