Search This Blog

Tuesday, July 1, 2014

First sysadmin impressions on RHEL 7

The 10th of June 2014 was a special day for the Linux world, as Redhat released the much anticipated version 7 of its Enterprise operating system product RHEL. Code named 'Maipo' and having gone through a Beta (11/12/2013) and Release Candidate RC (23/04/2014) stage, RHEL 7 is now available (GA) and is based on Fedora 19 and the upstream kernel 3.10.

Following the blog tradition started with a 'first impression' article on RHEL 6 some years ago, I will do the same here and present a number of goodies that will keep the sysadmins/devops folks busy, in the process of deploying RHEL 7 in their production environments.

Rather than replaying in detail the excellent info of the RHEL 7 Release notes as well as the nice roadmap presentation of the 2014 RedHat Summit, I will just summarize the important changes that it brings to the daily work of sysadmin/devops teams accompanied by some hands-on videos on the concepts.

The very first thing you will notice is a 'Fedorazation' of the Graphical Installation environment. Yes, those of you that are not familiar with the new graphical Anaconda environment of the latest Fedora releases, you will discover that the old RHEL 5/6 serialized wizard menus are gone in favour of a central screen where you click to setup various aspects of the installation AND then you launch the install. You can find  a video below that demonstrates a basic RHEL 7 installation on a VM environment.

Another major change you will feel from Day 1 is the obligatory-ish interaction with systemd. Faster boot times with parallel service startup and increased manageability have their price: The days of init scripts and the 'services' command are gone-ish for RHEL. I am using the '-ish' bit because although Redhat chose to maintain the traditional concepts of runlevels for backwards compatibility (read: to prevent most non systemd friendly sysadmins to start screaming), this will certainly not be the case for the near future of RHEL 7 minor releases. Thus, now is a good time to start thinking in terms of units and targets, as opposed to init scripts and runlevels. Below you will find a video where I demonstrate the concept of interacting with systemd to start and stop services, or change the default runlevel, erhh, sorry, target of your system. Once you get used to 'systemctl', your new buddy, things will not look so bad/alien.


You should get used to the idea of using Linux Control Groups (cgroups). The cgroup concept is not a new one, however now is the time to get used to the new tools for administering cgroups, as the technology is one of the conceptual blocks that RHEL 7 brings in the arena of Linux Containers (see latter paragraphs). To help you, I have made a video that demonstrates how various systemd commands can be used to administer cgroups.

In the filesystem arena, you have some notable supported volume capacity expansion. In particular, XFS which is now your default filesystem (that's right, even / on an automatically partitioned install is on XFS by default) can support volumes of up to 500 Tbytes (in RHEL 6 the limit was 100 Tb). ext4 is still an option and supported in volumes no greater than 50 Tbytes (RHEL 6 had them limited to 16 Tb per volume). My advice is to really not use ext4 again on your migrated RHEL 7 systems. As I wrote four years ago in the RHEL 6 first impressions article, XFS was the way to go in terms of performance and scalability. Today, XFS is mature enough to trust for production and can be tailored to medium/large size volumes. I just do not see ext4 fitting somewhere anymore.

I have yet to produce some comparative performance figures of XFS running on the same hardware for RHEL 6 and 7. Stay tuned for this, as it takes time and effort to do some meaningful comparison.

Staying on the filesystem arena, Btrfs is offered as a tech preview on RHEL 7, but I am not happy at all with its current state (kernel  3.10.0-123.4.2.el7.x86_64 #1 SMP Thu Jun 5 21:43:43 EDT 2014). During some iozone benchmarking, btrfs broke down and that's quite disappointing. If one considers the fact that btrfs represents really the ZFS commercial Unix arena equivalent of the Linux world, even a tech preview should have been more stable. Redhat folks, I am looking forward to a more stable btrfs tech preview, please work on that task.

For those of you that are development oriented, some major version components include:

  • GNU bash, version 4.2.45(1)-release
  • gcc version 4.8.2 20140120 (Red Hat 4.8.2-16)
  • Perl  is on version 5.16.3 built for x86_64-linux-thread-multi
  • Python is on version 2.7.5

This should really update some long outdated components of RHEL 6 (especially on the gcc side of things) that were seriously hindering software development without some effort to install separate libraries, compilers, etc.

Finally, for the devops oriented folks, one of the greatest technologies that RHEL 7 is introducing is that of containers. In essence, a container is a way to package your tech apps using a lightweight environment that provides resource isolation. This cannot only seriously save some memory and CPU cycles (in comparison to hosting apps in virtualized OSes), but it can also please your developers by introducing Docker Image based containers, a technology that allows you to speed up portable application deployment. I have made a video that demonstrates the basics of using Docker (see below) and I hope this will help you orient yourselves around the technology.

Whether Docker containers will eventually replace your hypervisors is hard to say. Containers do not provide (yet) features such as redundancy and live workload migration, two business important features offered by most virtualization technologies today. Nevertheless, application serving and deployment via Docker containers is more efficient than throwing virtualized operating system images. I shall place some concrete numbers behind that claim in order to convince you to start looking at the technology.

First of all, I shall explain how I obtained those figures. I performed the tests on two identical servers ( Dell PE 1950s) with the same amount of RAM (16 Gigs), the same CPUs ( 8 cores Intel(R) Xeon(R) CPU E5345  @ 2.33GHz cores) and with identical disk drives and I/O controllers. One of these servers was running a standard KVM environment on RHEL 7. The other was running a Docker container engine on RHEL 7.

The next steps were to install:
  • An Apache server on the RHEL 7 KVM hosted VM (the served VM is on Fedora 20)
  • Docker pull and run the Fedora/Apache image from the Docker repository on the second server, to approximate the same environment as the VM on the Linux container. I also had to run the container by doing a:
    docker run -p 8080:80 -i -t --name=thirdcon fedora /bin/bash
         in order to give it a port on a network (testing on the local server and
         bypassing the network latency would not yield a reliable measurement)

At that point, I had two web servers running, one in a VM and one in a Docker container. I configured both Apache s to serve a simple text index.html file (no images, graphics).

I then employed the ab Apache benchmarking tool from a third independent host and for each server URL, I fired a benchmarking load with different levels of concurrency like this:

ab -c 2 -n 100000 http://serverURL/index.html
ab -c 4 -n 100000 http://serverURL/index.html
ab -c 8 -n 100000 http://serverURL/index.html

representing different numbers of concurrent requests, up to the number of cores of the VM and the container. The results are summarized in the graphs below and they are quite revealing.

The first graph shows the maximum number of concurrent requests per second achieved in each concurrency level. Docker clearly wins over KVM with just over 8000 requests per second. You should also note that the more we increase the concurrency level, the greater the difference in the result between the two technologies.

The second graph plots the time-to-completion in seconds for each benchmark. Again, Docker is faster by a margin of several seconds. That might seem small, but it really is not. A few seconds here and there, in loaded servers running larger workloads concurrently means a great deal.

This concludes the original first impression assessment of RHEL 7. I hope this contributed towards convincing you to give it a go.

That's all for now!