Vagrant on Fedora with libvirt (reprise)

Vagrant has become the de facto tool for devops. Faster iterations, clean environments, and less overhead. This isn’t an article about why you should use Vagrant. This is an article about how to get up and running with Vagrant on Fedora with libvirt easily!

Background:

This article is an update of my original Vagrant on Fedora with libvirt article. There is still lots of good information in that article, but this one should be easier to follow and uses updated versions of Vagrant and vagrant-libvirt.

Why vagrant-libvirt?

Vagrant ships by default with support for virtualbox. This makes sense as a default since it is available on Windows, Mac, and GNU/Linux. Real hackers use GNU/Linux, and in my opinion the best tool for GNU/Linux is vagrant-libvirt. Proprietary, closed source platforms aren’t hackable and therefore aren’t cool!

Another advantage to using the vagrant-libvirt plugin is that it plays nicely with the existing ecosystem of libvirt tools. You can use virsh, virt-manager, and guestfish alongside Vagrant, and if your development work needs to go into production, you can be confident in knowing that it was already tested on the same awesome KVM virtualization platform that your servers run.

Prerequisites:

Let’s get going. What do you need?

  • A Fedora 20 machine

I recommend hardware that supports VT extensions. Most does these days. This should also work with other GNU/Linux distro’s, but I haven’t tested them.

Installation:

I’m going to go through this in a logical hacking order. This means you could group all the yum install commands into a single execution at the beginning, but you would learn much less by doing so.

First install some of your favourite hacking dependencies. I did this on a minimal, headless F20 installation. You might want to add some of these too:

# yum install -y wget tree vim screen mtr nmap telnet tar git

Update the system to make sure it’s fresh:

# yum update -y

Update: I’m actually now using vagrant 1.6.5, and you should try that instead. It should work for you too. Modify the below to match the newer version.

Download Vagrant version 1.5.4. No, don’t use the latest version, it probably won’t work! Vagrant has new releases practically as often as there are sunsets, and they typically cause lots of breakages.

$ wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.5.4_x86_64.rpm

and install it:

# yum install -y vagrant_1.5.4_x86_64.rpm

RVM installation:

In order to get vagrant-libvirt working, you’ll need some ruby dependencies. It turns out that RVM seems to be the best way to get exactly what you need. Use the sketchy RVM installer:

# \curl -sSL https://get.rvm.io | bash -s stable

If you don’t know why that’s sketchy, then you probably shouldn’t be hacking! I did that as root, but it probably works when you run it as a normal user. At this point rvm should be installed. The last important thing you’ll need to do is to add yourself to the rvm group. This is only needed if you installed rvm as root:

# usermod -aG rvm <username>

You’ll probably need to logout and log back in for this to take effect. Run:

$ groups

to make sure you can see rvm in the list. If you ran rvm as root, you’ll want to source the rvm.sh file:

$ source /etc/profile.d/rvm.sh

or simply use a new terminal. If you ran it as a normal user, I think RVM adds something to your ~/.bashrc. You might want to reload it:

$ source ~/.bashrc

At this point RVM should be working. Let’s see which ruby’s it can install:

$ rvm list known

Ruby version ruby-2.0.0-p353 seems closest to what is available on my Fedora 20 machine, so I’ll use that:

$ rvm install ruby-2.0.0-p353

If the exact patch number isn’t available, choose what’s closest. Installing ruby requires a bunch of dependencies. The rvm install command will ask yum for a bunch of dependencies, but if you’d rather install them yourself, you can run:

# yum install -y patch libyaml-devel libffi-devel glibc-headers autoconf gcc-c++ glibc-devel patch readline-devel zlib-devel openssl-devel bzip2 automake libtool bison

GEM installation:

Now we need the GEM dependencies for the vagrant-libvirt plugin. These GEM’s happen to have their own build dependencies, but thankfully I’ve already figured those out for you:

# yum install -y libvirt-devel libxslt-devel libxml2-devel

Update: Typically we used to now have to install the nokogiri dependencies. With newer versions of vagrant-libvirt, this is no longer necessarily required. Consider skipping this step, and trying to install the vagrant-libvirt plugin without specifying a version. If it doesn’t work, try vagrant-libvirt version 0.0.20, if that doesn’t work, install nokogiri. Feel free to post your updated experiences in the comments!

Now, install the nokogiri gem that vagrant-libvirt needs:

$ gem install nokogiri -v '1.5.11'

and finally we can install the actual vagrant-libvirt plugin:

$ vagrant plugin install --plugin-version 0.0.16 vagrant-libvirt

You don’t have to specify the –plugin-version 0.0.16 part, but doing so will make sure that you get a version that I have tested to be compatible with Vagrant 1.5.4 should a newer vagrant-libvirt release not be compatible with the Vagrant version you’re using. If you’re feeling brave, please test newer versions, report bugs, and write patches!

Making Vagrant more useful:

Vagrant should basically work at this point, but it’s missing some awesome. I’m proud to say that I wrote this awesome. I recommend my bash function and alias additions. If you’d like to include them, you can run:

$ wget https://gist.githubusercontent.com/purpleidea/8071962/raw/ee27c56e66aafdcb9fd9760f123e7eda51a6a51e/.bashrc_vagrant.sh
$ echo '. ~/.bashrc_vagrant.sh' >> ~/.bashrc
$ . ~/.bashrc    # reload

to pull in my most used Vagrant aliases and functions. I’ve written about them before. If you’re interested, please read:

KVM/QEMU installation:

As I mentioned earlier, I’m assuming you have a minimal Fedora 20 installation, so you might not have all the libvirt pieces installed! Here’s how to install any potentially missing pieces:

# yum install -y libvirt{,-daemon-kvm}

This should pull in a whole bunch of dependencies too. You will need to start and (optionally) enable the libvirtd service:

# systemctl start libvirtd.service
# systemctl enable libvirtd.service

You’ll notice that I’m using the systemd commands instead of the deprecated service command. My biggest (only?) gripe with systemd is that the command line tools aren’t as friendly as they could be! The systemctl equivalent requires more typing, and make it harder to start or stop the same service in quick succession, because it buries the action in the middle of the command instead of leaving it at the end!

The libvirtd service should finally be running. On my machine, it comes with a default network which got in the way of my vagrant-libvirt networking. If you want to get rid of it, you can run:

# virsh net-destroy default
# virsh net-undefine default

and it shouldn’t bother you anymore. One last hiccup. If it’s your first time installing KVM, you might run into bz#950436. To workaround this issue, I had to run:

# rmmod kvm_intel
# rmmod kvm
# modprobe kvm
# modprobe kvm_intel

Without this “module re-loading” you might see this error:

Call to virDomainCreateWithFlags failed: internal error: Process exited while reading console log output: char device redirected to /dev/pts/2 (label charserial0)
Could not access KVM kernel module: Permission denied
failed to initialize KVM: Permission denied

Additional installations:

To make your machine somewhat more palatable, you might want to consider installing bash-completion:

# yum install -y bash-completion

You’ll also probably want to add the PolicyKit (polkit) .pkla file that I recommend in my earlier article. Typically that means adding something like:

[Allow james libvirt management permissions]
Identity=unix-user:james
Action=org.libvirt.unix.manage
ResultAny=yes
ResultInactive=yes
ResultActive=yes

as root to somewhere like:

/etc/polkit-1/localauthority/50-local.d/vagrant.pkla

Your machine should now be setup perfectly! The last thing you’ll need to do is to make sure that you get a Vagrantfile that does things properly! Here are some recommendations.

Shared folders:

Shared folders are a mechanism that Vagrant uses to pass data into (and sometimes out of) the virtual machines that it is managing. Typically you can use NFS, rsync, and some provider specific folder sharing like 9p. Using rsync is the simplest to set up, and works exceptionally well. Make sure you include the following line in your Vagrantfile:

config.vm.synced_folder './', '/vagrant', type: 'rsync'

If you want to see an example of this in action, you can have a look at my puppet-gluster Vagrantfile. If you are using the puppet apply provisioner, you will have to set it to use rsync as well:

puppet.synced_folder_type = 'rsync'

KVM performance:

Due to a regression in vagrant-libvirt, the default driver used for virtual machines is qemu. If you want to use the accelerated KVM domain type, you’ll have to set it:

libvirt.driver = 'kvm'

This typically gives me a 5x performance increase over plain qemu. This fix is available in the latest vagrant-libvirt version. The default has been set to KVM in the latest git master.

Dear internets!

I think this was fairly straightforward. You could probably even put all of these commands in a shell script and just run it to get it all going. What we really need is proper RPM packaging. If you can help out, that would be excellent!

If we had a version of vagrant-libvirt alongside a matching Vagrant version in Fedora, then developers and hackers could target that, and we could easily exchange dev environments, hackers could distribute product demos as full vagrant-libvirt clusters, and I could stop having to write these types of articles ;)

I hope this was helpful to you. Please let me know in the comments.

Happy hacking,

James

 

25 thoughts on “Vagrant on Fedora with libvirt (reprise)

    • Hey Matt,

      The vagrant-kvm plugin isn’t very useful. It limits you to only one machine. Vagrant-libvirt is (IMO) the way to go! Having said that, I don’t mind if vagrant-kvm is also in Fedora, but I don’t know how useful it would be.

      If someone can make an RPM of vagrant-libvirt, this would be particularly useful! Maybe you know someone who can look into this? Thanks!

      • I can make sure it is on the giant todo list, and try to get it on someone’s attention. I’d also love to have this.

      • lol “giant todo list” — that sounds great!
        well i’ll keep hacking… if you find a victim, err willing rpm packager, that would be sweet!

        I’ll hopefully have a follow up article within the week with some of the new features of vagrant-libvirt and some other fun stuff which should hopefully help with the vagrant-libvirt case.

  1. Pingback: Five Things in Fedora This Week (2014-05-13) | Fedora Magazine

  2. Pingback: Pět věcí ve Fedoře za poslední týden (13. 5. 2014) | Fedora.cz

    • Hey Vinny! Thanks for checking it out.

      Quite true about yum install. I don’t like to do this, but only because I’m a paranoid soul, and I like to know what bits I have before I use them on my system. This also prevents the extremely low chance “detect user-agent of yum http getter and insert backdoor-ed rpm” scenario. In the off chance that the package was corrupted or modified some way, I’ll have it around in an rsnapshotted ~/Downloads/ for inspection. No I’ve never had to do this, not even once ;)

      As for the EPEL error! It’s my bad. I was in CentOS mode, and forgot this was on F20. I mixed it up because my _guests_ are CentOS 6. EPEL isn’t needed (nor does it work). Just yum install bash-completion :) Good catch, thanks!

    • FYI , i skipped ahead and hacked through the RVM part – works fine – just dont see the group – is that an issue from your perspective?

      • You didn’t get the group because you didn’t run the installer as root. In the article this could be more clear.’
        Also you need to logout and log back in for the group updates to take effect. Let me know if this solves your issue.

        Cheers!

        NOTE: commands that start with: # use root, commands that start with: $ are user.

  3. Hi james. I noticed you are manually installing the nokogiri deps. But I;m pretty sure that vagrant should bundle and embed all the deps for you. Just remembered this post, and thought id leave a comment b/c Im actually looking into a nokogiri issue right now and just discovered this.

  4. I hit an error installing nokogiri. It turns out I had to enable a (different?) default ruby version (not clear). Here’s what I did:
    $ rvm alias create default 2.1.2-devel
    Creating alias default for ruby-2.1.2-devel….
    $ rvm list

    rvm rubies

    ruby-2.0.0-p353 [ x86_64 ]
    ruby-2.1.1 [ x86_64 ]
    ruby-2.1.1-dev [ x86_64 ]
    =* ruby-2.1.2-devel [ x86_64 ]

    # => – current
    # =* – current && default
    # * – default

  5. II am on F20 and this is where I could reach upto …
    $ vagrant up –no-parallel –provider=libvirt

    Bringing machine ‘vm1’ up with ‘libvirt’ provider…
    Bringing machine ‘vm2’ up with ‘libvirt’ provider…
    ==> vm1: Starting domain.
    ==> vm1: Waiting for domain to get an IP address…
    ==> vm1: Waiting for SSH to become available…
    ==> vm1: Creating shared folders metadata…
    ==> vm1: Rsyncing folder: /home/deepakcs/work/vagrant/test1/ => /vagrant
    ==> vm2: Creating image (snapshot of base box volume).
    ==> vm2: Creating domain with the following settings…
    ==> vm2: — Name: test1_vm2
    ==> vm2: — Domain type: kvm
    ==> vm2: — Cpus: 1
    ==> vm2: — Memory: 512M
    ==> vm2: — Base box: centos-6
    ==> vm2: — Storage pool: default
    ==> vm2: — Image: /var/lib/libvirt/images/test1_vm2.img
    ==> vm2: — Volume Cache: default
    ==> vm2: — Kernel:
    ==> vm2: — Initrd:
    ==> vm2: — Command line :
    ==> vm2: Starting domain.
    ==> vm2: Waiting for domain to get an IP address…
    ==> vm2: Waiting for SSH to become available…
    ==> vm2: Starting domain.
    ==> vm2: Waiting for domain to get an IP address…
    ==> vm2: Waiting for SSH to become available…
    ==> vm2: Creating shared folders metadata…
    ==> vm2: Rsyncing folder: /home/deepakcs/work/vagrant/test1/ => /vagrant
    ==> vm2: Rsyncing folder: /home/deepakcs/work/git/puppet-gluster/puppet-gluster/vagrant/puppet/manifests/ => /tmp/vagrant-puppet-1/manifests
    ==> vm2: Rsyncing folder: /home/deepakcs/work/git/puppet-gluster/puppet-gluster/vagrant/puppet/modules/ => /tmp/vagrant-puppet-1/modules-0
    ==> vm2: Configuring and enabling network interfaces…
    ==> vm2: Running provisioner: puppet…
    Running Puppet with site.pp…
    Warning: Could not retrieve fact fqdn
    Warning: Host is missing hostname and/or domain: localhost
    Notice: Compiled catalog for localhost in environment production in 0.10 seconds
    Notice: /Stage[main]/Main/Node[default]/File[/etc/motd]/content: content changed ‘{md5}d41d8cd98f00b204e9800998ecf8427e’ to ‘{md5}096a572174f7cc9b2285e74dd4c789ac’
    Notice: Finished catalog run in 0.25 seconds

    But i dont see any puppet provisioning happening on my VMs (No activity per virt-manager)

  6. Pingback: Getting Vagrant with libvirt support on your Fedora 20 – Notes to self

    • Interesting that they added that! I guess this means rvm is now 5% less sketchy!

      I think that rvm, pip, ‘go get’, and every other “special” package manager needs to get a single upstream project, and build that into distros! It’s madness these days ;)

      Thanks for sharing!

  7. Ok, done that. Let’s see if it works. All commands fine, except I’ve installed 0.0.20 version as you suggested ;-)

    Vagrant gained popularity because of MacOS users. For Fedora 20+ I am pretty successful with a different stack: virt-builder and virt-install. You can achieve the same in a more cleaner way, but I have to admit with Vagrant there are plenty of existing recipes out there. And of course this only works in (modern) Linux distros. In case you don’t know Richs tool: http://libguestfs.org/virt-builder.1.html

    Yeah, middle command of the systemctl command is a bummer. Fortunately we have the Alt-dot shortcut for Bash, but I wonder what was the motivation for this change. I don’t really see any benefits here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s