New puppet-gluster features before Linuxcon

Hey there,

I’ve done a bit of puppet-gluster hacking lately to try to squeeze some extra features and testing in before Linuxcon. Here’s a short list:

If there are features or bugs that you’d like to see added or removed (respectively) please let me know ASAP so that I can try to get something ready for you before my Linuxcon talk. I also don’t have any hardware RAID or physical hardware to test external storage partitions (bricks) on. If you have any that you can donate or let me hack on for a while, there are some features I’d like to test. Contact me!

I’ve got a few more things queued up too, but you’ll have to wait and see. Until then,

Happy hacking,

James

Advertisements

Fresh releases! puppet-ipa, puppet-nfs, puppet-gluster

I’ve been a little slow in making release announcements, so here’s some news:

I’ve just released the third stage of my puppet-ipa module. At the moment it now supports installation, managing of hosts, and managing of services. It integrates with my puppet-nfs module to allow you to easily setup and run an NFSv4 kerberized server and client.

While we’re at it, that’s some more news: I’ve just released a puppet-nfs module to make your /etc/exports management easier. It’s designed to manage other security types, or even to work without kerberos or any authentication at all, but I haven’t tested those.

Back to puppet-ipa for a moment. I’d like you to know that I went to great lengths to make this a very versatile module. Some users probably want certain resources managed by puppet, and others not. With the included features, you can even specify exclusion criteria so that a certain pattern of hosts aren’t touched by puppet. This is useful if you’re slowly converting your ipa setup to be managed by puppet.

You can use $watch and $modify, two special parameters that I added to precisely control what kind of changes you want to allow puppet to make. These are kind of complicated to explain, but suffice it to say that this module should handle whatever situation you’re in.

For the security minded folks, puppet-ipa, never transfers or touches a keytab file. It will securely and automatically provision your hosts and services without storing secret information in puppet. The module isn’t finished, but it’s built right.

Gluster users might find this particular trio useful for offering gluster backed, kerberized, NFS exports. Here’s an example that I made just for you.

Since you sound like you’re having fun deploying servers like crazy, it’s probably useful to have a puppet-cobbler module. I’ve released this module because it’s useful to me, however it really isn’t release ready, but I think it’s better than some (most?) of the other puppet-cobbler code that’s out there. One other warning is that I have a large rearchitecturing planned for this module, so don’t get too attached. It’s going to get better!

So that’s your lot for today, have fun, and

Happy Hacking!

James

PS: If you’re in a giving mood, I’m in the need for some x86_64 compatible test hardware. If you’re able to donate, please let me know!

Playing with FreeIPA and puppet

So I just rolled a new vm to hack around with FreeIPA. Here are some things that I’ve come across so far. I was planning on configuring LDAP, and Kerberos manually, but the included webui looks like a lovely tool to have for the data entry, user administrator type who likes to click on things. Let’s explore…

/etc/hosts:

FreeIPA is choosy about how your /etc/hosts is formatted. It requires an entry that has a particular order, that is:

192.168.123.7    ipa.example.com    ipa

Obviously replace with your own values. This presents itself as:

The host name ipa.example.com does not match the primary host name ipa. Please check /etc/hosts or DNS name resolution

I had to dive into the source to figure this one out!

webui:

I’m in hack mode, and my laptop (hack station) is not participating in the domain that I’m pretending to manage. In addition, I’m not directly connected to the vm where I’m testing out FreeIPA. As usual, I port forward:

$ ssh root@ipa -L 1443:localhost:443

but when attempting to try the webui:

$ firefox https://localhost:1443/ipa/ui/

I get redirected to the official fqdn, and at port 443. After searching around, it turns out there is a: –no-ui-redirect option that you can pass to the ipa-server-install program, but it only comments out one line of the /etc/httpd/conf.d/ipa-rewrite.conf and doesn’t let me do exactly what I want. I’m sure someone with pro apache skills could hack this up properly, but I haven’t the patience.

As user ab in #freeipa kindly pointed out:

01:21 < ab> primary authentication method of web ui is Kerberos. 
            Your browser, when configured, will need to obtain a kerberos 
            ticket against web ui's server, that's why you're forced to connect 
            to fully qualified hostname
01:22 < ab> otherwise a browser would attempt to obtain ticket to 
            HTTP/localhost@REALM which does not exist
01:22 < ab> and wouldn't be what web server web ui is running on is using

which is a good point. For hacking purposes, I’m happy to forgo kerberos logins and type in a password manually, but since my use case is particularly small, I’ll just hack around this for now, and maybe a future FreeIPA will get this option. At the moment, it’s not supported.

A bad hacker could modify their /etc/hosts to include:

127.0.0.1    ipa.example.com ipa localhost.localdomain localhost

and run ssh as root (very bad!):

$ sudo ssh root@ipa -L 443:localhost:443 -L 80:localhost:80

to get easily get access locally. But don’t do this. It’s evil.

inactivity timeouts:

The web ui times out after 20 minutes. To increase this add:

session_auth_duration=42 minutes

to your /etc/ipa/default.conf, and restart httpd. You can have a look at the parser for an idea of what kind of values are acceptable.

puppet?:

As you might agree, it’s nice to have puppet modules to get you up and running. FreeIPA was easy to install, and my puppet module now makes it automatic. I’ve written a lot of fancy puppet code to manage your IPA resources. It’s not quite finished, and more resource types are on the way, but you can follow along at:

https://github.com/purpleidea/puppet-ipa

Happy hacking,

James

 

puppet lsi hardware raid module

In response to some discussion in the gluster community, I am releasing my puppet-lsi module. It’s quite simple, but it is very useful for rebuilding machines. It could do a lot more, but I wanted to depend on the proprietary LSI tools as little as possible. Running megacli with puppet would be a very doable hack, but I’m not sure enough devops out there who would use that feature.

Usage is straightforward if you like the sensible defaults:

include lsi::msm

The general idea is that you’ve probably already setup all your “virtual drive” RAID configurations initially, and now you’re deploying your setup using cobbler and puppet-gluster. This puppet-lsi module should install all the client side LSI tools, and make sure monitoring for the hardware RAID is working. Megacli and all the (evil?) vivaldi framework stuff will be up and running after puppet has run.

I haven’t tested this on a wide array of hardware, and there might even be newer LSI gear on the market. Please don’t test it on production servers. If you want help with this, you might have to sponsor some hardware, or send me somewhere where I can hack on some, because I don’t have a gluster test rig at the moment.

I am curious to hear what kind of RAID you’re using with gluster. Hardware? Software? Details rock. SGPIO with mdadm, and you’re my hero. I want to hear about that!

https://github.com/purpleidea/puppet-lsi/

I hope this was useful to you, and in the meantime,

Happy hacking,

James

PS: The most useful feature of this module, is that it sets up monitoring of your RAID, and lets you access the management daemon through the now installed LSI services.

puppet matlab module

I don’t like repetitive work, and installing matlab counts doubly as so. Once I figured out the correct steps, I automated it with a puppet module. The downside is that the install takes a while because puppet needs to copy the iso locally. This is okay because I can be busy doing something else while this is happening.

Using the module is quite easy:

matlab::install { 'R2011a':
    iso => 'puppet://files/matlab/MATHWORKS_R2011A.iso',
    licensekey => '#####-#####-#####-#####',    # provide your own here
    licensefile => 'puppet:///files/matlab/license.lic',    # get your own!
    licenseagree => true,    # setting this to true 'acknowledges' their (C)
    prefix => '/usr/local',
}

You might notice that this supports installing multiple releases on the same machine. You will have to provide your own license key and license file.

This isn’t routinely tested, so if the latest matlab installer changes and this breaks, please let me know. Personally, I’d recommend you use octave instead, but if you really need matlab, hopefully this will make your sysadmin happier.

Enjoy a copy of the code:

https://github.com/purpleidea/puppet-matlab

Happy hacking,

James

PS: matlab also supports installs that use a licensing server. This module doesn’t support these at the moment, but you’re welcome to send me a patch. What a waste of hardware!

 

puppet rsnapshot module

Today I am releasing a puppet module for rsnapshot. The nice feature of this module, is that it lets you configure multiple different instances of rsnapshot, so that they could all run in parallel. Rsnapshot doesn’t support this directly, so this puppet module does the heavy lifting of separating out and managing each instance.

https://github.com/purpleidea/puppet-rsnapshot

The examples/ directory should give you a hint on how to use it. For everything else, have a look at the code, or feel free to leave me a comment. I hope you find it useful!

Happy hacking,

James