Fresh releases! puppet-ipa, puppet-nfs, puppet-gluster

I’ve been a little slow in making release announcements, so here’s some news:

I’ve just released the third stage of my puppet-ipa module. At the moment it now supports installation, managing of hosts, and managing of services. It integrates with my puppet-nfs module to allow you to easily setup and run an NFSv4 kerberized server and client.

While we’re at it, that’s some more news: I’ve just released a puppet-nfs module to make your /etc/exports management easier. It’s designed to manage other security types, or even to work without kerberos or any authentication at all, but I haven’t tested those.

Back to puppet-ipa for a moment. I’d like you to know that I went to great lengths to make this a very versatile module. Some users probably want certain resources managed by puppet, and others not. With the included features, you can even specify exclusion criteria so that a certain pattern of hosts aren’t touched by puppet. This is useful if you’re slowly converting your ipa setup to be managed by puppet.

You can use $watch and $modify, two special parameters that I added to precisely control what kind of changes you want to allow puppet to make. These are kind of complicated to explain, but suffice it to say that this module should handle whatever situation you’re in.

For the security minded folks, puppet-ipa, never transfers or touches a keytab file. It will securely and automatically provision your hosts and services without storing secret information in puppet. The module isn’t finished, but it’s built right.

Gluster users might find this particular trio useful for offering gluster backed, kerberized, NFS exports. Here’s an example that I made just for you.

Since you sound like you’re having fun deploying servers like crazy, it’s probably useful to have a puppet-cobbler module. I’ve released this module because it’s useful to me, however it really isn’t release ready, but I think it’s better than some (most?) of the other puppet-cobbler code that’s out there. One other warning is that I have a large rearchitecturing planned for this module, so don’t get too attached. It’s going to get better!

So that’s your lot for today, have fun, and

Happy Hacking!

James

PS: If you’re in a giving mood, I’m in the need for some x86_64 compatible test hardware. If you’re able to donate, please let me know!

Playing with FreeIPA and puppet

So I just rolled a new vm to hack around with FreeIPA. Here are some things that I’ve come across so far. I was planning on configuring LDAP, and Kerberos manually, but the included webui looks like a lovely tool to have for the data entry, user administrator type who likes to click on things. Let’s explore…

/etc/hosts:

FreeIPA is choosy about how your /etc/hosts is formatted. It requires an entry that has a particular order, that is:

192.168.123.7    ipa.example.com    ipa

Obviously replace with your own values. This presents itself as:

The host name ipa.example.com does not match the primary host name ipa. Please check /etc/hosts or DNS name resolution

I had to dive into the source to figure this one out!

webui:

I’m in hack mode, and my laptop (hack station) is not participating in the domain that I’m pretending to manage. In addition, I’m not directly connected to the vm where I’m testing out FreeIPA. As usual, I port forward:

$ ssh root@ipa -L 1443:localhost:443

but when attempting to try the webui:

$ firefox https://localhost:1443/ipa/ui/

I get redirected to the official fqdn, and at port 443. After searching around, it turns out there is a: –no-ui-redirect option that you can pass to the ipa-server-install program, but it only comments out one line of the /etc/httpd/conf.d/ipa-rewrite.conf and doesn’t let me do exactly what I want. I’m sure someone with pro apache skills could hack this up properly, but I haven’t the patience.

As user ab in #freeipa kindly pointed out:

01:21 < ab> primary authentication method of web ui is Kerberos. 
            Your browser, when configured, will need to obtain a kerberos 
            ticket against web ui's server, that's why you're forced to connect 
            to fully qualified hostname
01:22 < ab> otherwise a browser would attempt to obtain ticket to 
            HTTP/localhost@REALM which does not exist
01:22 < ab> and wouldn't be what web server web ui is running on is using

which is a good point. For hacking purposes, I’m happy to forgo kerberos logins and type in a password manually, but since my use case is particularly small, I’ll just hack around this for now, and maybe a future FreeIPA will get this option. At the moment, it’s not supported.

A bad hacker could modify their /etc/hosts to include:

127.0.0.1    ipa.example.com ipa localhost.localdomain localhost

and run ssh as root (very bad!):

$ sudo ssh root@ipa -L 443:localhost:443 -L 80:localhost:80

to get easily get access locally. But don’t do this. It’s evil.

inactivity timeouts:

The web ui times out after 20 minutes. To increase this add:

session_auth_duration=42 minutes

to your /etc/ipa/default.conf, and restart httpd. You can have a look at the parser for an idea of what kind of values are acceptable.

puppet?:

As you might agree, it’s nice to have puppet modules to get you up and running. FreeIPA was easy to install, and my puppet module now makes it automatic. I’ve written a lot of fancy puppet code to manage your IPA resources. It’s not quite finished, and more resource types are on the way, but you can follow along at:

https://github.com/purpleidea/puppet-ipa

Happy hacking,

James

 

puppet lsi hardware raid module

In response to some discussion in the gluster community, I am releasing my puppet-lsi module. It’s quite simple, but it is very useful for rebuilding machines. It could do a lot more, but I wanted to depend on the proprietary LSI tools as little as possible. Running megacli with puppet would be a very doable hack, but I’m not sure enough devops out there who would use that feature.

Usage is straightforward if you like the sensible defaults:

include lsi::msm

The general idea is that you’ve probably already setup all your “virtual drive” RAID configurations initially, and now you’re deploying your setup using cobbler and puppet-gluster. This puppet-lsi module should install all the client side LSI tools, and make sure monitoring for the hardware RAID is working. Megacli and all the (evil?) vivaldi framework stuff will be up and running after puppet has run.

I haven’t tested this on a wide array of hardware, and there might even be newer LSI gear on the market. Please don’t test it on production servers. If you want help with this, you might have to sponsor some hardware, or send me somewhere where I can hack on some, because I don’t have a gluster test rig at the moment.

I am curious to hear what kind of RAID you’re using with gluster. Hardware? Software? Details rock. SGPIO with mdadm, and you’re my hero. I want to hear about that!

https://github.com/purpleidea/puppet-lsi/

I hope this was useful to you, and in the meantime,

Happy hacking,

James

PS: The most useful feature of this module, is that it sets up monitoring of your RAID, and lets you access the management daemon through the now installed LSI services.

Collecting duplicate resources in puppet

I could probably write a long design article explaining why identical duplicate resources should be allowed [1] in puppet. If puppet is going to survive in the long-term, they will have to build in this feature. In the short-term, I will have to hack around deficiency. As luck would have it, Mr. Bode has already written part one of the hack: ensure_resource.

Why?

Suppose you have a given infrastructure with N vaguely identical nodes. N could equal 2 for a dual primary or active-passive cluster, or N could be greater than 2 for a more elaborate N-ary cluster. It is sufficient to say, that each of those N nodes might export an identical puppet resource which one (or many) clients might need to collect, to operate correctly. It’s important that each node export this, so that there is no single point of failure if one or more of the cluster nodes goes missing.

How?

As I mentioned, ensure_resources is a good enough hack to start. Here’s how you take an existing resource, and make it duplicate friendly. Take for example, the bulk of my dhcp::subnet resource:

define dhcp::subnet(
      $subnet,
      # [...]
      $range = [],
      $allow_duplicates = false
) {
      if $allow_duplicates { # a non empty string is also a true
            # allow the user to specify a specific split string to use...
            $c = type($allow_duplicates) ? {
                  'string' => "${allow_duplicates}",
                  default => '#',
            }
            if "${c}" == '' {
                  fail('Split character(s) cannot be empty!')
            }

            # split into $realname-$uid where $realname can contain split chars
            $realname = inline_template("<%= name.rindex('${c}').nil?? name : name.slice(0, name.rindex('${c}')) %>")
            $uid = inline_template("<%= name.rindex('${c}').nil?? '' : name.slice(name.rindex('${c}')+'${c}'.length, name.length-name.rindex('${c}')-'${c}'.length) %>")

            $params = { # this must use all the args as listed above...
                  'subnet' => $subnet,
                  # [...]
                  'range' => $range,
                  # NOTE: don't include the allow_duplicates flag...
            }

            ensure_resource('dhcp::subnet', "${realname}", $params)
      } else { # body of the actual resource...

            # BUG: lol: https://projects.puppetlabs.com/issues/15813
            $valid_range = type($range) ? {
                  'array' => $range,
                  default => [$range],
            }

            # the templating part of the module... 
            frag { "/etc/dhcp/subnets.d/${name}.subnet.frag":
                  content => template('dhcp/subnet.frag.erb'),
            }
      }
}

As you can see, I added an $allow_duplicates parameter to my resource. If it is set to true, then when the resource is defined, it parses out a trailing #comment from the $namevar. This can guarantee uniqueness for the $name (if they happen to be on the same node) but more importantly, it can guarantee uniqueness on a collector, where you will otherwise be unable to workaround the $name collision.

This is how you use this on one of the exporting nodes:

@@dhcp::subnet { "dmz#${hostname}":
    subnet => ...,
      range => [...],
      allow_duplicates => '#',
}

and on the collector:

Dhcp::Subnet <<| tag == 'dhcp' and title != "${dhcp_zone}" |>> {
}

There are a few things to notice:

1. The $allow_duplicates argument can be set to true (a boolean), or to any string. If you pick a string, then that will be used to “split” out the end comment. It’s smart enough to split with a reverse index search so that your name can contain the #’s if need be. By default it looks for a single #, but you could replace this with ‘XXX123HACK‘ if that was the only unique string match you can manage. Make sure not to use the string value of ‘true‘.
2. On my collector I like to filter by title. This is the $namevar. Sadly, this doesn’t support any fancier matching like in_array or startswith. I consider this a puppet deficiency. Hopefully someone will fix this to allow general puppet code here.
3. Adding this to each resource is kind of annoying. It’s obviously a hack, but it’s the right thing to do for the time being IMHO.

Hope you had fun with this.

Happy hacking,

James

PS: [1] One side note, in the general case for custom resources, I actually think that by default duplicate parameters should be required, but that a resource could provide an optional function such as is_matched which would take as input the two parameter hash trees, and decide if they’re “functionally equivalent”. This would let an individual resource decide if it matters that you specified thing=>yes in one and thing=>true in the other. Functionally it matters that duplicate resources don’t have conflicting effects. I’m sure this would be particularly bug prone, and probably cause thrashing in some cases, which is why, by default the parameters should all match. </babble>

puppet matlab module

I don’t like repetitive work, and installing matlab counts doubly as so. Once I figured out the correct steps, I automated it with a puppet module. The downside is that the install takes a while because puppet needs to copy the iso locally. This is okay because I can be busy doing something else while this is happening.

Using the module is quite easy:

matlab::install { 'R2011a':
    iso => 'puppet://files/matlab/MATHWORKS_R2011A.iso',
    licensekey => '#####-#####-#####-#####',    # provide your own here
    licensefile => 'puppet:///files/matlab/license.lic',    # get your own!
    licenseagree => true,    # setting this to true 'acknowledges' their (C)
    prefix => '/usr/local',
}

You might notice that this supports installing multiple releases on the same machine. You will have to provide your own license key and license file.

This isn’t routinely tested, so if the latest matlab installer changes and this breaks, please let me know. Personally, I’d recommend you use octave instead, but if you really need matlab, hopefully this will make your sysadmin happier.

Enjoy a copy of the code:

https://github.com/purpleidea/puppet-matlab

Happy hacking,

James

PS: matlab also supports installs that use a licensing server. This module doesn’t support these at the moment, but you’re welcome to send me a patch. What a waste of hardware!

 

puppet rsnapshot module

Today I am releasing a puppet module for rsnapshot. The nice feature of this module, is that it lets you configure multiple different instances of rsnapshot, so that they could all run in parallel. Rsnapshot doesn’t support this directly, so this puppet module does the heavy lifting of separating out and managing each instance.

https://github.com/purpleidea/puppet-rsnapshot

The examples/ directory should give you a hint on how to use it. For everything else, have a look at the code, or feel free to leave me a comment. I hope you find it useful!

Happy hacking,

James