Finite state machines in puppet

In my attempt to push puppet to its limits, (for no particular reason), to develop more powerful puppet modules, to build in a distributed lock manager, and to be more dynamic, I’m now attempting to build a Finite State Machine (FSM) in puppet.

Is this a real finite state machine, and why would you do this?

Computer science professionals might not approve of the purity level, but they will hopefully appreciate the hack value. I’ve done this to illustrate a state transition technique that will be necessary in a module that I am writing.

Can we have an example?

Sure! I’ve decided to model thermodynamic phase transitions. Here’s what we’re building:

Phase_change_-_en.svg

How does it work?

Start off with a given define that accepts an argument. It could have one argument, or many, and be of whichever type you like, such as an integer, or even a more complicated list type. To keep the example simple, let’s work with a single argument named $input.

define fsm::transition(
        $input = ''
) {
        # TODO: add amazing code here...
}

The FSM runs as follows: On first execution, the $input value is saved to a local file by means of a puppet exec type. A corresponding fact exists to read from that file and create a unique variable for the fsm::transition type. Let’s call that variable $last. This is the special part!

# ruby fact to pull in the data from the state file
found = {}
Dir.glob(transition_dir+'*').each do |d|
    n = File.basename(d)    # should be the fsm::transition name
    if n.length > 0 and regexp.match(n)
        f = d.gsub(/\/$/, '')+'/state'    # full file path
        if File.exists?(f)
            # TODO: future versions should unpickle (but with yaml)
            v = File.open(f, 'r').read.strip    # read into str
            if v.length > 0 and regexp.match(v)
                found[n] = v
            end
        end
    end
end

found.keys.each do |x|
    Facter.add('fsm_transition_'+x) do
        #confine :operatingsystem => %w{CentOS, RedHat, Fedora}
        setcode {
            found[x]
        }
    end
end

On subsequent runs, the process gets more interesting: The $input value and the $last value are used to decide what to run. They can be different because the user might have changed the $input value. Logic trees then decide what actions you’d like to perform. This lets us compare the previous state to the new desired state, and as a result, be more intelligent about which actions need to run for a successful state transition. This is the FSM part.

# logic tree modeling phase transitions
# https://en.wikipedia.org/wiki/Phase_transition
$transition = "${valid_last}" ? {
        'solid' => "${valid_input}" ? {
               'solid' => true,
               'liquid' => 'melting',
               'gas' => 'sublimation',
               'plasma' => false,
               default => '',
        },
        'liquid' => "${valid_input}" ? {
               'solid' => 'freezing',
               'liquid' => true,
               'gas' => 'vaporization',
               'plasma' => false,
               default => '',
        },
        'gas' => "${valid_input}" ? {
               'solid' => 'deposition',
               'liquid' => 'condensation',
               'gas' => true,
               'plasma' => 'ionization',
               default => '',
        },
        'plasma' => "${valid_input}" ? {
               'solid' => false,
               'liquid' => false,
               'gas' => 'recombination',
               'plasma' => true,
               default => '',
        },
        default => '',
}

Once the state transition actions have completed successfully, the exec must store the $input value in the local file for future use as the unique $last fact for the next puppet run. If there are errors during state transition execution, you may choose to not store the updated value (to cause a re-run) and/or to add an error condition fact that the subsequent puppet run will have to read in and handle accordingly. This is the important part.

$f = "${vardir}/transition/${name}/state"
$diff = "/usr/bin/test '${valid_input}' != '${valid_last}'"

# TODO: future versions should pickle (but with yaml)
exec { "/bin/echo '${valid_input}' > '${f}'":
        logoutput => on_failure,
        onlyif => "/usr/bin/test ! -e '${f}' || ${diff}",
        require => File["${vardir}/"],
        alias => "fsm-transition-${name}",
}

Can we take this further?

It might be beneficial to remember the path we took through our graph. To do this, on each transition we append the new state to a file on our local puppet client. The corresponding fact, is similar to the $last fact, except it maintains a list of values instead of just one. There is a max length variable that can be used to avoid storing unlimited old states.

Does this have a practical use?

Yes, absolutely! I realized that something like this could be useful for puppet-gluster. Stay tuned for more patches.

Hopefully you enjoyed this. By following the above guidelines, you should now have some extra tricks for building state transitions into your puppet modules. Let me know if you found this hack awesome and unique.

I’ve posted the full example module here.

Happy Hacking,

James

 

4 thoughts on “Finite state machines in puppet

  1. Very innovative. So now if we could create a contextual connection between the FSM and the service workflow SLA, we could actually use puppet for orchestration instead of top down scheduling. :-)

    • This is an interesting comment that we should really talk more about.

      First of all, I’ve got more fun code coming :) You should tell me more about the problem you’re trying to solve.

      Secondly, you might notice that I don’t have any posts about orchestration on my blog. I’m not against it, but fundamentally I think it’s better if each individual puppet client can have it’s own algorithm to follow and works effectively with others to get the job done. This is similar to a company with good enough employees that the HR department (the puppetmaster) is around, but mostly stays out of the way.

      Another way to look at it is that the puppetmaster already does light orchestration in that it helps nodes exchange information in the puppetdb for using exported resources. This model is similar to the internet model where the individual nodes have most of the logic, the network is used for data exchange, and there is some mostly central data and logic to coordinate things (like DNS). In contrast, I see standard service orchestration as similar to the POTS (telephone) system, where all of the logic is stored at the central office (the orchestrator) and the clients are thin clients.

      Having said all that, I think there are a number of (design?) improvements to be made to puppet. If someone at puppetlabs wants to talk about these that would be cool, but it’s still pretty close to being the right thing.

  2. Yeah we should talk – I think the data center as it is today is an antiquated design and if we turn the tables on infrastructure and look at service based re-architecture, we could actually do some really cool things. Especially along your phone analogy. I think the orchestration that is being done today is a farce though – whole node information access is important as far as the systematic remediation is concerned, what is missing is the ITSM structure that connects the ingredient with the service requirements. It seems to me that folks are not able to re-establish the service guarantees of olden days, when we could actually go downstairs, enter the raised floor area and hug the machines :-).

    • “service based re-architecture”. hmmm, I guess I’d need more specifics. Not sure I’ll get all of your ideas across in blog comments however.

      I completely agree about your orchestration points. I think this lies in the fact that centralized problems are easier to solve than decentralized ones, and there isn’t necessarily an incentive for admins/architects to build those tools. I tried to do something like this for my puppet-gluster module. Have a look at the code if you’re interested. There’s still a lot to come though.

      I too enjoy raised floors. I found many surprises in old rooms when lifting up floor tiles. I’ve got a feeling you’re feeling some mainframe nostalgia.

Leave a comment