puppet rsnapshot module

Today I am releasing a puppet module for rsnapshot. The nice feature of this module, is that it lets you configure multiple different instances of rsnapshot, so that they could all run in parallel. Rsnapshot doesn’t support this directly, so this puppet module does the heavy lifting of separating out and managing each instance.

https://github.com/purpleidea/puppet-rsnapshot

The examples/ directory should give you a hint on how to use it. For everything else, have a look at the code, or feel free to leave me a comment. I hope you find it useful!

Happy hacking,

James

 

Forcing firefox to remember passwords

There are a handful of websites out there that decide that they know better than your browser and tell it to not offer to save passwords. They do this by setting a form autocomplete attribute to off.

Since we already agree that HTML and the web are a terrible idea, hopefully we can find a way to hack around this. It turns out that I didn’t have to, because many others have solved this hack before me. The cleanest version I found is here: http://www.howtogeek.com/62980/how-to-force-your-browser-to-remember-passwords/

It’s not that complicated actually, a little bookmarklet (javascript code, stored in a bookmark, and activated when you open it) is saved in your browser, and on activation, it loops through all the page’s forms and turns on the autocomplete off‘s.

I’ve copied the code here, in the interests of archiving this very useful hack. Here you go:

Shortened form:

javascript:(function(){var%20ac,c,f,fa,fe,fea,x,y,z;ac="autocomplete";c=0;f=document.forms;for(x=0;x<f.length;x++){fa=f[x].attributes;for(y=0;y<fa.length;y++){if(fa[y].name.toLowerCase()==ac){fa[y].value="on";c++;}}fe=f[x].elements;for(y=0;y<fe.length;y++){fea=fe[y].attributes;for(z=0;z<fea.length;z++){if(fea[z].name.toLowerCase()==ac){fea[z].value="on";c++;}}}}alert("Enabled%20'"+ac+"'%20on%20"+c+"%20objects.");})();

Long form:

function() {
   var ac, c, f, fa, fe, fea, x, y, z;
   //ac = autocomplete constant (attribute to search for)
   //c = count of the number of times the autocomplete constant was found
   //f = all forms on the current page
   //fa = attibutes in the current form
   //fe = elements in the current form
   //fea = attibutes in the current form element
   //x,y,z = loop variables

   ac = "autocomplete";
   c = 0;
   f = document.forms;

   //cycle through each form
   for(x = 0; x < f.length; x++) {
      fa = f[x].attributes;
      //cycle through each attribute in the form
      for(y = 0; y < fa.length; y++) {
         //check for autocomplete in the form attribute
         if(fa[y].name.toLowerCase() == ac) {
            fa[y].value = "on";
            c++;
         }
      }

      fe = f[x].elements;
      //cycle through each element in the form
      for(y = 0; y < fe.length; y++) {
         fea = fe[y].attributes;
         //cycle through each attribute in the element
         for(z = 0; z < fea.length; z++) {
            //check for autocomplete in the element attribute
            if(fea[z].name.toLowerCase() == ac) {
               fea[z].value = "on";
               c++;
            }
         }
      }
   }

   alert("Enabled '" + ac + "' on " + c + " objects.");
}

Happy hacking,

James

PS: Best friends forever if you can get firefox to natively integrate with the gnome-keyring. No I don’t want to force it to myself, get this code merged upstream please!

Overriding attributes of collected exported resources

This post is about a particularly elegant (and crucial) feature in puppet exported resources: attribute overriding. If you’re not already familiar with exported resources, you should start there, as they are the killer feature that makes configuration management with puppet awesome. (I haven’t found any explicit docs about this feature either, so feel free to comment if you know where they’re hidden.)

Setup: I’ve got a virtual machine which exports a resource to N different nodes. I’d like to define the resource with just one exported (@@) definition on my virtual machine.

Problem: One (or more) of the attributes needs to be changed based on which node it gets collected on. To make things more complicated, I’m using the same class definition on each of those N nodes to collect the resource. I don’t want to have to write N separate node definitions:

@@some::resource { 'the_name':
    foo => 'bar',
    #abc => 'different_on_each_node',
    tag => 'magic',
}

Solution: It turns out that for exported (or virtual) resources, you can specify attributes that get set upon collection. Naturally they can depend on a variable such as $name, which is unique to where they get collected:

Some::Resource <<| tag == 'magic' |>> {
    abc => "node-${name}",    # override!
}

Bonus: You can obviously use other variables throughout including in the collection (tag == ‘magic’) area, on both the source and the destination. Instead of a simple equality like I’ve used, you can actually specify a more complex expression, including other variables such as title (the $name).

Hope this takes your puppet coding to another level,

Happy hacking,

James

 

Mothers day hacks

Firstly Happy Mother’s day to my mother.

Google is, as usual, busily releasing doodles. Today, the doodle takes you through a Rube Goldberg -esque sequence, giving you four decisions to make along the way. Each decision gives you one of three different choices, and at the end, a unique drawing is displayed. I expect:

3 * 3 * 3 * 3 = 81

different permutations. At the end of the process, you can print your image. I got directed to:

https://www.google.ca/logos/2013/mom/print/3311.html

which suspiciously has a four digit filename composed of three’s and ones. Inspecting the file in firefox shows that the main image url is:

https://www.google.ca/logos/2013/mom/hdcards/3311.jpg

with a similarly suspicious “3311“. I decided that I wanted to see all the permutations without having to refresh google’s page. Jumping to a terminal, and using some bash expansion magic:

$ cd /tmp
$ mkdir hack1
$ cd hack1/
$ wget https://www.google.ca/logos/2013/mom/hdcards/{1..3}{1..3}{1..3}{1..3}.jpg
[snip]
FINISHED --2013-05-12 07:07:22--
Total wall clock time: 1m 23s
Downloaded: 81 files, 138M in 1m 19s (1.75 MB/s)

I am pleased to see that 81 files have downloaded, and that they’re all unique:

$ md5sum * | sort | uniq | wc -l
81

and all png’s:

$ file * | awk '{print $2}' | sort | uniq | wc -l
1
$ file * | head -1
1111.jpg: JPEG image data, EXIF standard

Now I can very easily browse through the images with eog, and select my favourite.

Hope this has been instructive,

Happy hacking,

James

 

Upgrading to centos 6.4 with shorewall onboard

In case you upgrade your CentOS 6.x box to version 6.4, the shorewall service might complain. With a scary message:

ERROR: Your kernel/iptables do not include state match support.
No version of Shorewall will run on this system

This is selinux at work, and the problem can easily be solved by running:

# restorecon -Rv /sbin

Thanks shorewall-users and

Happy hacking,

James

 

Fixing jerky scrolling in Firefox

Fedora did a lovely job of updating me to the latest version (v. 20) of Firefox. One problem I found, was that scrolling on certain pages was quite jerky. Performance was worse (or more likely) on pages with a frameset, and pages which were long. Pages with many images made this problem worse.

It turns out that the workaround is to disable hardware acceleration:

After you’ve unchecked this box, restart Firefox, and scrolling is now considerably smoother.

Hopefully this helped you out. Most likely there is some driver issue or deficiency with the X drivers. I’m using an excellent Thinkpad X201. I’ve also had at least two cases of X freezing while I was manipulating a Firefox window, so perhaps this is related, and hopefully this won’t happen to me anymore.

Happy hacking,

James

 

Knowing when to release and deploy your code (…and a mini script)

Knowing when to release and deploy your code can turn into a complicated discussion. In general, In general, I tend to support releasing early and often, for some value of $early and $often. I’ve decided to keep this simple and introduce you to one metric that I use…

I think that I am fairly diligent in adding plenty of comments to my source code. I might even sometimes add too many. I create plenty of XXX, FIXME, or TODO tagged comments as reminders of things to work on.

To me, XXX represents an important problem that should get looked at or fixed; FIXME, reminds me that I should definitely look into something, and finally, TODO gives me homework or things to pursue when I’m in need of a new project.

I try to resolve most if not all XXX tagged comments before making a 0.1 release, FIXME’s to consider something very stable, and a lack of TODO’s mean something is completely done for now.

To count all these, I wrote a little tool that greps through the top-level directories in my ~/code/ folder, and displays the results in a table. Feel free to give it a try, and use it for your own projects.

While I don’t see this as a particularly game changing utility, it scratches my itch, and helps me keep up my bash skills. The code is available here. Let me know if you have any improvements, or if the source isn’t enough documentation for you.

Happy hacking,

James

running your file manager from a terminal

I do a lot of my work in a terminal. For the unfamiliar, this might seem strange, however once you’re comfortable with your shell, this is the best place to be. I don’t restrict myself to it though. I often want to spawn a file manager, or a graphical text editor. When I run nautilus, I usually see something like this:

james@computer:~/some/awesome/directory$ nautilus .
Initializing nautilus-open-terminal extension
Shutting down nautilus-open-terminal extension
james@computer:~/some/awesome/directory$

This is useful, because I can open a file browser right where I want it, it’s annoying, because nautilus runs in that terminal until I close it. (This doesn’t happen if the nautilus process is always running, but since GNOME 3, it isn’t.)

My solution is a short bash script that runs nautilus, and leaves your terminal alone. I named my script nautilus, and placed it inside my ~/bin/. Here is the script:

#!/bin/bash
# run nautilus from a terminal, without being attached to it; similar to nohup
# use the full path of nautilus to avoid it calling itself (recursion!)
{ `/usr/bin/nautilus "$@" &> /dev/null`; } &

I hope this is useful for you too. Feel free to do the same for gedit, nemo, and any other app which you often find convenient to run from the terminal. You can generalize this by leaving out the nautilus program:

#!/bin/bash
if [ "$1" == "" ]; then
        echo "usage: ./"`basename $0`"  (to run a command nohup style)"
        exit 1
fi
# do a nohup bash style according to:
{ `"$@" &> /dev/null`; } &

I name the above script run.sh, and it helps me out from time to time, when I don’t want to touch my mouse.

In case you haven’t heard about it, there’s also an open-terminal extension for nautilus and nemo which lets you get to a terminal, from your file manager. A quick internet search should help you install it.

If you found this information useful, please let me know, and as always,

Happy hacking,

James

PS: If you plan to do this for gedit, you probably want to preserve stdin, so that you can still pipe things in. To do this, you’ll probably want:

{ `/usr/bin/gedit "$@" &> /dev/null`; } < /dev/stdin &    # accept stdin too!

adding range support to python’s http server to kickstart with anaconda

I’ve been working on automatic installs using kickstart and puppet. I’m using a modified python httpserver because it’s lightweight, and easy to integrate into my existing python code base. The server was churning away perfectly until anaconda started downloading the full rpm’s for installation. What was going wrong?

Traceback (most recent call last):
[...]
error: [Errno 32] Broken pipe
BorkedError: See TTBOJ for explanation and discussion

As it turns out, anaconda first downloads the headers, and then later requests the full rpm with an http range request. This second range request which begins at byte 1384, causes the “simple” httpserver to bork, because it doesn’t support this more elaborate feature.

After a bit of searching, I found rangehttpserver and was very grateful that I wouldn’t have to write this feature myself. This work by smgoller was based on the similar httpserver by xyne. Both of these people have been very responsive and kind in giving me special permission to the relevant portions of their code that I needed under the GPLv2/3+. Thanks to these two and their contribution to Free Software this let’s us all see further, instead of having to reinvent previously solved problems.

This derivative work is only one part of a larger software release that I have coming shortly, but I wanted to put this out here early to thank these guys and to make you all aware of the range issue and solution.

Thank you again and,
Happy Hacking,

James

learn how to do one minute hacks, in three minutes

I write this technical blog for you to enjoy, and to help me remember. So where do I get all this knowledge? I figure it out! Here’s how I learned to fix a small gedit annoyance in one minute, and within the next three, you’ll be able to do the same for other types of problems too. Ready? Set? Go!

I use gedit enough, that when I hack, I often end up using up more than the five allotted spaces in the “recent files” sections. I wanted to see eight. Since I knew it would have been silly for the developers to hard code the number five, I decided there was a chance that they stored it in the dconf settings. (BTW, there’s also an amazing “Dashboard” plugin which I use for more complex recent-files searching…)

Enter dconf-editor. Run this, and start browsing through the hierarchy. You’ll notice that the org.gnome.* hierarchy has a lot going on. Look around, and you’ll find a “gedit” section. Once there, you’ll probably recognize some of the key names, as preferences you’ve seen. I searched for the number 5 and I found it next to a ‘max-recents’ key.

You can edit this with the editor, or for your convenience, just run:

gsettings set org.gnome.gedit.preferences.ui max-recents 8

the corresponding ‘read’ command is:

gsettings get org.gnome.gedit.preferences.ui max-recents

of course. The interesting thing about these settings, is that if coded properly, their actions are “live”. Which means, you can toggle them on and off, and in most cases, you’ll see the results immediately. Similarly, if you toggle a particular setting in gedit, you should see the changes instantly in dconf-editor.

Have fun playing with this and,

Happy hacking,

James