the python subprocess module

i’m sure that i won’t be able to tell you anything revolutionary which can’t be found out by reading the manual, but i thought i would clarify it, and by showing you a specific example which i needed.

subprocess.Popen accepts a bunch or args, one of which is the shell argument, which is False by default. If you specify shell=True then the first argument of popen should be a string which is what gets parsed by the shell and then eventually run. (nothing revolutionary)

the magic happens if you use shell=False (the default), in which case the first argument then accepts an array of arguments to pass. this array exactly transforms to become the sys.argv of the subprocess that you’ve opened with popen. magic!

this means you could pass an argument like: “hello how are you” and it will get received as one element in sys.argv, versus being split up into 4 arguments: “hello”, “how”, “are”, “you”. it’s still possible to try to do some shell quoting magic, and achieve the same result, but it’s much harder that way.


>>> _ = subprocess.Popen(['python', '-c', 'print "dude, this is sweet"'])
>>> dude, this is sweet

vs.


>>> _ = subprocess.Popen("python -c 'print "dude, this isnt so sweet"'", shell=True)
>>> dude, this isnt so sweet

and i’m not 100% sure how i would even add an ascii apostrophe for the isn’t.

the second thing i should mention is that you have to remember that each argument actually needs to be split up; for example:


>>> _ = subprocess.Popen(['ls', '-F', '--human-readable', '-G'])
[ ls output ]

yes it’s true that you can combine flags into one argument, but that’s magic happening inside the program.

all this wouldn’t be powerful if we couldn’t pipe programs together. here is a simple example:


>>> p1 = subprocess.Popen(['dmesg'], stdout=subprocess.PIPE)
>>> p2 = subprocess.Popen(['grep', '-i', 'sda'], stdin=p1.stdout)
[ dmesg output that matches sda ]

i think it’s pretty self explanatory. now let’s say we wanted to add one more stage to the pipeline, but have it be something that usually gets executed with os.system:


p1 = subprocess.Popen(['dmesg'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['grep', '-i', 'sda'], stdin=p1.stdout)
p3 = subprocess.Popen(['less'], stdin=p2.stdout)
sts = os.waitpid(p3.pid, 0)
print 'all done'

this above example should all be pasted into a file and run; the call to waitpid is important, because it stops the interpreter from continuing on before less has finished executing.

hope this took the learning curve and guessing out of using the new subprocess module, (even though it actually has existed for a while…)

cheetah == fortran

turns out the cheetah python templating engine (2.0 since year 2006) is quite reminiscent of fortran (since the 1950’s) in their use of the #slurp directive (cheetah) and the $ string. either one, when appended to the end of a string, remove the implicit newline which usually gets printed. it took me ages to figure out how to suppress newline printing back when i did someone’s fortran homework, now i’ve had to struggle with it all over again.

i’m not a language designer, but it never seemed like the best idea to me! but what do i know? i hope this saves someone an hour of searching.

[py]inotify, polling, gtk and gio

i have this software with a gtk mainloop, using dbus and all that fun stuff that seems to play together nicely. i know about the kernel inotify support, and i wanted it to get integrated with that above stack. i thought i was supposed to do it with pyinotify and io_add_watch, but on closer inspection into the pyinotify code it turns out that it seems to actually use polling! (search for: select.poll)

thinking i was confused, i emailed a friend to see if he could confirm my suspicions. we both weren’t 100% sure, a little searching later i was convinced when i found this blog posting. i’m surprised i didn’t find out about this sooner. in any case, my application seems to be happy now.

as a random side effect, it seems that when a file is written, i still see the G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED *after* the G_FILE_MONITOR_EVENT_CHANGES_DONE_HINT event, which i would have expected to always come last. maybe this is a bug, or maybe this is something magical that $EDITOR is doing- in any case it doesn’t affect me, i just wasn’t sure if it was a bug or not. to make it harder, different editors save the file in different ways. gedit seems to first delete the file, and then create it again– or at least from what i see in the gio debug.

the code i’m using to test all this is:

#!/usr/bin/python
import gtk
import gio
count = 0
def file_changed(monitor, file, unknown, event):
  global count
  print 'debug: %d' % count
  count = count + 1
  print 'm: %s' % monitor
  print 'f: %s' % file
  print 'u: %s' % unknown
  print 'e: %s' % event
  if event == gio.FILE_MONITOR_EVENT_CHANGES_DONE_HINT:
    print "file finished changing"
  print '#'*79
  print 'n'
myfile = gio.File('/tmp/gio')
monitor = myfile.monitor_file()
monitor.connect("changed", file_changed)
gtk.main()

(very similar to the aforementioned blog post)

and if you want to see how i’m using it in the particular app, the latest version of evanescent is now available. (look inside of evanescent-client.py)

logging out of $SESSION

the software (evanescent) that i’m working on is supposed to log out the user from its current X session. originally i had some yucky looking code that ran a kill on gnome-session, which quickly got replaced with:

os.system('gnome-session-save --logout-dialog')

i’ve decided this was still a little crufty, so i’ve recently replaced this with:

bus = dbus.SessionBus()
remote_object = bus.get_object('org.gnome.SessionManager', '/org/gnome/SessionManager')
# specify the dbus_interface in each call
remote_object.Logout(0, dbus_interface='org.gnome.SessionManager')
# or create an Interface wrapper for the remote object
#iface = dbus.Interface(remote_object, 'org.gnome.SessionManager')
#iface.Logout(0)
# introspection is automatically supported
#print remote_object.Introspect(dbus_interface='org.freedesktop.DBus.Introspectable')

documentation can be found, although it took a little digging. the only catch is that this is gnome specific, and you need different code for kde, and each other DE. thanks to http://www.purinchu.net/wp/2009/06/12/oh-fun/ for the kde version of the above.

help! please contribute information on making this work on other platforms, and/or how to detect which platform is running. at the moment my ugly solution is:

if os.name == 'nt': return 'windows'
elif os.getenv('KDE_FULL_SESSION') == 'true': return 'kde'
elif os.getenv('GNOME_DESKTOP_SESSION_ID') != '': return 'gnome'
else: return ''

hth

ps: i’m unable to indent code in wordpress (i’m sure it’s my fault somehow) so the above code was trimmed to give you the right idea, without being complete. leave a message if you want some source files.