Matching arbitrary URL’s to custom Firefox profiles

We’re constantly clicking on all sorts of different URL’s throughout the day. These clickable links appear in webpages (including in “web apps” like gmail) in mail clients like Evolution, in terminals such as GNOME-terminal, and any other GTK+ app on your GNU/Linux desktop. I wanted to perform custom actions when arbitrary URL’s are clicked, including running certain links in separate Firefox profiles. There are a bunch of different steps you have to do to get this working, but it should be easy to follow along. I’m doing all of this on Fedora 23, but it should work on other GNU/Linux environments.

Firefox profiles:

Firefox supports multiple profiles in the same user session so that different users can share a session, or so that a single user can separate tasks into different environments. I’m interested in the latter use case. To add a new profile it’s recommended to close firefox completely, but I didn’t find this to be necessary. When I do close firefox, I like to surprise it with a:

killall -9 firefox

which will also cause any unsaved data in your browser to be lost. To create a new profile, now run firefox with -P:

firefox -P

This will open up a friendly dialog where you can add a new profile. After you’ve done this, my dialog now looks like:

A view of my firefox profiles as shown by running: firefox -P

A view of my firefox profiles as shown by running: firefox -P

to test that it is working, run firefox from the command line:

$ firefox https://ttboj.wordpress.com/
$ firefox -P ghttps https://github.com/purpleidea/
$ firefox https://twitter.com/#!/purpleidea
$ firefox -P ghttps https://www.gnu.org/philosophy/free-sw.html

You should get two separate sessions, where the commands with -P ghttps should be in your new “ghttps” session (or whatever you named it). Internet searches seem to report that some users can’t run two sessions at the same time without including the --no-remote option, but I didn’t seem to need it. YMMV.

Firefox launcher:

When you run firefox, it usually runs /usr/bin/firefox. I want a more clever launcher, so I’ve created a new bash script named ~/bin/firefox which is part of my path. The contents of this file are:

#!/bin/bash
# run firefox from a terminal, without being attached to it; similar to nohup
# thanks to @purpleidea from https://ttboj.wordpress.com/
# TODO: a better argv parser and more flexible url matching semantics
# NOTE: first close firefox and make a new profile with `firefox -P`, then set:
protocol='ghttps' # name of fake protocol
profile='ghttps' # name of your new firefox profile
prefix='https://example.com/'
argv=("$@")
argc=$(($# - 1))
url=''
if [ $argc -ge 0 ]; then
    url=${argv[$argc]}
    # avoid recursion!
    if [[ "$url" == "$protocol"* ]]; then
        url="https${url:${#protocol}}" # s/ghttps/https/
        argv[$argc]=$url # store it
    fi
fi

#echo $url
#echo "`date` ${argv[*]}" >> /tmp/firefox.log

# use a separate profile for special links
if [ "$url" != "" ] && [[ "$url" == "$prefix"* ]]; then
    # firefox with profile
    { `/usr/bin/firefox -P "$profile" "${argv[@]}" &> /dev/null`; } &
else
    # normal firefox
    { `/usr/bin/firefox "${argv[@]}" &> /dev/null`; } &
fi

Make sure the file is executable with chmod u+x ~/bin/firefox and in your $PATH. Now whenever you run the firefox program, it will automatically run firefox with a profile that corresponds to the pattern of URL that you matched. Feel free to improve this script with a more comprehensive pattern to profile correspondence mechanism.

Default applications:

Whenever any URL is clicked within GNOME, there is a central “Default Applications” setting which decides what application to run. My settings dialog for this control now looks like:

What the GNOME Settings->Details->Default Applications dialog looks like after I made one small change.

What the GNOME Settings->Details->Default Applications dialog looks like after I made the change.

I had to change the “Web” handler to be a “MyFirefox” instead of the previous default of “Firefox”. Those applications are listed in .desktop files which exist on your system. The system wide firefox desktop file is located at: /usr/share/applications/firefox.desktop and although the path to the executable in this file does not set a directory prefix, it unfortunately does not seem to obey my shell $PATH which includes ~/bin/. If you know how to set this so .desktop files include ~/bin/ in their search, then I’d really appreciate it if you left me a comment!

To work around the $PATH issue, I copied the above file into ~/.local/share/applications/firefox.desktop and edited it so that the three Exec= commands include a path prefix of /home/james/bin/. I also renamed the Name= entry so that it was visually obvious that a different .desktop file was in use. This will replace the firefox launcher throughout your desktop and as well in the “Default Applications” menu.

An excerpt of my file showing only the changed sections now looks like:

[Desktop Entry]
Name=MyFirefox
Comment=Browse the Web better
Exec=/home/james/bin/firefox %u
Actions=new-window;new-private-window;

[Desktop Action new-window]
Name=Open a New Window
Exec=/home/james/bin/firefox %u

[Desktop Action new-private-window]
Name=Open a New Private Window
Exec=/home/james/bin/firefox --private-window %u

Changing the name is optional, but it might be instructional for you.

It’s important that you not rename the file, because only files which are listed in one of the GNOME mime list files will show up in the “Default Applications” chooser. Once you’ve created the file, you can check in these settings to ensure it’s set as the default.

I forget if you need to close firefox, and logout and then back in to your GNOME session for this to work, so if things aren’t working perfectly by now, ensure you’ve done that once. You can test this by clicking on a link in your terminal and checking to see that it opens the correct firefox.

Redirecting internal firefox links:

Everything should now be working perfectly, until you click on a link within firefox which doesn’t redirect to your shell firefox wrapper. We want this to be seamless, so we’ll have to hack into the firefox API for that. Thankfully there’s a plugin which already does this for us, so we can use it and avoid getting our hands too dirty! It’s called “Redirector“. Install it.

Once installed, there is a settings dialog which can add some pattern matching for us. I set up a basic pattern that corresponds to what I wrote in my ~/bin/firefox shell script. Here’s a screenshot:

A screenshot from the firefox Redirector plugin.

A screenshot from the firefox Redirector plugin.

You can conveniently import and export your redirects from the plugin, and so I’ve included the corresponding .json equivalent for your convenience.

Does everything look correct? Take a second to have a closer look. You might think that I made a typo in the “Redirect to:” field”. There’s no such protocol as ghttps you say? That’s good news, because its use was intentional.

Custom protocol handlers:

Running an external command in response to certain links is what allows them to open external programs such as mail clients, PDF viewers, and image viewers. While some of these functions have been pulled into the browser, the need is still there and this is what we’ll use to trigger our firefox shell script. It’s actually important that we make an external system call because otherwise there would no way for a link in the default browser profile to open in browser profile number two. Running any such command is only possible with a custom or unique protocol. You might be used to seeing https:// for URL’s, but since these are captured by the browser as native links, we need something different. This is what the ghttps:// that we mentioned above is for.

To add a custom protocol, you’ll need to dive into your browsers internal settings. You can do this by typing about:config in the URL bar. You’ll then need to right-click and add four new settings. These are the settings I added:

(string)  network.protocol-handler.app.ghttps; /home/james/bin/firefox
(boolean) network.protocol-handler.expose.ghttps; false
(boolean) network.protocol-handler.external.ghttps; true
(boolean) network.protocol-handler.warn-external.ghttps; true

Please note that the leading values (in brackets) are the types that you’ll need to use. Omit the semicolons, those separate the key and the corresponding value you should give it. You’ll naturally want to use the correct path to your firefox script.

For reasons unknown to me, it’s required to set these variables, but the protocol handler still requires that you manually verify this once. To do this, I have provided a sample link to my blog using the fake ghttps protocol:

ghttps://ttboj.wordpress.com/

When you click on it the first time, you should be prompted with a confirmation dialog that asks you to reconfirm that you’re okay running this protocol, and the path to the executable. Browse to the ~/bin/firefox and click “Remember my choice for ghttps links”. The dialog looked like this:

You should only need to deal with this dialog once!

You should only need to deal with this dialog once!

If you’re using a different protocol, can you make a simple HTML file and open it up in your browser:

<html>
<a href="ghttps://ttboj.wordpress.com/">ghttps://ttboj.wordpress.com/</a>
</html>

At this point you may need to restart firefox. Your new protocol handler is now installed! Enjoy automatically handling special URL’s.

Bugs:

There is one small usability bug which you might experience. If the link that should pattern match out to the protocol exists with a target=_blank (open in new window attribute) then once you’ve activated the link, there will be a leftover blank firefox window to close. This is a known issue in firefox which occurs with other handlers as well. If anyone can work on this issue and/or find me a link to the ticket number, I’d appreciate it.

Curiosity:

The curious might wonder what my use-case is. I’ve been forced to use the most unpleasant online google document system. I’ve decided that I didn’t want to share my regular browser profile with this software, but I wanted URL integration to feel seamless, since people like to send the unique document URL’s around by email and chat. The document URL’s usually follow a pattern of:

https://docs.google.com/a/example.com/some-garbage-goes-here…

where example.com is the domain your organization uses. By setting the above string as the bash firefox $prefix variable, and with a similar pattern in the redirector plugin, you can ensure that you’ll always get documents opening up in browser sessions connected to the correct google account! This is useful if you have multiple google accounts which you wish to automatically segregate to avoid having to constantly switch between them!

Future work:

It would be great to consolidate the patterns as expressed in the Redirector database and the firefox bash script. It would probably make sense to generate a json file that both tools can use. Additional work to extend my bash script would be necessary. Patches welcome!

It would be convenient if there was an easy setup script to automate through the myriad of steps that I took you through to get this all working. If someone can provide a simple bash equivalent, I would love to have it.

Conclusion:

I hope you enjoyed this article and this set of techniques! Hopefully you can appreciate how stringing five different techniques together can produce something useful. A big thank you goes out to SlashLife from the #firefox IRC channel. This user pointed me to the Redirector plugin which was critical for intercepting arbitrary URL’s in firefox.

Happy Hacking,

James

PS: I’d like to apologize for not posting anything in the last three months! I’ve been busy hacking on something big, which I’ll hope to announce soon. Stay tuned and thanks for reading this far!

Replying to mailing lists with Evolution

I use the Evolution mail client. It does have a few annoying bugs, but it has a plethora of great features too! Hopefully this post will inspire you to help hack on this piece of software and fix the bugs!

Mailing list etiquette:

When replying to mailing lists, it’s typically very friendly to include the email address of the person you’re replying to in the to or cc fields along with the mailing list address. This lets that person know that someone has answered their question. In some cases, if they’re not subscribed to that mailing list, (if you don’t do this), then they might not see your reply at all.

To enable this feature, there is a check box inside of the Evolution mail preferences. It is labelled: “Ignore Reply-To: for mailing lists“.

You can find this option in the Evolution "Composer Preferences" tab, under the "Replies and Forwards" heading.

You can find this option in the Evolution “Composer Preferences” tab, under the “Replies and Forwards” heading.

This works, because by default, most mailing lists set the “Reply-To:” address to be that of the mailing list. In this case, when you click “Group Reply” (“Reply to all”) in your MUA, then that field will be ignored, and the correct recipients will be selected in your composer window.

If instead you simply click “Reply”, then you will be prompted to choose the kind of reply you’d like to send.

evolution-send-private-reply

Doesn’t this annoy users?

No, this actually gives the recipients more choice! If they’d prefer not to see your reply in their inbox, they can set up a filter so that mail that includes the mailing list address goes to a special folder. If they prefer to see your reply in their inbox, then they can configure their filters so that mail that comes exclusively from the mailing list address goes to a specific folder.

Instead of choosing the "contains" (in_array) operator, you could have chosen "is" (equals).

Instead of choosing the “contains” (in_array) operator, you could have chosen “is” (equals).

Won’t this cause duplicate messages being sent to the user?

Again, that’s up to the user. Most mailing lists allow you to configure this setting.

In this particular example, it is a very low-volume list, therefore I don't filter messages into a separate folder, they go to my inbox, so there's no need to get two copies.

This particular example is of a very low-volume list, therefore I don’t filter messages into a separate folder; they go to my inbox, so there’s no need to get two copies.

Ultimately, Evolution is a great MUA, which has the best message composer available. Some might prefer mutt+vim, but for my money, I’ll stick with Evolution for now.

Happy hacking,

James

PS: If you hack on Evolution, and write a good feature that I like, or fix a bug that affects me, I’m happy to feature you on this blog and tweet about you as soon as your code hits git master! </free promotion for good hackers!>

Testing Evolution’s git master and GNOME continuous

I’ve wanted a feature in Evolution for a while. It was formally requested in 2002, and it just recently got fixed in git master. I only started publicly groaning about this missing feature in 2013, and mcrha finally patched it. I tested the feature and found a small bug, mcrha patched that too, and I finally re-tested it. Now I’m blogging about this process so that you can get involved too!

Why Evolution?

  • Evolution supports GPG (Geary doesn’t, Gmail doesn’t)
  • Evolution has a beautiful composer (Gmail’s sucks, just try to reply inline)
  • Evolution is Open Source and Free Software (Gmail is proprietary)
  • Evolution integrates with GNOME (Gmail doesn’t)
  • Evolution has lots of fancy, mature features (Geary doesn’t)
  • Evolution cares about your privacy (Gmail doesn’t)

The feature:

I’d like to be able to select a bunch of messages and click an archive action to move them to a specific folder. Gmail popularized this idea in 2004, two years after it was proposed for Evolution. It has finally landed.

In your account editor, you can select the “Archive Folder” that you want messages moved to:

evolution-account-archive-folder

This will let you have a different folder set per account.

Archive like Gmail does:

If you use Evolution with a Gmail account, and you want the same functionality as the Gmail archive button, you can accomplish this by setting the Evolution archive folder to point to the Gmail “All Mail” folder, which will cause the Evolution archive action to behave as Gmail’s does.

To use this functionality (with or without Gmail), simply select the messages you want to move, and click the “Archive…” button:

evolution-context-menu-archive

This is also available via the “Message” menu. You can also activate with the Control-Alt-a shortcut. For more information, please read the description from mcrha.

GNOME Continuous:

Once the feature was patched in git master, I wanted to try it out right away! The easiest way for me to do this, was to use the GNOME Continuous project that walters started. This GNOME project automatically kicks off integration builds of multiple git master trees for the most common GNOME applications.

If you follow the Gnome Continuous instructions, it is fairly easy to download an image, and then import it with virt-manager or boxes. Once it had booted up, I logged into the machine, and was able to test Evolution’s git master.

Digging deep into the app:

If you want to tweak the app for debugging purposes, it is quite easy to do this with GTKInspector. Launch it with Control-Shift-i or Control-Shift-d, and you’ll soon be poking around the app’s internals. You can change the properties you want in real-time, and then you’ll know which corresponding changes in the upstream source are necessary.

Finding a bug and re-testing:

I did find one small bug with the Evolution patches. I’m glad I found it now, instead of having to wait six months for a new Fedora version. The maintainer fixed it quickly, and all that was left to do was to re-test the new git master. To do this, I updated my GNOME Continuous image.

  1. Click on Control-Alt-F2 from the virt-manager “Send Key” menu.
  2. Log in as root (no password)
  3. Set the password to something by running the passwd command.
  4. Click on Control-Alt-F1 to return to your GNOME session.
  5. Open a terminal and run: pkexec bash.
  6. Enter your root password.
  7. Run ostree admin upgrade.
  8. Once it has finished downloading the updates, reboot the vm.

You’ll now be able to test the newest git master. Please note that it takes a bit of time for it to build, so it is not instant, but it’s pretty quick.

Taking screenshots:

I took a few screenshots from inside the VM to show to you in this blog post. Extracting them was a bit trickier because I couldn’t get SSHD running. To do so, I installed the guestfs browser on my host OS. It was very straight forward to use it to read the VM image, browse to the ~/Pictures/ directory, and then download the images to my host. Thanks rwmjones!

Conclusion:

Hopefully this will motivate you to contribute to GNOME early and often! There are lots of great tools available, and lots of applications that need some love.

Happy Hacking,

James

Upgrading from Fedora 18 to Fedora 19

It was time to take the plunge and upgrade from Fedora 18 to Fedora 19. Fedora 18 was one of the worst releases ever, so I figured it could only get better. I ran my backups as usual, however this time I didn’t seem to need them, the upgrade process went off without a hitch! I used the fedup-cli process over the network. I always run these things inside of screen.

Here are my post install notes and comments:

Brown folder icons:

Someone broke the icon theme, and folders are now an ugly brown. Even though you’ll see a “Fedora” entry in the GNOME tweak tool, icon theme section, it won’t work. You need to install a theme package first:

# yum install fedora-icon-theme

tweak tool will now let you fix the brown icon issue.

Dash doesn’t launch new windows:

The GNOME shell is back to its old habit of trying to imitate Mac OSX. Thankfully there’s an official extension to fix this: https://extensions.gnome.org/extension/600/launch-new-instance/

If you search for “Classic Mode” on https://extensions.gnome.org/ you’ll find some other useful add ons. You might enjoy: AlternateTab, AvoidOverview, and SystemMonitor.

Evolution is snappier:

Congratulations to the evolution developers, this release seems a bit snappier! I haven’t tested it thoroughly, however closing evolution now happens in under ten seconds! Usually it would either hang or take much longer to close. Keep up the good work!

Clocks deletes your old clocks:

The clocks application deleted all the clocks that I had added. I suppose there are worse forms of data loss, but this is still pretty unprofessional! I had added one for every new place I had visited. Goodbye memories!

YUM breaks pexpect scripts:

A new version of YUM, now prompts you differently:

# yum install foobar[...]
Is this ok [y/d/N]:

No it’s not okay that you’re confusing my brain by adding a d. I actually don’t have any pexpect scripts depending on this, but after years of seeing y/N, the change is not welcomed. It should have been handled with a YUM download target instead.

Password prompts are annoying:

The GNOME shell handles most of the password prompts. This makes sense because it can help prevent you typing your password into a chat room. The problem is that if you need to run an external password manager to find a password, you’re out of luck. Maybe someone can add an option to minimize the focus stealing window. In addition, the “remember password” checkbox should NOT be on by default! It still is for evolution, and perhaps other apps too.

GNOME shell isn’t smooth, but it’s better:

Fedora 17 provided a smooth GNOME shell experience. Fedora 18, somehow killed the performance, and no fixes could be found. The performance seems to be a bit better in Fedora 19, but it’s still not perfect. Drivers are probably partly to blame.

New version of Gedit breaks some things:

The Gedit dashboard plugin no longer seems to work. After debugging the issue, it seems to be a packaging problem. To fix:

# yum install python3-dbus

and dashboard should work when enabled. The gedit-autotab plugin is thoroughly broken. I’d love to get that working again for smarter spaces!

That’s all for now,

Happy hacking,

James