Sync Google Chrome Search Engines

Unfortunately, as of Chrome 8, search engines are not synced as part of the browser sync feature. When dev builds corrupt one's profile, it's obnoxious to manually recreate all keyword searches. Here's the process I used this morning, on OS X, for importing search engines into a new profile:

• cd Library/Application\ Support/Google/Chrome/Default
• sqlite3 Web\ Data
• .mode list
• .separator |
• .output /tmp/keywords.psv
• select * from keywords;
• .exit
• edit /tmp/keywords.psv to remove Google, Yahoo and Bing (they're included in new profiles)
• delete the old Default directory
• start Chrome
• configure Sync
• close Chrome
• sqlite3 Web\ Data
• .mode list
• .separator |
• .import /tmp/keywords.psv keywords

That's it. If you're importing from an older profile, you might have to manually adjust the export file so that it matches the new keywords table schema.

Dynamically Adding Columns with DBIx::Class

I have a DBIx::Class object representing an eBay auction. The underlying table has a description column which contains a lot of data. The column is almost never used, so it's not included in the DBIx::Class row object definition. I do, however, have one script that needs this column. I used to have code like

my $rs = $schema->resultset('Lots');
my $lots = $rs->search(
   { bar => undef },
   { '+select' => 'description', '+as' => 'description' },
);
...
$lot->update({ description => '...' });

but after upgrading to DBIx::Class version 0.08123 (from a really old version), it started giving the error

DBIx::Class::Relationship::CascadeActions::update(): No such column description at ...

Since this code runs inside an independent cron job, I can dynamically modify the schema without affecting any of the other scripts or the main website.

my $rs = $schema->resultset('Lots');
$rs->result_source->add_columns('description');
MyApp::Schema::Lots->add_columns('description');
MyApp::Schema::Lots->register_column('description');
my $lots = $rs->search({ bar => undef });
...
$lot->update({ description => '...' });

Since DBIx::Class doesn't support lazily loaded columns, this seems to be the best available workaround at the moment.

HDBC-Postgresql on Snow Leopard

The Problem

After upgrading to a new Mac Mini (running Snow Leopard on a 64-bit architecture), I installed the Haskell Platform. I then used cabal to install HDBC and HDBC-postgresql. I used Homebrew to install PostgreSQL. Everything installed fine.

I compiled a small Haskell script I'd been writing on my old Mac (32-bit architecture). It compiled fine, but when I ran a script that tried connecting to PostgreSQL, I got the following error.

can't load .so/.DLL for: pq (dlopen(/usr/local/Cellar/postgresql/8.4.4/lib/libpq.dylib, 9): no suitable image found.  Did find:
/usr/local/Cellar/postgresql/8.4.4/lib/libpq.dylib: mach-o, but wrong architecture)

Obviously, there was a problem with 32 vs 64 bit architectures. libpq, which I had built on the 64-bit machine, was a 64-bit build:

$ file /usr/local/Cellar/postgresql/8.4.4/lib/libpq.5.2.dylib
/usr/local/Cellar/postgresql/8.4.4/lib/libpq.5.2.dylib: Mach-O 64-bit dynamically linked shared library x86_64

but GHC was 32-bit:

$ file /Library/Frameworks/GHC.framework/Versions/612/usr/lib/ghc-6.12.3/ghc
/Library/Frameworks/GHC.framework/Versions/612/usr/lib/ghc-6.12.3/ghc: Mach-O executable i386

Incidentally, 64-bit builds of GHC on OS X are still pending.

Failed Attempts

My first thought was to simply rebuild PostgreSQL as a universal binary. That way Haskell could load the 32-bit version it needed and everything else could stay 64-bit. I knew that Perl was a universal binary:

$ file /usr/bin/perl
/usr/bin/perl: Mach-O universal binary with 3 architectures
/usr/bin/perl (for architecture x86_64):        Mach-O 64-bit executable x86_64
/usr/bin/perl (for architecture i386):  Mach-O executable i386
/usr/bin/perl (for architecture ppc7400):       Mach-O executable ppc

According to perl -V, the necessary CFLAGS and LDFLAGS options are -arch i386 -arch x86_64 -arch ppc. Several attempts invoking Homebrew with those options in the environment failed. I came across some tips about building a PostgreSQL universal binary. I tried several variations of his latest build script, but wasn't able to make it work.

The tricks for building a universal binary seemed easy enough, so I decided to modify the Homebrew formula using those tricks. In the process, I discovered that Homebrew has a convenient way for enabling universal builds within a formula. Just add to the formula the line:

ENV.universal_binary if ARGV.include? '--universal'

and universal binaries are built automatically. Even with this change (and rebuilding OSSP-UUID as a universal binary), I still got compiler errors when building PostgreSQL. The problem appeared to be the way that certain macros were expanded when compiling for multiple architectures.

Final Solution

At this point, I gave up on building from source. I came across the handy PostgreSQL installer for OS X. It installs a 32-bit version of PostgreSQL. After the installation, I linked libpq into /usr/local/lib and now everything works fine:

cd /usr/local/lib
ln -s /Library/PostgreSQL/8.4/lib/libpq.5.2.dylib  libpq.dylib

Although I might encounter problems down the road without a 64-bit PostgreSQL, this solution got me past the roadblock and taught me a lot about compiling for multiple architectures.

Blocking Status Updates from Facebook Apps

Facebook is useful for staying in touch with friends and getting an idea what people are up to. Unfortunately, many Facebook applications post status updates on behalf of their users. To avoid wading through piles of status updates from Pillow Fight or FarmVille or ... I enlisted the help of the Google Chrome extension AdThwart. That extension does a good job of blocking ads. You can also use it to block arbitrary HTML elements in a page by using CSS selectors. Here's how I block those verbose Facebook apps:

• in AdThwart's configuration options, choose "Add your own filters"
• Add: facebook.com##div[data-ft*="app_id"]

P.S. Blocking the offending application doesn't alone solve the problem. Blocking prevents invites but doesn't hide the auto-posted status updates.

Snowbirding with Prolog

This time of year, when it's so cold in Wyoming I'm tempted to snowbird and wait out the chilly months somewhere toasty. Of course, many places have warm weather, so how do I choose one and how do I get there? I could drive to Tucson but that means spending two days on the road (losing billable hours), buying meals, getting a hotel, etc. I could fly to Tucson, but then I have to rent a car. Vegas has cheap flights and rental cars, maybe that's more economical. There are so many variations and each possible choice can affect other choices.

I've been learning Prolog and this seemed like a great chance to use it. I've coded up a simple snowbirding model and am pretty pleased with the results. It's really easy to add new cities and new possibilities. For instance I initially had rental cars as the only local transportation option after flying to a city. With a few lines I added options for Zip Car and public transit. Running the program instantly accounted for those two new possibilities in cities where they applied.

Here's a sample of the first 10 snowbirding options produced by the program:

1361.28 st_george [drive, drive]
1853.84 vegas [drive, drive]
2218.51 mesquite [drive, drive]
3441.47 vegas [taxi, fly, buses, fly]
3836.91 amarillo [drive, drive]
3861.47 vegas [park(long), fly, buses, fly]
4341.47 vegas [park(airport), fly, buses, fly]
5000.83 tucson [drive, drive]
5177.63 tucson [taxi, fly, buses, fly]
5597.63 tucson [park(long), fly, buses, fly]

The first line tells us that driving to St. George, UT is the least expensive option at $1,361.28 per year (housing and food costs are not included yet). The 10th most expensive option is using long-term parking at the Denver airport, flying to Tucson, using public transit while I'm there and flying home. That option runs about $5,600 per year. All these calculations are for a 3 month stay for a family of 6.

It seems like there are lots of economic decisions which could be modeled this way and for which Prolog is a natural fit. The full Prolog code is available and included below. I'm new to the language, so I gladly welcome any suggestions for improvement.

PostgreSQL Really Slow During Backups

These are my notes from debugging a website problem caused by backing up the underlying database. Symptom: When backing up the database, the website is very sluggish The old backup technique amounted to

pg_dump database > file.sql \
&& rysnc file.sql remote:file.sql

I removed the rsync component and the symptom persisted. Obviously that wasn't the cause. Based on a hint in the PostgreSQL mailing list, I reduced the total IO load by completely eliminating writes with

pg_dump database > /dev/null

That fixed the problem but unfortunately it makes a lousy backup strategy. By compressing the SQL before it hits the disk, I was able to reduce the total IO load and the symptom went away. The final backup incantation amounts to

pg_dump database | gzip --rsyncable > file.sql.gz \
&& rysnc file.sql.gz remote:file.sql.gz

SMS MIME Type

Background

The MIME standards specify a Content-Type header for indicating the kind of information included in a MIME part. They also specify the multipart/alternative type which allows one to specify multiple representations of a single message.

For example, you can send an email message with a text/plain part and a text/html part. Each part contains essentially the same content but the latter is an HTML representation and the former is a plain text representation. If the recipient doesn't understand HTML, he can show the text version. A sender can craft each part to provide the best representation possible for each particular format.

Use Case

Nearly every mobile phone provider has an email to SMS gateway. Anywhere an email address is used, I can provide my gateway email address and receive the email as a text message. The problem is that neither the phone provider nor my phone knows how to summarize an arbitrary email into a 160 character text message. It does the best it can, but important information is almost always lost.

I would like to see a MIME content type for SMS messages: text/sms. One could then provide an SMS alternative along with HTML, plain text and others. The one sending the email — who presumably knows what information in the email is most important — would perform the summary. The end result would be much more useful SMS representations of emails.

It looks like there's an application/vnd.3gpp.sms which might be intended for this use although I can't tell for sure.