Category Archives: Linux

ImportError: No module named pysqlite2

Currently my favorite web app environment includes:

  • CentOS 5.5
  • Python 2.6 (built from source)
  • Pylons 1.0 (w/ SQLAlchemy 0.5.8)
  • nginx

Whenever I setup a new environment, and execute “paster serve –reload development.ini” for the first time, I always get a stack trace from SQLAlchemy that ends with the following error message:

ImportError: No module named pysqlite2

And it always takes me a few minutes to figure out how to fix it again. So I’m taking a few minutes to jot it down here so I don’t forget again!

yum install sqlite-devel -y
wget peak.telecommunity.com/dist/ez_setup.py
python2.6 ez_setup.py
easy_install pysqlite
Share

Using Linux to Get Internet Access on your Verizon Phone for Free

I recently dumped Sprint for Verizon because Sprint provides such shoddy reception in my area. When researching phones online, I came across a few mentions on how to get free Internet access on some of the models. However, because the suggestions included routing through public proxies, I opted not to do it. However, finally purchasing a phone, I decided to try rolling my own hack.

Before you begin

  1. You need to be using a recent Motorola phone or know how to get into your phone’s hidden programming menu. I’m using my new V325i, but this will work with at least 35 other Verizon phones.
  2. Despite the fact that Verizon probably doesn’t want you to know about this, you aren’t stealing anything from them; you’ll still inccur airtime charges. The $5/mo extra you’d normally pay for web access to essentially rent their proxy server. You’re just saving a few bucks by using your own proxy server instead.
  3. This will only get you standard web access, the hack may or may not work with V-Cast (Verizon’s EV-DO network) or MVNO providers like Amp’d that piggyback on Verizon’s network. I haven’t tested them, but I’m guessing not.

Why not use a public proxy server?

I first learned about the phone part of this project from various cell phone forums and blogs (such as this one). However, they all recommended setting my phone to use a randomly-chosen, public proxy server listed on the web. Either they don’t know or don’t care, but the idea of piping all of my web traffic through some unknown party gives me the willies. At a minimum, when and where I visit will be logged. At worst, every bit of data that is sent from or received by my phone will also be logged–including email. Meh… I’m a trusting soul, but not that trusting. I’d also prefer something a bit more reliable.

Why not Squid?

When you hear “proxy server” and “Linux” in the same sentence, someone is almost always talking about Squid. I’ve used Squid; I both know and like it, but it just doesn’t cut it for this project because it doesn’t (yet) fully support HTTP version 1.1. HTTP/1.1 has been around since 1999 and is supported by Netscape Navigator 4 / IE4.01, so I’m not really sure why it’s so far behind in the times. However, Squid will take HTTP/1.1 requests sent from your phone and pass them onto the web server as HTTP/1.0 requests. Even though servers aren’t supposed to reply with 1.1 features when sent a 1.0 request, many ignore this rule and respond with chunked transfer encoding. For reasons I still don’t understand, this chokes the web browser on my phone. If you use Squid for this project, your phone won’t connect to about half of the websites you attempt to access (including wap.google.com). Well… at least on my phone, YMMV.

Choosing and installing the proxy server

After a bit of experimenting, I decided to settle on Polipo. It’s small, fast, well-documented, free, and most importantly: it works with minimal hassle. I am also using a Fedora Core box with a public IP. If your server is behind NAT or a firewall you’ll have to figure out how to correctly set up port-forwarding and any other relavant workarounds. It generally isn’t too difficult, and there are plenty of HOWTOs and other resources out there that can help.

The first step is to obviously download and install Polipo. While there are a handful of RPMs out there, I decided just to compile it myself because it’s so simple. By following the single-page installation instructions you should have Polipo up-and-running in about 10 minutes.

Configuring Polipo

After you’ve got a basic install of Polipo up and running, we need to configure it with a few extra options. First, we’re going to have to create a configuration file. By default, Polipo looks for a file named “/etc/polipo/config”, so I’m just going to stick with that for now. Inside the file add the following lines:

proxyAddress = 212.75.25.169
allowedClients = 70.217.0.0/16
authCredentials = jamie:mypassword
redirector = /etc/polipo/homepage.pl

Out of the box, Polipo only binds to the server’s loopback address. The “proxyAddress” option contains the IP address we want to listen to instead. (Note: If you’re using NAT, this isn’t the public address, but your local IP address, probably 10.x.x.x or 192.168.x.x).

The next setting, “allowedClients” is the IP address and netmask of your cell phone. Through some experimentation, I’ve determined that my phone always connects from an IP address of 70.217.x.x, but yours will vary. I figured this out by first setting allowedClients to “0.0.0.0/0″ (which allows anyone to use your proxy). From my cell, I then accessed a website I manage and checked the site’s access logs. Another idea would be to write a quick Perl or PHP script that emails you the IP address of the visitor when the URL is accessed. You could leave it as “0.0.0.0/0″, but I wouldn’t recommend it for security reasons.

Along with allowedClients, “authCredentials” is another way to help keep spammers from hijacking your proxy. It’s simply a username and password seperated by a colon. It uses the basic authentication, so it isn’t super-secure, but should be good enough to keep out unwanted guests when used with “allowedClients”.

Finally, we have the “redirector” setting. This points to a quick little Perl script which contains the following:

#!/usr/bin/perl
$|=1;
while (<>) {
 s@http://homepage@http://wap.google.com@;
 print;
}

The script runs in a loop; if you input anything other than “http://homepage”, it simply echos back your input. Inputting “http://homepage”, however, will result in an output of “http://wap.google.com”. Change “http://wap.google.com” in the script to whatever homepage you want to use on your phone. I opted to use Perl because it’s easy and fast, but you can use something else if you’d like. (If your copy of the Perl interrupter isn’t in “/usr/bin/”, modify the script appropriately). Why do we need this extra step? More on that later…

Now that we have Polipo configured and running, it’s time to attack the second half of the project.

Configuring your cell phone

The first thing you’ll have to figure out is how to access your phone’s hidden programming menu. This varies from phone to phone. If a little googling doesn’t turn up anything, I’d highly recommend that you visit howardforums.com. Your question has most likely already been asked and answered there. For those with a Motorola v325i, the programming menu is accessed by pressing <CLR>073887* in rapid succession, and then entering a security code of 000000.

Once you get to the menu, select “Web Sessions” and then “[New Entry]“. Name the entry whatever you’d like. The fields that we’re concerned with are “Gateway IP 1″, “Port 1″, “User ID”, and “Password”. Polipo’s default port is 8123 and you’ll enter the same username/password you chose in the Polipo config file. Again, if your proxy server is behind NAT, make sure you enter your public, and not the private LAN IP, here.

If you don’t have a public, static IP address, you can also use a free dynamic DNS service, like DynDNS, to assign a hostname to your dynamic IP address. If you’re going this route, then fill in the “Gateway 1″ field with your dyndns hostname and leave “Gateway IP 1″ empty.

You may also wish to change the “Cache Start-Up” option from its default “On” to “Off”. If it’s set to “off”, your phone will retreive a fresh copy of your homepage everytime you start an Internet session. If your homepage is something that changes frequently (like CNN.com), you’ll probably want to make this change. However, since I’m just using Google as my homepage, I’d prefer that the cached homepage load faster.

One thing that you’ll notice that you can’t change is the homepage; it’s hard-wired to “http://homepage”. This is the reason for using the redirection Perl script with the proxy server. Anytime “http://homepage” is requested, Polipo serves up your configued webpage instead. There is a way to change it in the phone, but the method requires transfering a file from the phone to a PC with a data cable, hex editing it, and copying it back. Since mucking around with your phone’s filesystem will almost surely void its warranty (and might brick it), there’s no good reason to do it when an easy workaround is available.

Set this new connection as the default one, and you should be all set! Free Internet access!

Conclusion

This is definately not a 10-minute hack. But assuming you have the Linux skills, spending an hour could save you $60/yr. Or, if your entire office is using Verizon phones, you could save your company hundreds (or thousands) of dollars a year and have the opportunity to prove your salt. But of course, the coolest part about a hack is the flexilbity it affords. Using your own proxy server, you could give your company’s cell phones access to a private Intranet not previously accessable via Verizon. If you’re a parent, you can limit the sites that your children visit on their phones, or restrict the time of day they can browse the web (like during class time?). There are quite a few possibilities.

Since I am limited to only the gear I have, I haven’t attempted to try this config with anything other than a static, public IP address and my Motorola v325i. So if you do this with dynamic DNS, behind NAT, or with a different phone, please post some feedback. Thanks!

Share

Using Apache’s mod_speling to catch unneded 404 erorrs caused by mispelligns

(This article is Part 2 in a series about Apache’s loadable modules. The previous topic was mod_ext_filter).

As former Microsoft-ite who made the leap to OSS, one of the things that took a little getting used to with Apache are that filenames are case-sensitive. While that’s long since been hammered into my soul, a great many other people browsing the web aren’t aware that some webservers are case-senstive, while others are not. Sometimes I’ll see “http://www.example.com/signup” in my access logs when what the person really wanted is “http://www.example.com/Signup”. I could easily catch these simple typos with a mod_rewrite rule, but what if the user entered “http://www.example.com/Sign-up” or “http://www.example.com/Signpu”? Trying to plan for the ways a URL could be miscontrued would be make for a lot of ugly mod_rewrite rules. A better choice is to load Apache’s mod_speling module.

Mod_speling is very easy to implement. And what’s more, unlike mod_ext_filter, it can be used from within an .htaccess file–shared hosting users rejoice! The module is normally loaded by default (though spell-checking is disabled), but you might want to double-check Apache’s http.conf:

LoadModule speling_module modules/mod_speling.so

Enabling mod_speling is as easy as:

<IfModule mod_speling.c>
        CheckSpelling on
</IfModule>

A couple of points to remember:

  • mod_speling will only catch and correct up to one URL misspelling at a time. For true fuzzy matching you’ll have to use a more advanced solution.
  • If Apache finds several likely matching URLs, the user will be presented with a list to pick from.
  • Renaming files and not updating links, relying upon mod_speling to automagically fix them, isn’t a great idea. The module does a directory scan each time it intercepts a 404. If you have a lot misspelled files and a lot of pageviews, you will experience a noticable performance degradation.

For more information, check out the official Apache docs.

Share