I started using tmux today. Its a terminal multiplexer / task switcher for UNIX-likes, very much in the same vein as GNU Screen. However, its a from-scratch implementation, designed to be clean, sane and easy to configure. The more liberal 3-clause BSD license is a plus also, since it means that OpenBSD has been able to integrate it into the source tree, so that its available out of the box.
Comparison with GNU Screen
I’ve been a heavy screen user for many years – almost all my work is done on remote screen sessions. However, screen configuration has always been essentially black magic to me. For this reason, tmux and its nice manual page is a breath of fresh air. `tmux list-commands’ is very straight forwarde and easy to grok. Furthermore, I like that everything in tmux is scriptable from the command line – you can run commands like `tmux resize-pane-up -t comms’ to resize the pane on a session called ‘comms’.
The other thing I really like about tmux is its default status bar. Some people might hate this, but I find it very useful to have a clock and a list of windows along with the process executing in them. This took quite some work to set up to my liking in GNU screen, but the default in tmux is great.
My config
One thing I don’t much like is the default of C-b as the ‘prefix’ command. I suppose this makes some sense, since the author doesn’t want to clobber GNU screen key bindings. Perhaps he will consider changing it to C-a, like in GNU screen, in the future. In any case, this isn’t hard to change. Also, I am constantly using C-a C-a to switch back to the previous window – the default for this action in tmux is C-b l. Much less friendly in my opinion – of course, its also easy to change!
So here are the contents of my $HOME/.tmux.conf:
set-g prefix C-a
bind-key C-a last-window
Getting tmux
I’m sure that packages exist for most operating systems. You can grab the source from http://tmux.sourceforge.net/. On OpenBSD, you can simply run `pkg_add -i tmux’ to get the binary on your system.
Pylons is pretty neat because its really a framework for building a framework. You can pick and choose WSGI middleware and slot it all together with whatever templating engine or database abstraction layer you like. Pylons just gives you most of the glue you’d need – stuff like unit and functional test harnesses, request routing, caching, and various handy decorators. The Pylons approach has some disadvantages however, since its not quite as well integrated as say Django, which has more of a one-size-fits-all, monolithic approach. Pylons also has excellent documentation.
Its interesting that Turbo Gears 2.0 is built on top of Pylons. It seems to aim to provide you with a more consistent out-of-the-box solution – they standardise on one templating language (Genshi), one database abstraction layer (SQLAlchemy), etc.
It will be interesting to see where Turbo Gears goes from here. It could be quite compelling if the community catches on, but it seems to me that its a little late to the game. I’m not certain what jumping to Turbo Gears 2.0 from Pylons would buy me at this point, and I’d imagine many people feel the same way.
I’ve been playing with the recently-released HTTP API for accessing the Best Buy product catalog. While its a little strange to use at first, its actually pretty useful. One of the things I am interested in is online retail, specifically how to make Internet shopping easier. Lets imagine I am looking for information on a particular digital camera – the Nikon Coolpix S210.
A Python skeleton
First, lets get our little Python test harness together. Also, you are going to need your own Best Buy Remix API key. Here is a skeletal Python HTTP client:
importhttplib
API_KEY='<Your API Key Here>'
QUERY="/v1/products(name=Coolpix*&modelNumber=S210)"
OPTS="?sort=name.desc&show=all&format=json"
c = httplib.HTTPConnection('api.remix.bestbuy.com')
c.request('GET', "%s%s&apiKey=%s"%(QUERY, OPTS, API_KEY))
r = c.getresponse()
data = r.read()print data
Save the above to a file like bb.py.
Our first Best Buy query
Now lets try to write a sample query for the Nikon Coolpix S210. Although the Best Buy Remix API docs are a bit sparse, we can guess that items must have an attribute called ‘name’. In fact they do! So lets try searching for the camera by name.
# Same code as above, but we change the value of QUERY:
QUERY="/v1/products(name=Nikon Coolpix S210)"
Looks pretty reasonable. Unfortunately, Best Buy is going to give us back a 400 error:
$ python bb.py
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>400 - Bad Request</title>
</head>
<body>
<h1>400 - Bad Request</h1>
</body>
</html>
Gimme everything!
It turns out that Best Buy don’t name their products in the most intuitive way. Lets try a wildcard on just `Coolpix’ instead:
# Same code as above, but we change the value of QUERY:
QUERY="/v1/products(name=Coolpix*)"
This time, we are going to get tons of data back, in JSON format. Best Buy remix defaults to XML, but I prefer JSON so I added the format=json parameter to the query. Ok, so now we have an overwhelming amount of data on Coolpix cameras – but we really just want information for the S210.
Best Buy’s quirky product schema
Well, there is a solution. Best Buy don’t store the model number in the `name’ attribute – instead they store it in a separate `modelNumber’ attribute. If we query for name=Coolpix* AND modelNumber=S210, we should get the expected result, finally:
# Same code as above, but we change the value of QUERY:
QUERY="/v1/products(name=Coolpix*&modelNumber=S210)"
Et voila! Now Best Buy gives us back all the information it has about the Nikon Coolpix S210. This is pretty detailed stuff, including all those details like compatible memory formats, digital zoom, along with the price and availability. Very cool! Just for kicks, lets show the whole script to send a query to Best Buy, parse the JSON response, and finally print the price:
importhttplibimport json
API_KEY='<Your API Key Here>'
QUERY="/v1/products(name=Coolpix*&modelNumber=S210)"
OPTS="?sort=name.desc&show=all&format=json"
c = httplib.HTTPConnection('api.remix.bestbuy.com')
c.request('GET', "%s%s&apiKey=%s"%(QUERY, OPTS, API_KEY))
r = c.getresponse()
data = r.read()
camera_info = json.loads(data)print"price: %s"%(camera_info['products'][0]['regularPrice'])
It so happens that I end up dealing with the Python ORMSQLObject pretty often. I don’t really like ORMs very much, since in my experience they make those 80% of database things that are already easy to do with plain SQL easier, while making the other 20% of database things which are already hard impossible. They do save some boiler-plate, and let you express your schema and queries in Python (or whatever programming language you are using) instead of SQL – but this tends to break down at a certain level of complexity, and just gets in the way. You end up doing a huge amount of wrangling with the ORM to do something which is very simple in plain old SQL. Fundamentally, SQL was designed as a declarative query language for the relational model and not to represent object hierarchies in the same way programming languages do, hence ORMs are always going to be a nasty hack in my opinion.
SQLObject vs. SQLAlchemy
I actually much prefer SQLAlchemy to SQLObject. SQLAlchemy has a more explicit divide between its various components – you aren’t forced to use the ORM stuff if you don’t want to. It can be used just as a handy database abstraction layer with programmatic SQL and connection pooling and so on if you want. And if you truly want to go the full ORM mapping route, they provide that too. For truly tricky things, SQLAlchemy will be happy to provide you with a DB-API 2.0 cursor so that you can execute whatever custom SQL you wish.
Monolithic vs. modular
This is my main problem with SQLObject – its very difficult to figure out how to get at the underlying database connection. I don’t know how its possible to use the connection pooling and programmatic SQL builder without using the ORM but perhaps it is doable. The documentation for SQLObject is far inferior to the documentation of SQLAlchemy I’m sorry to say. Just try to figure out how to use transactions reliably with SQLObject! Even when I managed to put together some code which according to the documentation should work, SQLObject decided to interleave the actions in separate transactions. With SQLAlchemy I never had this problem.
Getting at the cursor
While seemingly undocumented, it is in fact possible to get the underlying driver’s connection object, and from there grab a DB-API cursor. The pattern is:
# Set up the SQLObject connection as usual
connection = connectionForURI('sqlite:/:memory:')# Grab the database connection
dbobj = connection.getConnection()# Get a cursor from the low-level driver
cursor = dbobj.cursor()# <do stuff with the cursor>
cursor.close()
Lots of people have a small home network. Usually you have a combo box which acts as a router/firewall/file server. Then you have a couple of other machines hooked up, and you share the Internet using NAT. A private DNS server is helpful in this kind of scenario for two reasons:
Recursive resolver cache can speed up common DNS lookups.
Private authoritative resolver lets you easily refer to machine in your home by name, instead of remembering IP addresses.
The DNS Dichotomy
For many years there has been a dichotomy in the DNS server implementation world, pretty much between the ISC’s BIND and just about everything else. The essence of this dichotomy is that BIND integrates both recursive and authoritative resolvers into a single, monolithic daemon process, while practically everything else splits these two functions explicitly. For example, the well-known alternative DNS package djbdns, has one tool – tinydns – for the authoritative portion while another – dnscache – implements the caching recursive resolver functionality.
Convenience costs you security
The monolithic BIND approach has certain limited benefits – mainly that it is convenient to configure and install a private DNS server which acts both as a cache and as an authority for the private domain. Unfortunately, this design has severe implications for the robustness of the software. It serves both to increase complexity within a single process while ignoring the principle of least privilege. Essentially, BIND is a horribly complicated beast, with serious security vulnerabilities being found pretty often – and even the smallest security flaw can result in major problems due to the single process design.
Alternative approaches
Unbound: A modern, secure DNS server
While djbdns might be one of the better-known BIND alternatives, I recently came across Unbound, a BSD licensed recursive resolver. One of the authors of Unbound is also an OpenBSD developer, which inspires confidence in the security of the software.
Unbound also does simple authoritative resolution
One of the nifty features of Unbound is that you can very simply configure it to act as an authority for your private domains. Due to this feature, you can have a single daemon on your home network router acting as both a cache and server for your local domain. This is very nice. In fact, I have found the Unbound configuration format to be considerably nicer to deal with than that of BIND.
Setup under OpenBSD
This describes how I set up Unbound on my OpenBSD machine – it should be a pretty similar procedure on most other operating systems.
# install the package
$ sudo pkg_add -i unbound
Now you have the binaries on disk, you can edit the configuration to set up your private domain. Unbound runs as a recursive resolver out of the box, so this is just about all the configuration you’ll need to do.
# edit the config
$ sudovi/var/unbound/etc/unbound.conf
For a single machine, add the following under ’server’, replacing ‘inet’ with the desired name of your local domain, and ‘joust’ with the name of your machine:
local-zone: "inet." static
local-data: "joust.inet. IN A 192.168.1.1"
Since you want the DNS server to be accessible from other machines, you probably want it to listen on 0.0.0.0 (all available interfaces). Make sure you have some kind of firewall in place before you do this, though – you don’t want to let random Internet hosts query your DNS server:
interface: 0.0.0.0
# Make sure you have a packet filter to block queries from the Internet.
# Alternatively, set this only for your local network.
access-control: 0.0.0.0/0 allow
Now you can start up Unbound:
$ sudo/usr/local/sbin/unbound
And of course you probably want it to come up on boot, so follow these instructions:
$ pkg_info -D unbound
Information for inst:unbound-1.2.1p0
Install notice:
You should add:
syslogd_flags="${syslogd_flags} -a /var/unbound/dev/log"
to /etc/rc.conf.local to create a syslog socket in the unbound chroot.
You may also want to add the following to /etc/rc.local to start unbound
at boot:
if[-x/usr/local/sbin/unbound ]; thenecho-n' unbound'; /usr/local/sbin/unbound
fi
Have you ever tried to run a very network-intensive P2P application on Windows XP SP2 or higher? If so, you may have encountered very strange behaviour with the Windows TCP/IP network stack. Specifically, you won’t be able to open any new TCP/IP connections, so web-browsing, email checking, SSH, etc will all be basically unusable. At the same time, P2P transfers might be running fine for all you can tell.
I spent some time trouble-shooting all aspects of my set up including testing the router, my Internet connection, and even re-installing Windows a couple of times – all to no avail. Finally I figured out the source of the problem, and how to fix it!
It turns out that this behaviour is due to a feature introduced in Windows XP Service Pack 2 which limits the number of TCP/IP sockets you can have in the ‘half-open’ state to 10. A connection in the TCP half-open state means that one side of the socket has disappeared and stopped responding, without properly notifying the other side. After 10 such connections, the Windows kernel queues up all new socket connection attempts and you will experience this network “freeze” behaviour. Apparently the reason for Microsoft having this low default is to slow the spread of virii and limit infected hosts from participating in DoS attacks etc.
Unfortunately, P2P applications in particular are very negatively affected by this low default, because they are constantly opening new connections to peers, they very quickly fill up the default limit of 10 half-open connections.
The good news is that once you realise this is the problem you are encountering, its trivial to fix. There is a very simple patch which raises this default to any value, with a recommended setting of 100. I have used this patch a couple of times, setting the default to 100, and it has completely solved the issue for me. Yet another annoying, niggling thing to install on a fresh Windows system! Oh well.
I’ve always been under the impression that there was an important distinction between the “natural flavours” and “artificial flavours” listed on food ingredients. After having read this food science article (adapted from Fast Food Nation)Why McDonald’s Fries Taste So Good, I realise the difference is essentially meaningless:
Natural flavors and artificial flavors sometimes contain exactly the same chemicals, produced through different methods. Amyl acetate, for example, provides the dominant note of banana flavor. When it is distilled from bananas with a solvent, amyl acetate is a natural flavor. When it is produced by mixing vinegar with amyl alcohol and adding sulfuric acid as a catalyst, amyl acetate is an artificial flavor. Either way it smells and tastes the same.
Natural flavours no healthier than artificial
I had for some reason assumed that natural flavours were probably not quite as bad for you as artificial flavours, however this is not necessarily the case:
A natural flavor is not necessarily more healthful or purer than an artificial one. When almond flavor — benzaldehyde — is derived from natural sources, such as peach and apricot pits, it contains traces of hydrogen cyanide, a deadly poison. Benzaldehyde derived by mixing oil of clove and amyl acetate does not contain any cyanide. Nevertheless, it is legally considered an artificial flavor and sells at a much lower price. Natural and artificial flavors are now manufactured at the same chemical plants, places that few people would associate with Mother Nature.
So, basically “natural flavours” and “artificial flavours” are the same thing, and are best avoided if possible.
I often use Windows as a terminal to my various UNIX systems. Sometimes its helpful to run proprietary software – and I don’t have time/inclination to mess around with half-baked emulators/ports/binary blobs/whatevers under Linux. I either run a completely open system like OpenBSD or I run Windows.
Anyway, I never use Windows to do any real work. I always shell into a remote system to actually get things done. Either PuTTY or – if you prefer real OpenSSH like me – OpenSSH via Cygwin/X work fine for getting a terminal. WinSCP or Cygwin’s OpenSSH for scp(1) are good for file-transfer under most circumstances. However, one of the nice things about Windows is the Explorer shell. It – and its KDE knock-off – are useful for certain file management operations. Why not leverage it?
So I started looking for a way to mount remote filesystems via SSH, so that they appear as native Windows volumes. And I found a way to do it, for free.
I figured this thing was surely going to be a PITA and probably not work to boot. In fact, you just install three things – some Microsoft runtime library, the main Dokan library, and Dokan SSHFS – and there you go. There is simple GUI app to set up remote mounts that supports all the things you’d expect, saving sessions, both password and public key authentication.
Does it actually work
Yes, although it doesn’t seem to support symlinks. A symlink to a directory on a remote system appears as a file under Dokan. So no $HOME/public_html for you – oh well.
Final thoughts
Its fun to look at your horribly un-organised UNIX home directory in Windows, and see just how messy it is. Almost makes me want to start cleaning things up. But then I remember I know how to use locate(1) and find(1).
A few weeks ago I needed a designer to produce some HTML/CSS and Photoshop templates for a web project I’m working on. While I have good working knowledge of HTML and CSS, I am not very interested nor efficient at working with it. And even worse, I’m quite poor at coming up with graphic designs. This is a project I’m willing to spend some money on, so I figured I should hire a designer to come up with something slick and good looking. I’m very aware of how important an attractive looking design is for a website!
What I needed
I had a pretty concise idea of what I was looking for – the theme, and so on. I also was pretty sure that I only needed one or two individual page templates and that I could fill in the rest myself once the overall design was fleshed out. I originally hoped I could work with one of my friends who is an excellent designer – I would happily pay a friend a decent amount of money for good results. Unfortunately it turned out that my friend was too busy to take on my project.
So I decided instead to try outsourcing the project. There are plenty of outsourcing websites out there, I happened to have heard good things about Elance.com, so I went with that one.
Doubts about outsourcing
I certainly had some doubts about outsourcing this project. Would I be able to specify it clearly enough, and would the provider give me decent results in a timely manner for a reasonable price? I imagined that communication could potentially be very difficult, given language barriers and text-only communication. However, I figured it was definitely worth trying. Even if it was a horrible failure, I would at least have learned something about the realities of outsourcing.
Specifying the project
Like I said, the scope of my project was quite narrow. I just needed a design template in PhotoShop (so I could change the graphics myself, or give it to another designer) and converted to HTML/CSS, so I could start making nice looking Pylons templates for my web project. In my specification, I was careful to repeat myself in different ways to ensure clear communication. I made sure to emphasize the fact that this was a single template consisting of a single page. Repetition is key when dealing with outsource providers, indeed this is true when dealing with just about anyone. I used to be a great fan of writing extremely concise emails – but have learned from experience that adding a few more layers of redundancy in the form of repetition – greatly aids communication. I learned the skill of close reading of text during my French degree – the simple fact is that most people do not have this kind of training. It never hurts to rephrase what you are saying a couple of times, with plenty of examples.
Choosing a provider
Within 24 hours of posting the job specification on Elance, I had ten different providers bidding on my job! I was quite surprised by this response, and even overwhelmed. How are you to choose between ten different providers? Of course, Elance helps you somewhere here – providers have ratings, and previous clients submit reviews. However, they all looked pretty much the same in terms of ratings. The main place they differed was in price – offers for my job ranged from $100 to $500. I took a quick look at their portfolios. The work was slick and good-looking across all the providers.
Test responsiveness
I decided to test communication. I slightly reworded the specification and sent it to each of the providers to ensure they understood what was involved, and also to measure responsiveness. Most of the providers had clearly “got it” the first time around, and they all replied quite promptly (< 24 hours). In one case, the provider actually lowered their price by $100 after I sent the re-worded spec.
The bottom line
Ultimately, I decided to just go with the cheapest provider. I was aware that I might not end up with optimal quality, but I was treating this as an experiment. Worst case scenario was that I would end up with sub-par work and slightly out of pocket. Best case scenario, I get great results for an exceptionally good price.
Pushing work forward
So I give the job to my chosen provider, and then what? I decided to give him a couple of days to chew on it. He had said he would have it done in a week.
Day four
Three days later, he has a mockup design which I like the look of. I tell him I’m impressed and to go ahead and chop that up into XHTML and CSS for me, using the perfect 3-column layout with no CSS hacks.
Day five
Nothing from provider. I decide to send a status request message. Asking for updates often is a must. I imagine that these providers have several jobs underway at the same time, and whoever prods them the most gets their attention. Squeaky wheel gets the grease and all that. Be a little pushy.
Day six
Nothing. I send another status request.
Day seven
Get an apologetic reply from provider saying he’s going on vacation for three days and will have it when he gets back. I’m not in a huge hurry so I don’t really mind – I’m only paying $100 afterall.
Day twelve
I send another status request.
Day fourteen
Nothing from provider. At this point I’m getting a bit annoyed about the unresponsiveness. I consider my options, and decide to reduce the amount for the job to $0. Elance allows you to do this – its essentially a way of canceling the contract. The provider has to agree to it however, if you are stuck with an utterly unresponsive provider, you can ask Elance to step in to resolve to conflict.
Day fifteen
It appears that my attempt at cancelling the job has succeeded in escalating priority of my job. Provider sends me a very apologetic and message including tear jerker story about being in the hospital with injury etc. Impossible for me to verify any of this of course, but I just want the work done. I decide to give the guy another chance to finish the job. I’m not in a huge hurry, but at the same time I’m itching to get the nice template into my project.
Day sixteen
Provider finally sends me full XHTML/CSS source for the template along with the PhotoShop image file I requested. I’m happy with the work he’s done, and release the funding from escrow. Job complete!
Conclusion
I think I got the website template done for around 20% the cost of hiring a local designer. I was ultimately very happy with the final product. However, there were periods of uncertainty and a quantity of headache involved in getting the work completed. I would definitely use Elance again, and recommend it for this type of work. However, its important to have a very clear idea of just what you want, and also you should be prepared to have to prod the provider regularly to get the work done.
It’s up to you to decide on the technique to employ for the shoulder-to-overhead movement (shoulder press, push press, push jerk, etc.). Efficient technique will be rewarded.
Of course, the push jerk is one of the most efficient ways to get a weight overhead – because you use the hips and legs to drive the weight up, instead of just the shoulders. According to the CrossFit Journal, an athlete skilled in the push jerk can lift 70% more than they can press. That is a dramatic increase.
While I’ve worked the press a bit, especially with kettle bells, I have minimal experience with the push jerk. Yesterday’s push jerk WOD, then, was welcome practice! One of the sticking points for me is shoulder flexibility – I find it very hard to keep my arms locked out overhead with the weight centered over my spine. I need to work on both the flexibility – with more shoulder dislocates – and on the movement itself. Since I don’t have a rack, I of course need to clean the weight up off the floor before I can jerk it. I found this excellent video on both power clean and then push jerk technique:
Recent Comments