Fusion tables are intriguing because they offer spatial query processing and very thorough Google Maps integration. What this means is that you can completely avoid setting up any database infrastructure such as PostGIS and any tile generation services like Tilestache.

I employed these two services to come up with a simple app called Gyms Near Me which does what it says on the tin! It uses Google Maps with a Fusion Tables Layer to show colour-coded pins for all the gyms in your vicinity. When you select a gym, a JSONP call finds the five nearest gyms by ordering by ST_DISTANCE from the selected latitude/longitude with a LIMIT of 5. The really cool thing about Fusion Tables Layers is that they are delivered as PNG tiles which gives the potential to a have huge number of points on a map, avoiding the issues you might encounter if you were creating individual markers via javascript.

The app is now live, so I’ll soon see if it gets picked up for any Google search keywords. Looking at autocomplete it seems like Gyms Near Me is the top term so maybe it will get picked up! In the future I might also look at feeding in gyms for global locations and saving map positions like Gyms in London.

The trick is to define both a hash and an equality function for the custom data structure. The prime numbers in the 3D vector hash function come from this paper. The “correct” hash function depends a lot on the distribution of the keys and only real world benchmarking will give a satisfactory answer as to what works best for your data.

The output of the above code would be something like:

Initially, we looked at existing search engines like Xapian and Lucene and although being very effective at finding articles containing the same words, they weren’t particulary good at finding the articles with the longest common strings. More info on this can be found here.

Besides spotting churnalism, the software seemed to have other potential uses and the Sunlight Foundation believes it could be applied to spotting duplicated federal law. To facilitate this application they have generously offered to fund the open sourcing of this software, and the results should hopefully be soon tested against the OpenStates project. I’d like to thank Martin Moore, Ben Campbell, Tom Lee and everyone else at Sunlight for the opportunity to work on this great project!

Week 1

This is my first attempt at a truly open-source project so I’ve had a lot to learn about making software accessible by more than just the development team I am working with. SuperFastMatch originally started as the algorithm, developed over many sleepless nights, for comparing press releases with newspaper articles used by Churnalism.com. This was a single use-case, but as the site became known to the public, other use-cases became apparent. One is to track legislation for lobbyist influence. Others might include tracking plagiarism in fiction or making sure academic texts have correct citations.

The point is that when more than one use-case comes up, a piece of code needs to develop into a framework which is easy to understand, is well documented and is easily integrated and installable in concert with other software. Seeing as the majority of the existing code is written in Python it made sense to look at how some other Python projects made use of existing infrastructure to meet these ends. I examined Celery, a distributed job engine and tracker, as I was familiar with it from previous work, and saw how it made use of tools to assist its open source development. Below are some ideas and tools that I borrowed from it:

• Github has become the de facto platform for sharing and distributing code. It’s free for open source projects and is built on top of git, which has many advantages over previous generations of source code control software. It also offers static file hosting, from where you are probably reading this documentation!

• Pypi is a centralised index of Python packages that allows for the versioning and sharing of Python software without the need to remember lots of urls and is reliable enough to be referred to in bootstrap scripts.

• Sphinx is a documentation generator that works well with Python packages as well as C++ (and hopefully Lua!). This page and the rest of the docs are written in reStructured Text and then passed throught a batch process which converts them into HTML, after which they are uploaded to the static Github hosting.

• Paver is a combination of tools which allows for the scripting in Python of many development workflows such as bootstrapping, documentation generation, packaging and test execution. I’m used to using Fabric for deployment, and Paver seems to offer a similar experience for development. It will be interesting to see if I integrate some of the C++ build process into the pavement.py file.

• VirtualEnv is an essential tool for isolating Python packages into an application-specific environment. This ensures you always know which versions of software are present and can accurately recreate this when you deploy to either a production or staging server. It comes with Pip installed in the created environment for easy integration with Pypi.

• Django is the platform on which Churnalism.com is built and is widely used so it seemed like the natural choice for building an example usage of SuperFastMatch.

Week 2

Now that all the open source infrastructure is in place it’s time to start learning a bit more about the use-case in hand, US Congress and state legislation.

Legislature

The US Congress has two chambers, both elected, the House of Representatives and the Senate. Both chambers can introduce bills and the bill can take one of many paths through both chambers. The passage of a bill through Congress appears to be quite undefined with a whole series of possible routes and referrals being possible before an Act is signed by the President.

The end result of acts of Congress is the US Code which is an up to date record of all enrolled bills. States in the US are also capable of passing their own local laws.

Data Access

The US has a well maintained digital archive of all House of Representative Bills from 2004, and a more recent archive of Senate Bills. The data is accessible in XML form which allows for metadata extraction of Bill information and easy manipulation of the Bill text itself, ideal for the purposes of this project. Text-only bills are available from 1993 onwards.

Week 3

Framework design

The aim of the project is to design a reusable tool for bulk text comparison and analysis. To succeed in becoming reusable, at least two use-cases have to be considered and a clean, easy to understand interface for the user has to be designed. Django offers many ways of designing models for data storage, and this flexibility is useful - but there can be a number of caveats that can obstruct progress further down the line.

Model Inheritance allows for the subclassing of a Model which itsef permits further extension. This is an ideal pattern for different types of Document. Say that you have Press Releases and News Articles, or State Law and Congress Law, all with potentially similar content. When you search, you might not know what to expect as results and would like to search all Document types simultaneously. By defining a Document base model, this is possible because it can be assumed that the necessary content in it’s indexable form is present.

With Django, there is a choice of either abstract or multi-table inheritance (and also proxy…). Multi-table inheritance results in an extra table, while abstract inheritance just adds fields to the subclasses’ tables. The simplest choice is abstract inheritance and I went for this. However, the disadvantage is that you cannot define a ForeignKey to the abstract base class itself but the related clean indexed content needs to be stored somewhere as a cache so that updates can be detected. To solve this issue I turned to content types, which makes it possible to relate any model instance to any other model instance, regardless of type. This means that the cleaned Content can be related to any of Document’s subclasses. At this point it’s probably worth looking at the model definition.

Faceting

One of the limitations of the original Churnalism code is the inability to search for news articles, only press releases. However, both document types have very similar features, such as date published, an author, a publisher and of course the content itself. Therefore it makes sense to also be able to search for news articles with either a news article, press release or other text as the input text. This would be useful for journalists to check for plagiarism and is useful in lots of other contexts.

The challenge is that the script that builds the search results needs a limit on the number of results to return to make the data and server load manageable. However, imagine searching for both news articles and press releases with a limit of 20 results. It might be the case that 20 news articles have higher ranking than matching press releases. The press releases might be a more interesting result though, so excluding them could omit valuable data. The solution to this is faceted search where extra data, in this case the document type is stored along with the document id in the index. The number of results returned is per document type, and therefore useful data should not be missed. This has implications for total index disk space usage, but is a truly useful addition to the capabilities of SuperFastMatch.

Other fields, such as published date, could be used as filter, but then we are entering the territory of advanced search engines like Xapian and Solr which are already good at that type of thing!

Week 4

Kyoto Cabinet

Storing and processing data is a challenging task with many, many options! Often the performance characteristics and ceilings of a platform only become apparent after a large amount of implementation for that platform has occurred. These evaluations can take a lot of time and definitely help form an opinion on which platform is good at a particular task. SuperFastMatch works on the simple concept of storing every hash of every window of text in a document with the associated document id (and now document type). This is done for every document and yields an inverted index of hashes to documents. Thus, for a search document or text, hashing its windows allows for a fast search of matching documents.

The storage requirement for this is very high - typically 5x the original document size. Access speed is very important, given that a search of a document n characters long requires (n-window_size) lookups. Also index creation speed is vital. If it takes 2 days to index 1 days worth of news, Churnalism.com would be behind the current affairs very quickly!

The obvious starting candidate for a platform is SQL, and I experimented with Postgres, testing out such features as the intarray data type for storing document ids and partitioned tables as a means for bulk loading daily data whilst maintaining insert speed. However, even with 24-bit hashing (ie. 16,777,216 possible hashes), lookup and insert speeds proved to be poor.

The next candidate was the sexy new key value store on the block! Redis has lots of nice features ideal for storing the index. High speed inserts and the sets command set, great for ensuring no duplicate id is saved for a hash. However, again a performance ceiling reared its skull-damaging surface! Redis operates great when the data set fits totally in memory, but as soon as that limit is surpassed, the paging virtual memory kicks in - but very slowly. The release notes and commit history showed that this was a known issue, but it made Redis impractical for use.

A lateral solution was to consider using Hadoop in the form of Elastic Map Reduce as a way of solving the daily index creation problem. It was interesting to stray into the realms of Java land (with its verbosity quite overwhelming at times!). Issues found included the need to code a simple CSVReader - not included in the vanilla distribution. The difficulty of getting something out of Hadoop in binary rather than text file form. The misleading and, in the end, very expensive pricing of both the EC2 service and the S3 costs for downloading the processed data to an external server. I can see that if you are Yahoo and have 1000’s of servers, Hadoop is a great way to distribute jobs across many machines. However, if you have a continuous data processing task that could be run on one very powerful machine the economics of it don’t quite make sense.

At this point, despair was near! A final gambit seemed to be the service offered by the search giant themselves. App Engine had a lot of ticks in it’s favour. The datastore is capable of massive scaling with no configuration required. API calls to the datastore are very performant (and now can run in parallel). The task queue is fantastic at allowing extra instances to be pulled up as and when needed for processing the index. All in all, Appengine was the preferred choice and I had a great working prototype. The one major headache was the billing cost for Datastore API writes. Because the data was incoming in unsorted form, each hash for each document required an API write, which added up to about £6,000 just to index 3.5 million news articles! This meant any mistakes in indexing would surely kill the project’s budget. A postscript to this might be that the recent addition of the shuffle and reduce phases of the map-reduce project might make the insertion costs considerably less. Also the recently announced full-text search API has a numeric data type that could be misused to simulate the inverted index.

So after a very large amount of time spent evaluating performance, cost and practical implementation it became clear that was a definite advantage to investing in some serious hardware to negate the higher than expected cost of high performance cloud infrastructure. A server configured by Pete at Mythic Beasts with 64GB of RAM and an Intel X25-M faciliated the speedy operation of the final solution that we decided upon.

And that final solution was to use a combination of Kyoto Cabinet and Kyoto Tycoon written by the talented Mikio Hirabayashi (now gainfully employed by Google) who kindly incorporated some feature requests to do with the bit length of Murmur hashing exposed to Lua and gave very useful implementation advice. The pros of Kyoto Cabinet are numerous and include:

• Very fast insertion and lookup speed, whether accessing from disk or memory.
• High tunability of indexes in terms of memory usage and algorithms employed in sorting of keys.
• Embedded Lua scripts can be run in multithreaded HTTP aware server.
• Designed to be used both as a toolkit and a framework for development allowing tighter integration as bottlenecks are encountered, ie.e the library and header files allow for everything to be used in a custom C++ project if desired.
• Great documentation.

Filtering Junk

After examining the data for US Congress bills it has become clear that this a far less heterogenous data set than that found with Churnalism.com. For instance phrases like “is amended by adding at the end the following” and “Notwithstanding any other provision of” appear in nearly every bill. These phrases in themselves do not indicate similarity with another bill, but they might if they are part of a longer chunk. So how to ignore the stock phrases when they they are just stock phrases, but include them when they are part of something more unique?

A natural by-product of the index creation, especially as the corpora becomes larger, is that for every window, or in fact hash, there is a sequence of document ids and document types, which are themselves vital for search, but when accumulated together the count per hash gives an indication of the originality or cliché of that window. Say that this sentence is present in both search text and a match candidate:

“The Tobacco Smoker’s compensation bill is amended by adding at the end the following”

The last half of the sentence consists of very frequently occurring windows, whilst the start is very likely less common. We want the whole sentence to be given prominence in the fragment results. We can do this by averaging the counts of each hash, perhaps with a high cut-off, say 100, for the high counts. This would allow us to filter out the very common phrases, which would tend to score around 100. This is yet to be implemented, but should be easy to achieve and is scheduled for version 0.4.

Another possible use of this sort of “heatmap” might be to visualise the originality of texts, with common phrases coloured red perhaps and more original phrases in a cooler tone!

Well, not for the first time, there has been somewhat of a gap in the continuity of posts on this blog. I could say sorry, but then I’d risk ending up here as Jack has noted.

There is a rites of passage in Web Programming, whereby you write your own blog software in whatever framework you are learning; you write a few posts and then stop because each new post has some CSS requirement or weird formatting that makes you give up. I did that.

I write code. I often read other people’s code. I’d like to share my code. Stack Overflow is good for this if you have a question, or know the solution to another person’s problem. But what if you have found a nice solution to a problem in your own code. Sharing code on the previous two incarnations of this blog was a pain. Moving posts between different blog platforms is a pain. Logging in to whatever website to make a post is a pain. Lots of pain!!!

Content is generally written in Markdown at a point in time on a subject. Jekyll aims to keep that simplicity and Octopress builds on that to make sharing code easy. So here goes with this (yet another) new framework. Hopefully the big advantage will be that all the content is now much easier to play with using text editors and grep!

First, a chronology (personal details included for clarity!):

• I live with my New Zealand girlfriend in a flat in Liverpool City Centre.
• We split up in August 2009 and she moves out from the flat by the end of that month.
• In September 2009 I post this form to the Revenue Services applying for the Single Occupancy Discount of 25%. I do not know the forwarding address of my girlfriend at this point - such is the nature of ended relationships!
• In October 2009 I get the same form resent to me asking for the forwarding address. My ex-girlfriend now gives me this information. I post the form again.
• I hear nothing for the next two months, so ring up Liverpool Direct in January 2010. They inform me that they sent a letter refusing the discount on the grounds that they cannot confirm my ex-girlfriends new address and fiscal records indicate that she still lives at this address. This letter, dated the 24th November 2009, never reached me.
• I request that Experian explain how their data indicates that my ex-girlfriend still lives at my flat. They explain the concept of a financial association. On the 5th January 2009 I request that this information is removed as it is incorrect. On the 7th January 2009 I receive acknowledgment that this has taken place.
• I appeal the decision in person at Dale Street’s One Stop Shop on the 12th January 2010 and inquire as to the nature of this fiscal information.
• No reply. I phone Liverpool Direct and they inform me of another unreceived letter, dated 20th January 2010. Again, another refusal, saying that my ex is still in occupation, according to an unnamed “Credit Reference Agency”. I request copies of these letters. Mysteriously, they all turn up at the start of Februray 2010.
• On the 4th February 2010, I phone Experian again and discover the information has not been removed, as they had previously said had been the case. I request that the information is removed again. It turns out the specific information relates to a joint personal loan application in April 2009 (that was turned down) to help my ex-girlfriend consolidate her considerable debt. Interestingly, the association is based around the fact that we lived together in London before we lived together in Liverpool. Experian removed the Liverpool association, but not the London association. A key fact in this horribly bungled and failed process.

And so we go on ….

Kafka would have been proud of the communication cycle that occurs with Liverpool Direct’s council tax department. You phone the call centre and supply your reference number. You give the operator information about your case. They make a “service request” that goes to the revenues service in a separate building. A randomly-selected revenues officer writes a vague reply without details of which Credit Reference Agency is used and what alternative evidence can be accepted. This reply is sent in the post. The Royal Mail have a deal with someone, whereby they don’t deliver the letter. You phone Liverpool Direct, they make a “service request” for letters to be resent. The letter does not help explain what next steps need to be taken.

Revenues Officers refuse to take phone calls explaining and justifying their decisions.

The fact that needs to be up for debate is why the council can use a failed personal loan application in April 2009 as proof that two people share a flat between September 2009 and February 2010. Also, why do the council even have access to this information? The answer to that question seems to be a market opportunity spotted by Experian to reuse credit data as something that can be assumed to be the truth of where people reside at certain points in time. My case study is an example of where Experian’s data can be proved to be incorrect. There seems to be no indication on the Liverpool City Council website or in any of their communications on how to correct the wrong data. For the record, the request form is here.

Another fact up for debate is why council officials can have a one-way dialogue with borough residents. The answer to this is the help desk outsourcing of Liverpool Direct. In exceptional circumstances, process must be overridden and direct conversation permitted.

How could this whole problem have been avoided? Very simply and cheaply. A letter could have been sent to the forwarding address I provided, asking for confirmation of residence. This was not done. Shocking.

Two questions for MP’s and councillors. How much is being spent on the Residency Checker licensing? How many single occupancy discount requests have been wrongly declined against based on bad data from Experian?

Brings back memories of trying to do animation in Render Bender and writing parallax star fields in Assembler!

Linq allows queries to be composed at runtime and built to arbitrary levels of complexity, as needed by the consumer of the code. This is a win situation compared to versioning strings of SQL that are not type-checked and prone to error. However, if we go down the Linq route we also need an ORM solution to go with it. Currently there are four choices:

• Subsonic
• Linq to SQL
• The Entity Framework V1
• Nhibernate with Linq to Nhibernate

I like Linq to SQL because it is lightweight, the documentation is fairly complete and I’ve invested time learning it. It has some missing features, though, and now Microsoft are deprecating it in favour of the Entity Framework V2.

Please don’t do this! Improve what is already a good product with some simple additions:

• Add a many-to-many association
• Add the two other patterns of inheritance: Class Table Inheritance and Concrete Class Table Inheritance
• Improve the CreateDatabase() method and extend the MetaModel to include database default values (newid() for example) and a PostCreateDatabase hook for inserting any custom SQL
• Add a versioning feature to the mapping source which can deal with migrations from one version to another. Currently Linq to Sql works well for DB-first design, as long as you automate the process as follows:
• Introspect the db with sqlmetal to create a .dbml file
• Patch the dbml file with changes using MSBuild Community Tasks for instance
• Run the second half of the sqlmetal process to generate the code

This means that you don’t end up with nice clean POCO code, however. I would prefer, in a DDD way, to start with some POCO’s and then create mappings using a fluent interface, and then have some SQL DDL’s created. Maybe this is possible now. I just don’t want to plough through the horrid NHibernate documentation and the EF looks like a mess that won’t be sorted out until VS 2010 hits the streets. Subsonic is an option but Rob Conery seems awfully busy at the moment!

It was quite a lot of fun searching all the short words ending in ‘ly’, and believe me, there are quite a few left! I ended up going for:

• rap.id.ly
• luc.id.ly
• viv.id.ly

and

• app.ly

If you are interested in lightening me of one of these domains, then feel free to comment!

So, along comes an interesting company by the name of Syabas with a product that will potentially change the way you enjoy your living room. The PopcornHour is a small box, maybe twice the size of an external hard drive. It gets delivered from China for about £100, but comes without an internal hard drive. The model I received at the end of last year accepts an old-fashioned PATA hard drive, whilst the newer A110 model takes the better-performing SATA types. You don’t need to get a hard drive if you have some fancy Network Attached Storage on your home network, but I don’t think you get all the extra applications without the internal drive.

I went for a 500GB Western Digital, which is excellent in terms of quantity of media that it can store. Unfortunately a bug exists in combination with that drive, so don’t buy it! Details here. A better buy might be one of these. Don’t worry if you do pair the errant drive with the PopcornHour, there is a preference to stop the remote from repeating itself, which is annoying, but does solve the problem.

So, what does it do? Everything!!! You can watch virtually any format of media on it, from .mkv to quicktimes, from DIVX avi’s to DVD backups. You name it, it seems to play it. It even has an interface to watch YouTube and other online video sources. It plugs into any television via SCART or HDMI, so it works nicely with the new LCD high definition screens starting to creep into our lives. Perhaps the biggest boon is that it has a scheduled BitTorrent daemon running, so you can queue up your .torrent files and let it happily chug through them overnight, without having to leave your laptop turned on. What would be nice is an unrar utility built in, to unpack the compressed downloads that are so prevalent.

The really cool bit is that Syabas keep adding features. Every month or two a new firmware update is released, which packs even more utility and bugfixes into the small box beneath your telly! The latest update includes the Transmission web client for visual pleasure. However, it does seem to be a bit slow at the moment, compared to the previous, but still present, BTPD. I’m sure this will be fixed in the next update though!

So here are some pros and cons:

PROS

• Plays everything you can throw at it
• Relatively cheap
• Commitment to future updates from Syabas
• Makes BitTorrent a pleasure rather than a file download nightmare
• You’ve got one before any of your friends!

CONS

• No wireless networking built in
• Interface is a bit cludgy
• Requires a bit of technical know-how to get going
• Remote is painful with my chosen hard drive
• USB ports at the front of the box

In conclusion, get one!

Full credit must go to Jonas for releasing his WordPress theme to the world under a GPL license. I was using DasBlog which had more than a whiff of ASP.NET to it, but served a purpose when I had a DiscountASP.NET server account without any cheap SQL access. Now, with my pleasant 1 & 1 hosted server I’ve looked towards Django to provide coding pleasure and here is the result! The code is inspired by, but not lifted from Django Basic Apps and the templates and styling are heavily influenced by Jonas’ html and css.

I realise that both those bits of work are released under different licenses which means I should distribute the derived code freely. If anyone is interested, then I would be happy to put together a Google Code project to share it, although I cannot vouch for the quality of the code!!

Anyway, I’m back!

P.S. As soon as Sophie gets back from holiday I should have a nice brand spanking new logo!

Full film here.

Bottom line: it used to be possible to buy a saver return on the website which meant as long as you picked the outward day of travel, you could return on any subsequent day within a month of the outward journey date. Things have changed …

A Half Saver Return is now offered, which basically means that you are pinned down to a return date too. The added benefit of a £5 penalty to change this date comes for free. If anyone from Virgin ever reads this, then let me tell you: YOU ARE #£*&!

I can understand fare increases, over-inflated as they are, but the constant reduction in ease of travel is a disgrace. The train system of this country is not RyanAir; it is a public service for the population to travel around the country for business, pleasure and family reunions. It should be easy to use, flexible and affordable. Tying people down to deciding on times and days of travels weeks ahead of time is not on. Things happen: the Underground fails to get people to mainline train stations on a daily basis, MerseyTravel staff go on strike, the Liverpool tunnels close unexpectedly for maintenance. It is virtually impossible to ensure you can get to Euston for any set time without either hiring a taxi or getting there hours in advance. I suppose this is an argument against the penalty for missing an Advance ticket booking, but the decrement of the availability to buy full saver returns on the website hints that eventually all tickets will be in the discount airline style. That is, unless you can afford the high-priced Open tickets.

I have previously written to two Department of Transport secretaries on ticketing issues and they have both said that only the saver and open return tickets are regulated. Please regulate that they are available on the website to purchase, or please get rid of Virgin.

Rant over.

The slot was scheduled on a Sunday evening and sometimes the films would run past midnight, but my mother, another film buff, was more than happy to let me stay up and watch the films with her, despite an imminent school day. A quick search yields some of these intros and I definitely intend to watch them before I watch any of the movies again. Bring back Alex Cox!

Travelling in Vietnam by myself has so far been a varied and interesting experience mixed with lows and extreme highs. To spoil the ending, the best part of it by far has been interacting (ie. talking, smoking, drinking, bartering, clinging on to while riding scooters, joking) with the Vietnamese people (they (I hate using that word) call themselves Vietnami). The warmth, friendliness and genuine smiles just cannot be matched by any other culture that I have encountered (not that many in reality …). The smile itself is an act of greeting and once that has been established the usual starter questions are ‘Where you come from?’, ‘How long you stay in Hue/Hoi An/Doc Let/Sai Gon?’, ‘You travel alone?’, ‘Michael Owen?’.

The Vietnami have good English vocabulary but some of the pronunciations can be hilarious - I met a ceevill(sic) engineer who was travelling back from Nha Trang (said Na Tran) after having been there for the Miss Earth beauty contest. I’m not sure if he was the official photographer or had just taken lots of photos, but Miss Sweden was his definite favourite! He had a wife and child in Hoi-An and the whole trip seemed innocent enough. A sudden connection was made in my head though. Two Russian Lesbians from Vladivostok who were staying in my previous residence (Ki-Em - very good for honeymoons) had mentioned that they were keen to visit Nha Trang that day and I suddenly realised why!

Another hilarious conversation arose in a Sai Gon park one boiling hot afternoon. I was speaking to Tuan (a he) and Tham (a she) and discussing life in Sai Gon/Ho Chi Min City and Tuan mentioned that, at university, he had entered and won a beauty contest. I said he was a very handsome man (in truth he looked like he had seen better days) and he returned the compliment and proceeded to give me his mobile number in a completely heterosexual way! I got Tham’s number too! The pattern seems to be start talking , and people literally queue up to have a chat - wanting to practice their English. I spoke to an old lady who had perfect English and French - it seems that the older people possess good French as a vestige from the colonial days. Indeed a lot of French tourists are here and in some restaurants the lingua franca is French.

The downside of the interaction can be the hawking of goods and services in tourist trap areas. On every corner is a cyclo-rider intent on giving you an hour’s tour on his vehicle or, if you sit down for a rest, a selection of postage stamps collections and old coins in ring binders will be offered for your perusal. I soon got the hang of fending off this menace though. A friendly smile, a shake of the head, and a raising of the palm seems to elicit an equally friendly relief of the sales pitch. Another technique that works well particularly well at train stations is just offer the chap who won’t leave you alone a cigarette and have one with him. The conversation will soon drift to football! A funny faux-pas to make is to offer a cigarette to a lady. They almost crease themselves laughing at the impossibility of accepting one! Vietnami men do not like their women to smoke, apparently.

Everything is cheap here, very cheap. The difficulty is knowing the price of things, which only comes from being ripped off the first time. A packet of Vietnami-brand cigarettes (White Horse, Hero, Craven-A (as smoked in Liverpool)) is 10,000 Dong (about 30p) while the American Brands are 17,000 Dong (just over 50p). Beer is about 10,000 Dong for a bottle and fresh beer, which is actually very good, is 3,000 Dong (10p!!!!). The other thing which is very cheap, is 4 star accomodation! I can’t stop myself from reserving myself into the poshest hotel in every city I visit. It’s a bad habit, but this seems like one of the few countries in the world where it is an affordable option. You are looking at roughly 75 pounds/night for the highest end, although the Majestic did hit a 100. It was worth it though for the cocktails on the top floor balcony, overlooking the Sai Gon River. Very Graham Greene/The Quiet American.

As to the issue of travelling alone. I’m enjoying it a lot. The bad bits are occasional bouts of boredom with one’s own thoughts and the Western perception that a man travelling alone is something slightly strange. The uncomfortable moment comes at breakfast time in these swanky hotels when all the Amercian and English middle-age types and honeymooners are gorging on their Continental servings of bacon, omelettes, croissants and coffee. They avoid my friendly waves and smiles and I imagine, in my deepest mode of paranoia, that they think I am a sex tourist, here to indulge dark cravings for the things of beauty that are Vietnami women. Or perhaps they think I just look a bit scruffy for this type of place!

Talking of sex tourists, I have seen a few Northern English men in their 50’s/60’s walking hand-in-hand with some stunning 20 year old beauties. Just another aspect of globalisation I suppose.

The benefits of single travel are fairly numerous. No need to make any decisions until you have to. The ability to stay in posh places without having to justify the cost to anyone else. The potential for serendipity to kick in is greater. More able to meet unlikely people (such as a bunch of sailors gambling on the slow train from Hoi-An to Hue). The inclination to meet Vietnami people is greater, if simply for the conversation. In fact, I suppose subconsciously, I have been veering away from trying to find other Western travellers to hang out with, preferring the more difficult task of speaking to Non-English speakers.

Now, the food … The reason I came really, apart from the Socialism! It gets better the further North you go. Sai Gon was all Pho Ga and Pho Bo. Hue and Hoi-An both have their speciality dishes. Cao Lau (porky crouton soup) and Banh Bao (White Roses - soft dumplings) where omnipresent in every Hoi-An eatery and where very good. The Hoi-An fish was about as fresh as I’ve ever eaten. Hue offers the Banh Khoai (Song Que style pancake mixed with a very peanuty sauce and unripe banana!) and other dishes that I intend to devour tonight. I am resolving to eat more street food too, served on the pavement and seeming to be the most popular amongst the locals. Trouble is the rain keeps falling. Apparently it’s drier in the North, where I’m heading next (Sa Pa). The more remote accomodation seem to offer breakfast, lunch and dinner as part of the deal. The Ki-Em place was of this ilk and the food there was, while simple, incredibly tasty and beautifully presented. It was a compromise between Western dishes and Vietnamese ingredients, but it worked very well and I spent four days happily stuffing my face there and then swimming in the beautiful ocean.

A few travel tips:

• Dollars are accepted everywhere except train stations and are easier to convert into mental pounds (divide by 2), but you tend to get change in Dong and some less scrupulous cigarette vendors/ taxi drivers will use a less favourable exchange rate. Also $1 is the smallest denominator available. Better to operate in Dong and give the exact amount where possible.
• Carry a wallet where you can conceal the vast majority of your dosh in a zipped section. Every time you pay for something, the seller will have a good peer in your wallet, and if it’s visibly stuffed with $20 bills your bartering edge has definitely been lost.
• Don’t buy cigarettes from train station platforms - they are all fake!
• Talk football. Most people, including the girls, seem to know the names of their favourite teams full squad, along with transfer histories. West Ham is surprisingly popular!

My advice, come to Vietnam, it’s really good!

using System;
using System.Collections.Specialized;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;

public partial class UserDefinedFunctions
{
    [Microsoft.SqlServer.Server.SqlFunction]
    public static SqlString URLDecode(SqlString URL)
    {
       string[]Encoders=new string[]{\" ","%20","!","%21","\\"","%22","#","%23","$","%24",
           "%","%25","&","%26","'","%27","(","%28",")","%29","*","%2a","+","%2b",",","%2c",
           "-","%2d",".","%2e","/","%2f",":","%3a",";","%3b","<","%3c","=","%3d",">","%3e",
           "?","%3f","@","%40","[","%5b","\\\\","%5c","]","%5d","^","%5e","_","%5f","`","%60",
           "{","%7b","|","%7c","}","%7d","~","%7e","","%7f","€","%80","","%81","‚","%82",
           "ƒ","%83","„","%84","…","%85","†","%86","‡","%87","ˆ","%88","‰","%89","Š","%8a",
           "‹","%8b","Œ","%8c","","%8d","Ž","%8e","","%8f","","%90","‘","%91","’","%92",
           "“","%93","”","%94","•","%95","–","%96","—","%97","˜","%98","™","%99","š",
           "%9a","›","%9b","œ","%9c","","%9d","ž","%9e","Ÿ","%9f","","%a0","¡","%a1",
           "¢","%a2","£","%a3","","%a4","¥","%a5","|","%a6","§","%a7","¨","%a8","©","%a9",
           "ª","%aa","«","%ab","¬","%ac","¯","%ad","®","%ae","¯","%af","°","%b0","±",
           "%b1","²","%b2","³","%b3","´","%b4","µ","%b5","¶","%b6","·","%b7","¸","%b8",
           "¹","%b9","º","%ba","»","%bb","¼","%bc","½","%bd","¾","%be","¿","%bf","À",
           "%c0","Á","%c1","Â","%c2","Ã","%c3","Ä","%c4","Å","%c5","Æ","%c6","Ç","%c7",
           "È","%c8","É","%c9","Ê","%ca","Ë","%cb","Ì","%cc","Í","%cd","Î","%ce","Ï",
           "%cf","Ð","%d0","Ñ","%d1","Ò","%d2","Ó","%d3","Ô","%d4","Õ","%d5","Ö","%d6",
           "","%d7","Ø","%d8","Ù","%d9","Ú","%da","Û","%db","Ü","%dc","Ý","%dd","Þ",
           "%de","ß","%df","à","%e0","á","%e1","â","%e2","ã","%e3","ä","%e4","å","%e5",
           "æ","%e6","ç","%e7","è","%e8","é","%e9","ê","%ea","ë","%eb","ì","%ec","í",
           "%ed","î","%ee","ï","%ef","ð","%f0","ñ","%f1","ò","%f2","ó","%f3","ô","%f4",
           "õ","%f5","ö","%f6","÷","%f7","ø","%f8","ù","%f9","ú","%fa","û","%fb","ü",
           "%fc","ý","%fd","þ","%fe","ÿ","%ff"};
        string URLString = URL.ToString();
        for (int i =1;i<Encoders.Length;i+=2)
        {
            if (URLString.Contains(Encoders[i]))
            {
                URLString = URLString.Replace(Encoders[i],Encoders[i-1]);
            }
            else if (URLString.Contains(Encoders[i].ToUpper()))
            {
                URLString = URLString.Replace(Encoders[i].ToUpper(),Encoders[i-1]);
            }
        }
        return new SqlString(URLString.ToString());
    }
};