How to filter non-address information from a string

Question

I'm trying to reduce the time users need to spent on filling in an address form. This form requires the street address, postal code, city, district, and sub district.

To do so I query Open Street Map's Nominatim API like so:

var
  request = require('request'),
  address = 'Grand Parkview Asoke, Unit 255/109 15th Flr., Sukhumvit 21 Road',
  baseUri = 'http://nominatim.openstreetmap.org/search?format=json&addressdetails=1',
  query   = '&accept-language=en&q=' + encodeURIComponent(address);

request(baseUri + query, function(err, res, body) {
  console.log(JSON.parse(body));
});

I then parse things like the postcode from the returned body.

The problem is that this only works for "normal addresses" that do not contain irrelevant things like the floor number. In other words, this works:

var address = 'Sukhumvit 21 Road';

But this does not work:

var address = 'Grand Parkview Asoke, Unit 255/109 15th Flr., Sukhumvit 21 Road';

Now I am querying the API many times with a very crude set of possible trials, like so:

  //create trials
  var
    trials = [],
    addressParts = address.split(' ');

  for (var i = 0, il = addressParts.length; i < il; i++) {
    if (il - i >= 2) trials.push(addressParts.slice(i, il).join(' '));
  }

Which means that it will attempt all of these strings:

Grand Parkview Asoke, Unit 255/109 15th Flr., Sukhumvit 21 Road
Parkview Asoke, Unit 255/109 15th Flr., Sukhumvit 21 Road
Asoke, Unit 255/109 15th Flr., Sukhumvit 21 Road
Unit 255/109 15th Flr., Sukhumvit 21 Road
255/109 15th Flr., Sukhumvit 21 Road
15th Flr., Sukhumvit 21 Road
Flr., Sukhumvit 21 Road
Sukhumvit 21 Road ==> it works!

Which requires many requests and is therefore very slow.

Is there a smarter way to filter out this "non-address" information? Note that I'm also looking for a way to do this in non-western scripts such as Thai.

Answer 1

If you do not have a well defined form with specific address fields. A good approach would be to split the string address into well defined parts. eg.

"Grand Parkview Asoke, Unit 255/109 15th Flr., Sukhumvit 21 Road"
 1st part: "Grand Parkview Asoke"
 2nd part: "Unit 255/109 15th Flr."
 3rd part: "Sukhumvit 21 Road"
 // Of course it would be more complex than just splitting at commas.

Addresses traditionally are written in a form where information is dense towards the end of the string. By dense I mean the probability of the amount of information available on internet is on the higher side. So a query with the last part ie. Sukhumvit 21 Road on the source of information is likely to give you more results than the full string in one shot.

Now depending on the no. of results you receive. you may build an approach like:
1) More than one results: Add more information to your query string eg. Unit 255/109 15th Flr., Sukhumvit 21 Road
2) No results: Remove a part of the query string eg. 21 Road

Else as others have already suggested, if you break down your form into distinct address parts viz. Street address, etc. You will be in a better state to form queries.

Then again, this is what I though. There will definitely be much better approaches based on mathematical modeling of the problem.

Answer 2

You should look at sequence tagging problems---this is essentially what you have here. One prominent example is named entity recognition.

The task is to see what you're trying to extract from the string as words with a particular tag. Let's say the tag is 'Relavant'. You can then think of each word in your string as having a corresponding tag:

'Grand Parkview Asoke, Unit 255/109 15th Flr., Sukhumvit 21 Road'
'[NR]  [NR]            [NR] [NR]    [NR] [NR]  [R]       [R][R]

where I used the tag [R] to indicate the word is relevant to the query. You can then build your query with just the relevant words (or indeed the longest contiguous string of relevant words to increase robustness, if that's more suitable).

The task then is to build a sequence tagger that identifies relevant from irrelevant words in the query. You'll want to approach this as a supervised problem, which means training data (although you could acquire training data by assuming that any query that returns a result is valid and all others are invalid). The most competitive sequence taggers are conditional random fields, which can do tagging quickly and with high accuracy.

Beware though---this isn't a quick fix. You'll need to invest a fair amount of time into gathering the data, identifying relevant features, and evaluating. I don't know how important this stuff is to you!