Is spacial search in P2P network possible?

Question

I want to build a Javascript/HTML5 geolocation based social network and I wonder the best choice of possible architectures. Client-server can be simple to develop but drawback is the system ressources that could be very high, especially because the application must manage moves (worst case: a user that is in a car must see others users that are around him in cars).

Basicaly, in a client-server architecture, server tasks will be :

1. collects and stores latitude and longitude of the users (could have thousands of them)
2. makes geo distance search for that user (to get the list of users present around him in a radius)
3. builds and sends to the client an XML file with position of the users in the list

These 3 operation must be done periodically, every 3 or 5 seconds because I want a "live" map that shows users in the list moving in their environnement (city, town).

All these 3 points could be optimized :

1. client send his position when moving of 10 meters to reduce amount of data to process
2. "spherical rectangle" search in MyISAM table with spatial index (use of MBRContains) to off load MySQL database.
3. common output file : the XML that is sent can be the same if 2 users are located in a radius of x meters (the 2 users are close each-other).

It is hard to make load estimation at this stage but I think client-server architecture is not appropriate for that type of application and peer2peer could be a nice answer if 2 clients could communicate when they are near each other.

My point is:

Is there any methode to make possible a client to blind search other clients that are located in a certain radius without the help of a central server ? (it is possible with UDP broadcast :-)

edit : Correction. UDP Brodcast allow a client to poll a machine wherever it is, in certain range or IP address.

Thank you for your help, Florent

Answer 1

The answer actually depends on many things so I'll help out with basic strategy. To understand things out you'll need to understand how does Kademlia works (Kademlia is a DHT P2P network that stores information).

In Kademlia at first startup each node picks random ID which is a 160 bit number that represents point in a space of all possible 160 bit IDs.

The ID of the information that needs to be stored is obtained with SHA-1 function (it receives arbitrary string, and outputs 160 bit number that is treated like ID of the information that needs to be stored)

After that you have the ID of the information, you publish it, the information is physically stored on a node that has it's ID close to information ID.

(The illustration is taken from here)

The information is queried via it's ID. Both the information lookups or node lookups takes O(log(N)) hops to obtain the required information. The "XOR" metric is used in Kademlia (in your case it can be ordinary Euclidian metric).

Each node maintains an array of buckets, each bucket contains addresses of nodes that are appropriate to the current bucket. The appropriate'ness is a measure of how close the IDs are. consider example:

           0                              160
Node 1 ID: 1101000101011111101110101001010...
Node 2 ID: 1101011101011111101110101001010...
Node 3 ID: 1101000101011001101110101001010...

After applying XOR metric to Nodes #1,2 i.e (computing the number that represents the virtual distance between these nodes) we get:

index - 012345678901234
xor   - 000001100000000... (the difference is in 5-th msb bit)
order - msb         lsb

After applying Xor metric to Nodes #1,3 we get:

index - 012345678901234
xor   - 000000000000011... (the difference is in 13-th msb bit)
order - msb         lsb

Apparently Node 1 is closer to Node 3 since it has difference in less significant bits than the distance from Node 1 to Node 2. And therefore from a point of view of a Node 1, it's neighbor Node 3 goes to 13-th bucket(higher index means closer IDs), and Node 2 goes to to 5-th bucket which contains a group of nodes that are 5 MSB radixes away from a current node ID.

Such data structure allows each node to know it's surroundings in variety of 160 levels of distances.

Back to your example, to allow efficient geospacial queries you'll need to replace Kademlias XOR metric with ordinary Euclidian metric. In this case you will have your ID's as a 3D or 2D vectors, and unfortunately due to fact that Euclidian metric results with floating point numbers which are not directly suitable for this type of algorithm so you will need to convert them to a discrete binary numbers somehow in a way similar to what XOR function does. After that, finding node's neighboring nodes is a trivial task.

Hope this helps. Oh by the way look to HyperDex, new searchable distributed datastore closely tied to euclidian metric, might help...

Answer 2

You will have to have central peers/servers, because you need to centralize some information to be able to perform you functionalities.

I would go for the following:

1. Assign square miles (or whatever size you want) to specific servers.

2. Have devices send a 'I am here' message with their coordinates to some dispatcher that will forward these to the correct square mile server for handling.

3. Have servers register when a device enters a square mile they manage. This could be a central map to make sure a device is registered to one and only one square.

4. Forward this message to all other devices in the square.

5. And/or make sure you include to which square this message is intended and make sure the devices checks it before displays it to the user.

Tune the size of the square and the rate of 'I am here' message. That's it.