How does the use of startrow and stoprow not result in a full table scan in HBase?

Question

It is commonly suggested to use a range scan via startrow and stoprow as opposed to a Rowkey Prefix Filter (for example, here). The reasoning for this is because a Rowkey Prefix Filter results in a full table scan of the rowkey, whereas a range scan via startrow and stoprow do not result in a full table scan. Why doesn't it? Most people say "because the rowkey is stored in lexographical order," which of course, doesn't explain why the Rowkey Prefix Filter cannot leverage this.

At anyrate, how exactly does a range scan via startrow and stoprow not result in a full table scan of the rowkey?

Take this small example in python to show why I don't understand how the lexagraphical ordering of the rowkeys means anything when it comes to avoiding a full table scan:

rowkeys = ['a1', 'a2', 'a3', 'b1', 'b2', 'b3', 'c1', 'c2', 'c3']

def range_scan(startrow, stoprow):
    is_found = False
    for rowkey in rowkeys:
        if startrow <= rowkey < stoprow:
            is_found = True
            yield rowkey
        else:
            if is_found:
                raise StopIteration()

Clearly, the HBase algorithm differs from this. How does it?

TLDR: How exactly does HBase avoid a full table scan when doing a range scan with startrow and stoprow?

score 8 · Accepted Answer · answered Oct 23 '16 at 15:33

In HBase a table is split into regions. A region is a set of rows that is served by a specific region server. A region server has (in general) multiple regions from multiple tables for which it handles the requests.

Because the rows are sorted by key it is known in the hbase master which start and stop keys are the boundaries of a each region AND on which region server this region can be queried.

So if a scan is done that uses an explicit start and stop row then the query is directed ONLY to the regions/region servers that have keys that may be in the requested range.

If the query has a 'small' range of keys then you'll find that in almost all queries only a single region is accessed.

It is common to have dozens of regions for a HBase table and by using the start and stop row to limit the scan you may find that 99 of the 100 regions of your table do not even know a query on the table has been done.

How does the use of startrow and stoprow not result in a full table scan in HBase?

1 Answers1

Linked