How to use paginate() with a having() clause when column does not exist in table

Question

I have a tricky case ...

Following database query does not work:

DB::table('posts')
->select('posts.*', DB::raw($haversineSQL . ' as distance'))
->having('distance', '<=', $distance)
->paginate(10);

It fails with message: column distance does not exist.

The error occurs when paginate() tries to count the records with

select count(*) as aggregate from {query without the column names}

As the column names are stripped, distance is not known and an exception is raised.

Does somebody have a work around to be able to use pagination is this case ?

Thanks

Answer 1

You can calculate the distance in the WHERE part:

DB::table('posts')
    ->whereRaw($haversineSQL . '<= ?', [$distance])
    ->paginate(10);

If you need the distance value in your application, you'll have to calculate it twice:

DB::table('posts')
    ->select('posts.*', DB::raw($haversineSQL . ' as distance'))
    ->whereRaw($haversineSQL . '<= ?', [$distance])
    ->paginate(10);

Answer 2

This is somewhat of a problem with the query builder as all selects are discarded when doing an aggregate call (like count(*)). The make-do solution for now is to construct the pagniator manually.

$query = DB::table('posts')
    ->select(DB::raw('(c1 - c2) as distance'))
    ->having('distance', '<=', 5);

$perPage = 10;
$curPage = Paginator::getCurrentPage(); // reads the query string, defaults to 1

// clone the query to make 100% sure we don't have any overwriting
$itemQuery = clone $query;
$itemQuery->addSelect('posts.*');
// this does the sql limit/offset needed to get the correct subset of items
$items = $itemQuery->forPage($curPage, $perPage)->get();

// manually run a query to select the total item count
// use addSelect instead of select to append
$totalResult = $query->addSelect(DB::raw('count(*) as count'))->get();
$totalItems = $totalResult[0]->count;

// make the paginator, which is the same as returned from paginate()
// all() will return an array of models from the collection.
$paginatedItems = Paginator::make($items->all(), $totalItems, $perPage);

Tested with the following schema using MySQL:

Schema::create('posts', function($t) {
    $t->increments('id');
    $t->integer('c1');
    $t->integer('c2');
});

for ($i=0; $i < 100; $i++) { 
    DB::table('posts')->insert([
        'c1' => rand(0, 10),
        'c2' => rand(0, 10),
    ]);
}

Answer 3

This is not a satisfying answer, but if you only need to display simple "Next" and "Previous" links in your pagination view, you may use the simplePaginate method. It will perform a more efficient query and it won't crash if you use having.

DB::table('posts')
->select('posts.*', DB::raw($haversineSQL . ' as distance'))
->having('distance', '<=', $distance)
->simplePaginate(10);

Answer 4

There is a better way and also works with links:

        $curPage = \Illuminate\Pagination\Paginator::resolveCurrentPage();
        $total = $model->get()->count();
        $items = $model->forPage($curPage, $showPerPag)->get();
        $paginated = new \Illuminate\Pagination\LengthAwarePaginator($items, $total, $showPerPage, $curPage, ['path' => request()->url(), 'query' => request()->query()]);

Answer 5

Using Eloquent, I know you can pass columns to the paginator, something like this:

Post::having('distance','<=', $distance)
   ->paginate(10, array('*', DB::raw($haversineSQL . ' as distance')));

Not sure if it works without Eloquent, but you could give it a try.

Answer 6

This is scope implements a Haversine formula search, with additional optimization for speed, which is documented here.

I wish there was a cleaner way to get raw SQL from the query object, but unfortunately toSql() returns SQL before the placeholders have been substituted, so I relied on several *Raw calls. It's not too bad, but I wish it was cleaner.

The code assumes you have columns lat and lng in your table.

const DISTANCE_UNIT_KILOMETERS = 111.045;
const DISTANCE_UNIT_MILES      = 69.0;

/**
 * @param $query
 * @param $lat
 * @param $lng
 * @param $radius numeric
 * @param $units string|['K', 'M']
 */
public function scopeNearLatLng($query, $lat, $lng, $radius = 10, $units = 'K')
{
    $distanceUnit = $this->distanceUnit($units);

    if (!(is_numeric($lat) && $lat >= -90 && $lat <= 90)) {
        throw new Exception("Latitude must be between -90 and 90 degrees.");
    }

    if (!(is_numeric($lng) && $lng >= -180 && $lng <= 180)) {
        throw new Exception("Longitude must be between -180 and 180 degrees.");
    }

    $haversine = sprintf('*, (%f * DEGREES(ACOS(COS(RADIANS(%f)) * COS(RADIANS(lat)) * COS(RADIANS(%f - lng)) + SIN(RADIANS(%f)) * SIN(RADIANS(lat))))) AS distance',
        $distanceUnit,
        $lat,
        $lng,
        $lat
    );

    $subselect = clone $query;
    $subselect
        ->selectRaw(DB::raw($haversine));

    // Optimize the query, see details here:
    // http://www.plumislandmedia.net/mysql/haversine-mysql-nearest-loc/

    $latDistance      = $radius / $distanceUnit;
    $latNorthBoundary = $lat - $latDistance;
    $latSouthBoundary = $lat + $latDistance;
    $subselect->whereRaw(sprintf("lat BETWEEN %f AND %f", $latNorthBoundary, $latSouthBoundary));

    $lngDistance     = $radius / ($distanceUnit * cos(deg2rad($lat)));
    $lngEastBoundary = $lng - $lngDistance;
    $lngWestBoundary = $lng + $lngDistance;
    $subselect->whereRaw(sprintf("lng BETWEEN %f AND %f", $lngEastBoundary, $lngWestBoundary));

    $query
        ->from(DB::raw('(' . $subselect->toSql() . ') as d'))
        ->where('distance', '<=', $radius);
}

/**
 * @param $units
 */
private function distanceUnit($units = 'K')
{
    if ($units == 'K') {
        return static::DISTANCE_UNIT_KILOMETERS;
    } elseif ($units == 'M') {
        return static::DISTANCE_UNIT_MILES;
    } else {
        throw new Exception("Unknown distance unit measure '$units'.");
    }
}

This can be used as such:

        $places->NearLatLng($lat, $lng, $radius, $units);
        $places->orderBy('distance');

The SQL generated, will look approximately like this:

select
  *
from
  (
    select
      *,
      (
        '111.045' * DEGREES(
          ACOS(
            COS(
              RADIANS('45.5088')
            ) * COS(
              RADIANS(lat)
            ) * COS(
              RADIANS('-73.5878' - lng)
            ) + SIN(
              RADIANS('45.5088')
            ) * SIN(
              RADIANS(lat)
            )
          )
        )
      ) AS distance
    from
      `places`
    where lat BETWEEN 45.418746  AND 45.598854
      and lng BETWEEN -73.716301 AND -73.459299
  ) as d
where `distance` <= 10
order by `distance` asc

Answer 7

You can use manual pagination as having behaving peculiar with pagination class.

$posts = DB::table('posts')
    ->select('posts.*', DB::raw($haversineSQL . ' as distance'))
    ->having('distance', '<=', $distance)
    ->get();

// Items per page
$perPage = 10;
$totalItems = count($posts);
$totalPages = ceil($totalItems / $perPage);

$page = Input::get('page', 1);

if ($page > $totalPages or $page < 1) {
    $page = 1;
}

$offset = ($page * $perPage) - $perPage;

$posts = array_slice($posts, $offset, $perPage);

$posts = Paginator::make($posts, $totalItems, $perPage);


dd($posts);