Postgresql too slow spatial intersection analyse

Question

I have a postgres database with a table (approx. 150 000 rows) with three columns; row_id, position and number of flowers counted. Since the position is rather close to each other might there be some error in the counting and I want to know the average count for each position, so I wrote following sql question:

WITH buffered_zones
     AS (SELECT row_id,
                St_buffer(pos, 0.00001) AS buffers
         FROM   flower_tbl
         limit 100),
     mean_vals
     AS (SELECT ft.row_id,
                pos,
                (SELECT Avg(ft.counted_flowers)
                 FROM   buffered_zones bz
                 WHERE  St_intersects(ft.pos, buffers)
                        AND bz.row_id = ft.row_id) AS counted_flowers
         FROM   flower_tbl ft
         GROUP  BY ft.row_id,
                   pos)
SELECT *
FROM   mean_vals

Which works and give me the average count for the first 100 points, which takes approx 15 sec. When I increase the limit to 1000 the run time increase to approx 1,5 min and so on. Hence I want to run the question for all 150 000 rows in less then say 30 sec. Any suggestions? There are a btree index on the row_id and a gist_index on the pos.

I added the query plan:

CTE Scan on mean_vals  (cost=4219889.10..4222939.24 rows=152507 width=44) (actual time=214.941..8703.535 rows=152507 loops=1)
  CTE buffered_zones
->  Limit  (cost=0.00..28.40 rows=100 width=36) (actual time=0.957..23.034 rows=100 loops=1)
      ->  Seq Scan on flower_table  (cost=0.00..43311.82 rows=152507 width=36) (actual time=0.955..23.003 rows=100 loops=1)
  CTE mean_vals
->  GroupAggregate  (cost=22486.79..4219860.70 rows=152507 width=40) (actual time=214.938..8583.157 rows=152507 loops=1)
      Group Key: ft.row_id, ft.pos
      ->  Sort  (cost=22486.79..22868.06 rows=152507 width=40) (actual time=191.499..249.276 rows=152507 loops=1)
            Sort Key: ft.row_id, ft.pos
            Sort Method: external sort  Disk: 7448kB
            ->  Seq Scan on flower_table ft  (cost=0.00..5185.07 rows=152507     width=40) (actual time=0.014..66.537 rows=152507 loops=1)
          SubPlan 2
            ->  CTE Scan on buffered_zones bz  (cost=0.00..27.50 rows=1 width=0)     (actual time=0.053..0.053 rows=0 loops=152507)
              Filter: ((ft.pos && buffers) AND (row_id = ft.row_id) AND _st_intersects(ft.pos, buffers))
                  Rows Removed by Filter: 100
Planning time: 1.868 ms
Execution time: 8726.110 ms