View does not update to add object that was previously ignored

Question

There is no document in view after the document has been inserted in bucket.

My document:

{
  "state": 1,
  "rdbms_id": 0,  
  "startDate": 1511882685998,
  "endDate": -1,  
  "type": "kv",
  "userid": 1222,
  "uuid": "84fd36ad-b8bd-4abb-90ac-eae407f9364a",  
  "amount": 1234,
  "source_id": 12
}

View index code:

function (doc, meta) {
  if(meta.type=="json"){        
    if(doc.type && doc.type === "kv"){      
      if(doc.startDate && doc.startDate<=Date.now()){
        if(doc.endDate && (doc.endDate>=Date.now() || doc.endDate==-1)){
          if(doc.state && doc.state==1){
          emit([doc.userid,doc.source_id], null);
          }
        }
      }
    }
  }
}

My view query:

curl http://Administrator:pass@localhost:8092/bss_write/_design/fihrist/_view/forUcrKVIds?limit=6&stale=false&connection_timeout=60000&inclusive_end=true&skip=0

View is updated when adding document with curl.

curl -H "Content-Type: application/json" -X POST -v -d 'Jsondocument' 'http://Administrator:pass@localhost:8091/pools/default/buckets/bss_write/docs/kv-84fd36ad-b8bd-4abb-90ac-eae407f9364a'

But the view is not updated when you add it with the program.

Cluster cluster = CouchbaseCluster.create("localhost");
cluster.authenticate("username", "");
Bucket bucket = cluster.openBucket("bss_write");

JsonObject doc = JsonObject.fromJson(Jsondocument);

bucket.upsert(JsonDocument.create(type+"-"+uuid, doc));

Do I need to run a procedure to programmatically update the view?

Update 2:

The problem is the time difference. When the document is added, the view is not updated because the date condition is not matched.

However, the view is not updated even if the view conditions are met after the document is added.

For example:

My computer timestamp is 1512054982454.
Couchbase server timestamp is 1512054876554.

function (doc, meta) {
  if(meta.type=="json"){        
    if(doc.type && doc.type === "kv"){      
---->if(doc.startDate && doc.startDate<=Date.now()){ //This returns false.
        if(doc.endDate && (doc.endDate>=Date.now() || doc.endDate==-1)){
          if(doc.state && doc.state==1){
          emit([doc.userid,doc.source_id], null);
          }
        }
      }
    }
  }
}

Does it not show up *ever*? Or just not immediately? Couchbase views often lag significantly behind the main data store due to the time required to re-index them. — theMayer, Dec 04 '17 at 18:16
Although I wait for a long time, it never appears. re-indexing is only done by removing and re-creating the view. — onder, Dec 05 '17 at 06:11
Ahh, I think I understand what you are saying. You're wanting the view to dynamically update itself. Such a thing is not possible without a corresponding update to the object itself. — theMayer, Dec 05 '17 at 15:56

score 2 · Accepted Answer · edited Jun 20 '20 at 09:12

Background

Let's break down what is going on here, from a high level:

A document is added to the bucket
The view engine processes the document to determine if it gets added to the view.
If a series of conditions are met, as described in your question, the document is added to the view. If not, the document is ignored. Specifically,

if(doc.startDate && doc.startDate<=Date.now())
If, at a later time, the document is updated, then the view will re-process it and it may or may not get included.

The issue

A basic issue exists, and that is the distinction between compilation of the view and query from the view. When you compile the view, it needs to include all possible combinations that you may want to select in the future. When you query from the view, only those items that meet the given query criteria are to be returned.

The problem here is that the view compilation phase is performing the query phase. The compile logic runs once for each object, and will not run again unless the object changes or you tell CB to recompute the view. If your compilation does not include the object in the first place, it never will.

The solution

Change your map/reduce function to emit all objects that you want, removing any dependencies on Date.Now. Change your query code to select only items matching the given criteria that is presently defined in your query.

Note that it may be more efficient to load the results of a broad query of the view into memory on your web service host, using java code to perform granular selection prior to serving the results to the client. I have found extremely high performance to result from this approach.

View does not update to add object that was previously ignored

1 Answers1