ElasticSearch - Malformed Query (RoR)

Question

I'm trying to set up an existing project using elasticsearch and ruby-on-rails. I can get the project up and running however elasticsearch queries return various error message for me and none of my colleagues:

[400] {"error":{"root_cause":[{"type":"parsing_exception","reason":"[and] query malformed, no start_object after query name","line":1,"col":896}],"type":"parsing_exception","reason":"[and] query malformed, no start_object after query name","line":1,"col":896},"status":400}

This obviously leads me to believe it's something to do with my setup. I've installed the same version of them (2.4), however even the setup command that was supplied with the project:

rake environment elasticsearch:import:all DIR=app/models FORCE=y

gives me a timeout error. I've checked and the service is running. Any help as to what could be wrong with my set up would be greatly appreciated as I'm at a dead end for 2 days on this.

Thanks in advance.

EDIT: I don't think the issue is with the query itself as this works for my colleagues however it is as follows:

{: date_range => {: from => Mon,
    01 Aug 2016 00: 00: 00 UTC + 00: 00,
    : to => Mon,
    31 Jul 2017 00: 00: 00 UTC + 00: 00
  }, : activities_only => nil
}, @pupils = [# < Pupil id: 1042, first_name: "Mark", last_name: "Mekhaiel", date_of_birth: "2017-03-19", level: 0, group_id: 95, created_at: "2017-07-31 08:59:59", updated_at: "2017-07-31 09:48:27", username: "m.mekhaiel1", picture: 0, color: 0, archived: false, sub_group_id: nil, score: 20, deleted_at: nil > ], @search = # < PupilCreditSearch: 0x007fc961801308 @interval = : month, @page = 1, @per = 25, @pupil_id = [1042], @date_range = {: from => Mon,
  01 Aug 2016 00: 00: 00 UTC + 00: 00,
  : to => Mon,
  31 Jul 2017 00: 00: 00 UTC + 00: 00
}, @queries = [# < Search::Query: 0x007fc964791bb8 @query = nil, @fields = [], @type = : multi_match, @options = {: fields => [],
  : type => : cross_fields
} > ], @filters = [# < Search::Filter: 0x007fc964756db0 @name = : pupil_id, @value = [1042], @type = : terms, @options = {} > ], @aggregations = {: years_created => # < Search::Aggregation: 0x007fc9647a32f0 @type = : date_histogram,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_histogram,
    : params => {: interval => : year
    }
  } > ,
  : months_created => # < Search::Aggregation: 0x007fc9647a30c0 @type = : date_histogram,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_histogram,
    : params => {: interval => : month
    }
  } > ,
  : date_ranges => # < Search::Aggregation: 0x007fc9647a2800 @type = : date_range,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_range,
    : params => {: keyed => true,
      : ranges => [{: from => "now/w",
        : to => "now+1w/w",
        : key => : this_week
      }, {: from => "now/M",
        : to => "now+1M/M",
        : key => : this_month
      }, {: from => "now-1w/w",
        : to => "now-2w/w",
        : key => : last_week
      }, {: from => "now-1M/M",
        : to => "now-2M/M",
        : key => : last_month
      }, {: from => "2016-09-01",
        : to => "2017-08-31",
        : key => : academic_year
      }]
    }
  } > ,
  : groups => # < Search::Aggregation: 0x007fc9647a2260 @type = : terms,
  @field = : pupil_id,
  @options = {: aggs => {: all_dates => # < Search::Aggregation: 0x007fc9647a22d8 @type = : date_histogram,
      @field = : created_at,
      @options = {: params => {: interval => : month
        },
        : aggs => {: total_points => # < Search::Aggregation: 0x007fc9647a2440 @type = : sum,
          @field = : points,
          @options = {} > ,
          : running_total => # < Search::Aggregation: 0x007fc9647a2350 @type = : cumulative_sum,
          @field = nil,
          @options = {: params => {: buckets_path => "total_points"
            }
          } >
        }
      } >
    }
  } > ,
  : options => # < Search::Aggregation: 0x007fc9647a1518 @type = : global,
  @field = nil,
  @options = {: aggs => {: filtered => # < Search::Aggregation: 0x007fc9647a1590 @type = : filter,
      @field = nil,
      @options = {: params => {: and => [{: range => {: created_at => {: gte => Mon,
                01 Aug 2016 00: 00: 00 UTC + 00: 00,
                : lt => Mon,
                31 Jul 2017 00: 00: 00 UTC + 00: 00
              }
            }
          }, {: terms => {: pupil_id => [1042]
            }
          }]
        },
        : aggs => {: categories => # < Search::Aggregation: 0x007fc9647a1b80 @type = : terms,
          @field = "category",
          @options = {: ignore => true
          } > ,
          : topic_ids => # < Search::Aggregation: 0x007fc9647a19a0 @type = : terms,
          @field = "topic_id",
          @options = {: ignore => true
          } > ,
          : learning_objective_ids => # < Search::Aggregation: 0x007fc9647a17c0 @type = : terms,
          @field = "learning_objective_id",
          @options = {: ignore => true
          } > ,
          : completed_activities => # < Search::Aggregation: 0x007fc9647a1608 @type = : terms,
          @field = : pupil_id,
          @options = {: aggs => {: activities => # < Search::Aggregation: 0x007fc9647a1680 @type = : filter,
              @field = nil,
              @options = {: params => {: and => [{: terms => {: category => [0, 1, 2, 3]
                    }
                  }]
                }
              } >
            }
          } >
        }
      } >
    }
  } >
}, @request = # < Search::Request: 0x007fc964753548 @params = {: query => {: bool => {: must => [{: match_all => {}
      }],
      : filter => [{: terms => {: pupil_id => [1042]
        }
      }]
    }
  },
  : aggs => {: years_created => {: date_histogram => {: field => : created_at,
        : interval => : year
      }
    },
    : months_created => {: date_histogram => {: field => : created_at,
        : interval => : month
      }
    },
    : date_ranges => {: date_range => {: field => : created_at,
        : keyed => true,
        : ranges => [{: from => "now/w",
          : to => "now+1w/w",
          : key => : this_week
        }, {: from => "now/M",
          : to => "now+1M/M",
          : key => : this_month
        }, {: from => "now-1w/w",
          : to => "now-2w/w",
          : key => : last_week
        }, {: from => "now-1M/M",
          : to => "now-2M/M",
          : key => : last_month
        }, {: from => "2016-09-01",
          : to => "2017-08-31",
          : key => : academic_year
        }]
      }
    },
    : groups => {: terms => {: field => : pupil_id
      },
      : aggs => {: all_dates => {: date_histogram => {: field => : created_at,
            : interval => : month
          },
          : aggs => {: total_points => {: sum => {: field => : points
              }
            },
            : running_total => {: cumulative_sum => {: buckets_path => "total_points"
              }
            }
          }
        }
      }
    },
    : options => {: global => {},
      : aggs => {: filtered => {: filter => {: and => [{: range => {: created_at => {: gte => Mon,
                  01 Aug 2016 00: 00: 00 UTC + 00: 00,
                  : lt => Mon,
                  31 Jul 2017 00: 00: 00 UTC + 00: 00
                }
              }
            }, {: terms => {: pupil_id => [1042]
              }
            }]
          },
          : aggs => {: categories => {: terms => {: field => "category"
              }
            },
            : topic_ids => {: terms => {: field => "topic_id"
              }
            },
            : learning_objective_ids => {: terms => {: field => "learning_objective_id"
              }
            },
            : completed_activities => {: terms => {: field => : pupil_id
              },
              : aggs => {: activities => {: filter => {: and => [{: terms => {: category => [0, 1, 2, 3]
                      }
                    }]
                  }
                }
              }
            }
          }
        }
      }
    }
  }
} >> , @options = # < ReportOptions: 0x007fc96b663258 @search = # < PupilCreditSearch: 0x007fc961801308 @interval = : month, @page = 1, @per = 25, @pupil_id = [1042], @date_range = {: from => Mon,
  01 Aug 2016 00: 00: 00 UTC + 00: 00,
  : to => Mon,
  31 Jul 2017 00: 00: 00 UTC + 00: 00
}, @queries = [# < Search::Query: 0x007fc964791bb8 @query = nil, @fields = [], @type = : multi_match, @options = {: fields => [],
  : type => : cross_fields
} > ], @filters = [# < Search::Filter: 0x007fc964756db0 @name = : pupil_id, @value = [1042], @type = : terms, @options = {} > ], @aggregations = {: years_created => # < Search::Aggregation: 0x007fc9647a32f0 @type = : date_histogram,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_histogram,
    : params => {: interval => : year
    }
  } > ,
  : months_created => # < Search::Aggregation: 0x007fc9647a30c0 @type = : date_histogram,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_histogram,
    : params => {: interval => : month
    }
  } > ,
  : date_ranges => # < Search::Aggregation: 0x007fc9647a2800 @type = : date_range,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_range,
    : params => {: keyed => true,
      : ranges => [{: from => "now/w",
        : to => "now+1w/w",
        : key => : this_week
      }, {: from => "now/M",
        : to => "now+1M/M",
        : key => : this_month
      }, {: from => "now-1w/w",
        : to => "now-2w/w",
        : key => : last_week
      }, {: from => "now-1M/M",
        : to => "now-2M/M",
        : key => : last_month
      }, {: from => "2016-09-01",
        : to => "2017-08-31",
        : key => : academic_year
      }]
    }
  } > ,
  : groups => # < Search::Aggregation: 0x007fc9647a2260 @type = : terms,
  @field = : pupil_id,
  @options = {: aggs => {: all_dates => # < Search::Aggregation: 0x007fc9647a22d8 @type = : date_histogram,
      @field = : created_at,
      @options = {: params => {: interval => : month
        },
        : aggs => {: total_points => # < Search::Aggregation: 0x007fc9647a2440 @type = : sum,
          @field = : points,
          @options = {} > ,
          : running_total => # < Search::Aggregation: 0x007fc9647a2350 @type = : cumulative_sum,
          @field = nil,
          @options = {: params => {: buckets_path => "total_points"
            }
          } >
        }
      } >
    }
  } > ,
  : options => # < Search::Aggregation: 0x007fc9647a1518 @type = : global,
  @field = nil,
  @options = {: aggs => {: filtered => # < Search::Aggregation: 0x007fc9647a1590 @type = : filter,
      @field = nil,
      @options = {: params => {: and => [{: range => {: created_at => {: gte => Mon,
                01 Aug 2016 00: 00: 00 UTC + 00: 00,
                : lt => Mon,
                31 Jul 2017 00: 00: 00 UTC + 00: 00
              }
            }
          }, {: terms => {: pupil_id => [1042]
            }
          }]
        },
        : aggs => {: categories => # < Search::Aggregation: 0x007fc9647a1b80 @type = : terms,
          @field = "category",
          @options = {: ignore => true
          } > ,
          : topic_ids => # < Search::Aggregation: 0x007fc9647a19a0 @type = : terms,
          @field = "topic_id",
          @options = {: ignore => true
          } > ,
          : learning_objective_ids => # < Search::Aggregation: 0x007fc9647a17c0 @type = : terms,
          @field = "learning_objective_id",
          @options = {: ignore => true
          } > ,
          : completed_activities => # < Search::Aggregation: 0x007fc9647a1608 @type = : terms,
          @field = : pupil_id,
          @options = {: aggs => {: activities => # < Search::Aggregation: 0x007fc9647a1680 @type = : filter,
              @field = nil,
              @options = {: params => {: and => [{: terms => {: category => [0, 1, 2, 3]
                    }
                  }]
                }
              } >
            }
          } >
        }
      } >
    }
  } >
}, @request = # < Search::Request: 0x007fc964753548 @params = {: query => {: bool => {: must => [{: match_all => {}
      }],
      : filter => [{: terms => {: pupil_id => [1042]
        }
      }]
    }
  },
  : aggs => {: years_created => {: date_histogram => {: field => : created_at,
        : interval => : year
      }
    },
    : months_created => {: date_histogram => {: field => : created_at,
        : interval => : month
      }
    },
    : date_ranges => {: date_range => {: field => : created_at,
        : keyed => true,
        : ranges => [{: from => "now/w",
          : to => "now+1w/w",
          : key => : this_week
        }, {: from => "now/M",
          : to => "now+1M/M",
          : key => : this_month
        }, {: from => "now-1w/w",
          : to => "now-2w/w",
          : key => : last_week
        }, {: from => "now-1M/M",
          : to => "now-2M/M",
          : key => : last_month
        }, {: from => "2016-09-01",
          : to => "2017-08-31",
          : key => : academic_year
        }]
      }
    },
    : groups => {: terms => {: field => : pupil_id
      },
      : aggs => {: all_dates => {: date_histogram => {: field => : created_at,
            : interval => : month
          },
          : aggs => {: total_points => {: sum => {: field => : points
              }
            },
            : running_total => {: cumulative_sum => {: buckets_path => "total_points"
              }
            }
          }
        }
      }
    },
    : options => {: global => {},
      : aggs => {: filtered => {: filter => {: and => [{: range => {: created_at => {: gte => Mon,
                  01 Aug 2016 00: 00: 00 UTC + 00: 00,
                  : lt => Mon,
                  31 Jul 2017 00: 00: 00 UTC + 00: 00
                }
              }
            }, {: terms => {: pupil_id => [1042]
              }
            }]
          },
          : aggs => {: categories => {: terms => {: field => "category"
              }
            },
            : topic_ids => {: terms => {: field => "topic_id"
              }
            },
            : learning_objective_ids => {: terms => {: field => "learning_objective_id"
              }
            },
            : completed_activities => {: terms => {: field => : pupil_id
              },
              : aggs => {: activities => {: filter => {: and => [{: terms => {: category => [0, 1, 2, 3]
                      }
                    }]
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

do you use the [elasticsearch-ruby ](https://github.com/elastic/elasticsearch-ruby) gem ? — Julien TASSIN, Aug 04 '17 at 17:48
posting your query and sample document you are trying to query and output you are trying to achieve will help — user93, Aug 07 '17 at 02:03
Sorry for the lack of comments - I was away all weekend, I'll update the question with the requested info — Mark, Aug 07 '17 at 09:40
elasticsearch : parsing_exception: no [query] registered for [filtered] :https://github.com/laravel/scout/issues/122,Elastic Search is returning a 400 parsing_exception error https://github.com/nuagenetworks/visualization-framework/issues/324 — Neha Chopra, Aug 10 '17 at 18:08
It has to be a malformed query. Looks like what you have pasted for query is Ruby code that uses instance variables and custom Ruby objects. Any one of these might be outputting invalid components that break the query. Can you please paste the eventual JSON object that is passed to Elasticsearch? — Sahil, Aug 11 '17 at 00:05
Can you check ruby version, elasticsearch version, ruby gem elasticsearch version, (against version your colleagues use) then from your computer, launch your apps to use the ES of one of your colleague. And do the opposite (one colleague use your elastic). You will narrow a little bit your issue. — wargre, Aug 11 '17 at 09:36
Are you able to connect to the elasticsearch with other means? Are you sure it is installed correctly and running? — Automatico, Aug 11 '17 at 12:18
Check the version of elasticsearch on your machine and your colleagues machine and also the version in gemfile.lock. — Aditya Varma, Aug 22 '17 at 12:14

score 2 · Accepted Answer · answered Nov 18 '17 at 02:27

The problem is with one of the aggregations, namely:

:filter => { :and => [...] }

...which leads me to conclusion that you actually may be running elasticsearch 5 and not 2.4, on your machine, because this syntax got dropped in version 5 in favour of:

:filter => { :bool => { :must: [...] } }

Verify the elasticsearch version with curl localhost:9200 (or the actual host/port your app is connecting to).

ElasticSearch - Malformed Query (RoR)

1 Answers1

Related