ElasticSearch - Different results for searching same index with 5 or 1 shards

Question

I am running a single node with ElasticSearch (1.4.4) and I have two indexes with the same documents indexed. The only difference is in the number of shards:

• Index 1: 5 shards, 0 replicas
• Index 2: 1 shard, 0 replicas

When I run a test of 100 different queries using the Java API, I get different results for each index. I have tried using DFS Query Then Fetch for both of them, but I still get different results.

I am wondering what's happening. I am not sure how elasticsearch is performing the queries in order to get different results.

UPDATE:

These are the settings and mappings for my indexes:

"settings": {
      "number_of_shards" :   1, // or 5 for the other index
      "number_of_replicas" : 0,
      "analysis": {
               "filter": {
                  "worddelimiter": {
                     "type": "word_delimiter"
                  }
               },
               "analyzer": {
                  "custom_analyzer": {
                     "type": "custom",
                     "filter": [
                        "lowercase",
                        "asciifolding",
                        "worddelimiter"
                     ],
                     "tokenizer": "standard"
                  }
               }
            }
  }

"mappings": {
     "faq": {
        "properties": {
           "answer": {
              "type": "string",
              "analyzer": "custom_analyzer"
           },
           "answer_id": {
              "type": "long"
           },
           "path": {
              "type": "string",
              "analyzer": "custom_analyzer"
           },
           "question": {
              "type": "string",
              "analyzer": "custom_analyzer"
           }
        }
     }
  }

And this is the query:

client.prepareSearch(index)
                    .setTypes(type)
                    .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) 
                    .setQuery(QueryBuilders.multiMatchQuery(inputText, "questions","answer^2","paths")
                    .setSize(BUFFER_SIZE)
                    .setFrom(i * BUFFER_SIZE)
                    .execute()
                    .actionGet();