Stack Overflow
Stack Overflow

Questions tagged [replication-factor]

22 questions
7
votes
3 answers

How to achieve strong consistency in Kafka?

Try to understanding consistency maintenance in Kafka. Please find the scenario and help to understand. Number of partition = 2 Replication factor = 3 Number of broker in the cluster = 4 In that case, for achieving the strong…
Karthikeyan Rasipalay Durairaj
  • 1,920
  • 13
  • 35
4
votes
1 answer

Kafka replication factor vs min.insync.replicas

Replication-factor is the total number of copies of the data stored in an Apache Kafka cluster. min.insync.replicas is the minimum number of copies of the data that you are willing to have online at any time to continue running and accepting new…
Dan
  • 651
  • 1
  • 14
  • 31
4
votes
1 answer

use of replication_factor in cassandra

I have this code: CREATE KEYSPACE “KeySpace Name” WITH replication = {'class': ‘Strategy name’, 'replication_factor' : ‘No.Of replicas’} What is the use of 'replication_factor' in the above Cassandra query?
Rinsha Rinz
  • 53
  • 10
4
votes
2 answers

Aerospike cluster with different replication factors

I have a cluster with replication factor of 6 (and 6 nodes). I want to change the replication factor to 3 without downtime. What happens if I change the RF node by node and restart each node? Is it possible to work?
Daniel Cukier
  • 11,502
  • 15
  • 68
  • 123
2
votes
1 answer

Warnings : Your replication factor 3 for keyspace books is higher than the number of nodes 1

When I tried: CREATE KEYSPACE IF NOT EXISTS books WITH REPLICATION = { 'class': 'SimpleStrategy', 'replication_factor': 3 }; I got: Warnings : Your replication factor 3 for keyspace books is higher than the number of nodes 1 I have installed…
best_of_man
  • 643
  • 2
  • 15
2
votes
1 answer

Altering hdfs replication factor dynamically in spark

In a Spark (3.2.0) application, i need to change the replication factor for different files written to HDFS. For instance, i write some temporary files, and i want them to be written with a replication factor 1. Then, i write some files that are…
belce
  • 203
  • 2
  • 5
2
votes
2 answers

The replication factor in Cassandra when creating a keyspace

When creating a new namespace in Cassandra, we need to give a number for a replication factor. Ex: Does the number, that we are giving as the replication factor, determine the number of nodes that initially create to store the replicate data? Can…
Sarah117
  • 91
  • 1
  • 7
2
votes
3 answers

Difference between consistency level and replication factor in cassandra?

I am new to cassandra and I wanted to understand the granular difference between consistency level and replication factor. Scenario: If I have a replication factor of 2 and consistency level of 3, how the write operation would be performed? When…
Vaibhav Gupta
  • 353
  • 1
  • 2
  • 17
2
votes
1 answer

hadoop distcp between clusters with different replication factors

As some background, we have 2 clusters which are currently used as production and development. As part of this, we are copying files (using hadoop distcp -update) from the production cluster to the development cluster after they have been produced…
Jo TR
  • 61
  • 6
1
vote
1 answer

Spring Cloud Stream Kafka Replication Factor and Retention Problem at Runtime

I have a spring boot project using Kafka. I configured it with Spring Cloud Stream Kafka auto configuration. I want to create my topics automatically with 3 replicas and 1 day retention. For this I added replication factor and retention.ms to my…
bugrayildirim
  • 13
  • 3
1
vote
0 answers

Spark replication factor for local filesystem

I am writing results of spark execution to local filesystem. Is there any way, I can provide a replication factor for the intermediary task results so that if a worker node goes down, there is no need to rerun the tasks already executed on that…
Shubham Jain
  • 392
  • 1
  • 3
  • 15
1
vote
1 answer

Replication factor in Cassandra

I am newbie to cassandra. What exactly replication factor in cassandra means? For example, I have 3 node cluster(node1,node2,node3) and If I create keyspace with replication factor 1,and insert data through node1,Can I read the data from other 2…
SujithTee
  • 125
  • 2
  • 3
  • 12
1
vote
2 answers

What to do with replication factor when reducing a Cassandra cluster?

I have a 3 nodes Cassandra cluster with 2 keyspaces. One of them has replication factor 1 and the other, replication factor 2. I want to reduce the cluster, using nodetool decommission, to remove 2 nodes and leave only one (single node cluster).…
Janbalik
  • 574
  • 6
  • 15
1
vote
1 answer

How long does it take to process the file If I have only one worker node?

Let's say I have a data with 25 blocks and the replication factor is 1. The mapper requires about 5 mins to read and process a single block of the data. Then how can I calculate the time for one worker node? The what about 15 nodes? Will the time be…
zeus0510
  • 13
  • 5
0
votes
1 answer

Keep getting the "INVALID_REPLICATION_FACTOR" errors for producer client

Context I am trying to mimic a distributed Kafka cluster via docker-compose projects, all are running on a single docker host. Cluster is having 2 brokers with a zookeeper, each running on its own separate docker-compose project. docker-compose…
melkorCba
  • 458
  • 5
  • 9
1
2