Stack Overflow
Stack Overflow

Questions tagged [sharding]

Sharding is a technique of partitioning database tables by row ("horizontally"); typically this technique requires a key to be selected that determines how the rows are to be partitioned.

Sharding is a concept in database design; it refers to the technique of physically partitioning a table or collection by row (also known as horizontal partitioning). To execute the partition, a key or key collection must be defined, which tells the database engine how to determine to which partition each record should belong.

References

1666 questions
15
votes
3 answers

How to view sharding keys in mongodb shell?

This might be a dummy question, but I cannot find any clue in all online doc. For a already-built-up mongodb cluster, How can I find which sharding key(s) is used for given collecton?
Morgan Cheng
  • 73,950
  • 66
  • 171
  • 230
15
votes
2 answers

How to use database sharding with EF Core and C#"

I'm currently in the process of converting my 6 years old C# application to .NET Core v3 and EF Core (and also using Blazor). Most of it is working except for the Sharding part. Our application creates a new database for each client. We use more or…
Paul Meems
  • 3,002
  • 4
  • 35
  • 66
15
votes
1 answer

Android weird test sharding

I'm experimenting with test sharding on Android and I'm getting pretty weird results: + adb -s emulator-5580 shell am instrument -e numShards 2 -e shardIndex 0 -e class com.package.etc.automation.Tests.SanityTest.SanityTest -w…
Łukasz Sromek
  • 3,637
  • 3
  • 30
  • 43
14
votes
4 answers

MongoDB: Sharding on single machine. Does it make sense?

created a collection in MongoDB consisting of 11446615 documents. Each document has the following form: { "_id" : ObjectId("4e03dec7c3c365f574820835"), "httpReferer" : "http://www.somewebsite.pl/art.php?id=13321&b=1", "words" : ["SEX",…
whysoserious
  • 728
  • 1
  • 6
  • 20
14
votes
1 answer

How does sharding handle the joining of related tables?

When I read about sharding, looks like authors don't take into account other tables the sharded table has to be joined to (even though they describe a shard as a "subset of an original database"). However, this is a very common situation and I still…
raiks
  • 1,270
  • 1
  • 15
  • 12
14
votes
3 answers

Mysql 5.5 Table partition user and friends

I have two tables in my db that have millions of rows now, the selection and insertion is getting slower and slower. I am using spring+hibernate+mysql 5.5 and read about the sharding as well as partitioning the table and like the idea of…
maaz
  • 4,371
  • 2
  • 30
  • 48
13
votes
3 answers

Querying sharded data in MySQL

I'm dealing with a lot of data in a MySQL database and I'd like to use sharding to scale out. I understand the principles of sharding, and I even know how I want to shard my data. When I look up database sharding, I cannot find any comprehensive…
Tucker
  • 7,017
  • 9
  • 37
  • 55
13
votes
1 answer

purpose of Azure iot hub device-to-cloud partitions

When creating a new Azure IOT Hub you are asked how many device-to-cloud partitions you need. You can select between 2-32 partitions for standard tiers. I understand that the SKU and number of units determine the maximum daily quota of…
GHariz
  • 336
  • 2
  • 11
13
votes
2 answers

Database sharding and JPA

I am working on a Java application that requires horizontal partitioning of data in different PostgreSQL servers. I would like to use a JPA framework and Spring for transaction management. The most popular frameworks for sharding data with JPA seem…
is-serp
  • 171
  • 1
  • 9
13
votes
3 answers

What would be the right steps for horizontal partitioning in Postgresql?

We have an E-commerce portal with a Postgresql 9.1 database. One very important table has at the moment 32 million records. If we want to deliver all items this table would grow to 320 million records, mostly dates. Which would be to heavy. So we…
Brambo76
  • 191
  • 2
  • 8
12
votes
9 answers

Database sharding and Rails

What's the best way to deal with a sharded database in Rails? Should the sharding be handled at the application layer, the active record layer, the database driver layer, a proxy layer, or something else altogether? What are the pros and cons of…
Teflon Ted
  • 8,696
  • 19
  • 64
  • 78
12
votes
2 answers

What to do instead of SQL joins while scaling horizontally?

What would I use instead of SQL joins when I have a large complex relational-database that just got too large to fit on a single machine? I've begun sharding the database across many machines, but as a result, I can no longer do joins…
David Xu
  • 5,555
  • 3
  • 28
  • 50
12
votes
5 answers

Difference between Sharding And Replication on MongoDB

I am just confuse about the Sharding and Replication that how they works..According to Definition Replication: A replica set in MongoDB is a group of mongod processes that maintain the same data set. Sharding: Sharding is a method for storing data…
Saad Saadi
  • 1,031
  • 10
  • 26
11
votes
3 answers

When people talk about scaling a website with 'shards', what do they mean?

I have heard the 'shard' technique mentioned several times with regard to solving scaling problems for large websites. What is this 'shard' technique and why is it so good?
Phil Wright
  • 22,580
  • 14
  • 83
  • 137
11
votes
1 answer

How to do rails db:migrate on multiple shards that are not master slave relationship at once on rails?

I have an app which which uses different database based on the subdomain. So essentially, the schema would be the same, but the data would differ for each databases. But when I release some new features and it would require some schema changes, I…
Suthan Bala
  • 3,209
  • 5
  • 34
  • 59
1 2
3
…
99 100