2022 Developer Survey is open! Take survey.

Questions tagged [sharding]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Many linked tables in a NoSQL? Tons of connections, but only 1 master?

My company uses a NoSQL database (Mongo) for their product. However, their product is incredibly slow - which might be due to either the efficiency of their code, or/and the design of their database. ...
user avatar
  • 101
1 vote
1 answer
85 views

CouchDB deleted database shards

Currently running clustered CouchDB v3.1.1 on an AWS EC2 Instance q=2 n=3 Number of Nodes: 1 I have moved the database directory to another drive on my VM. But can't find any on the documents ...
user avatar
  • 11
1 vote
0 answers
13 views

mongodb, remove one of the replica set's from a sharded cluster

I need to remove filed server01 (currently offline for a long time). The configuration looks like this: shards: { "_id" : "mongo1", "host" : "mongo1/server01:27017,...
user avatar
  • 45
0 votes
1 answer
808 views

Shards in Kubernetes

We have an application that we are running in kubernetes. However, this application only support a fixed number of people. It has multiple systems: DB Authentication system Web Application Media ...
user avatar
1 vote
0 answers
869 views

ElasticSearch not starting properly

On startup, our ElasticSearch server throws aJava exception I can see in the elasticsearch.log: [2019-02-27T15:51:59,908][WARN ][o.e.m.j.JvmGcMonitorService] [KibjtLy] [gc][young][263][19] duration [...
user avatar
3 votes
2 answers
251 views

Nosql autoscaling of compute

Most of the nosql autoscaling faces issue due to the fact the data have to be migrated during peak load. What if data is stored in a shared storage like CLVM which has less overhead(compared to NFS or ...
user avatar
  • 219
0 votes
0 answers
51 views

Different sharding chunks size in MongoDB

I have a problem in a MongoDB Sharding deployment. I've got 4 replicasets, all of them with a very similar chunks. But the estimated data per chunk is very different in all of them. mongos> db....
user avatar
  • 1
0 votes
0 answers
69 views

MongoDB sharding replicaset overload when I stop the secondary

I've got a MongoDB Sharding with 8 servers, and 4 replicasets. Each replicaset has primary, secondary and arbiter. The distribution is something like this: - RS01 -> Server1A (primary), Server1B (...
user avatar
  • 1
1 vote
1 answer
23 views

On-prem DBs sync’d across multiple global sites?

My organization has two North American offices that are close enough to share a single web app server and database with minimal latency. However, we’re opening an office in India and that amount of ...
user avatar
  • 111
1 vote
1 answer
212 views

Advice on sharding the Parse Server _User collection?

The out-of-the-box _User collection in Mongo for Parse Server has a unique index on both username and email. Given that, what's the ideal strategy for sharding this collection? My initial plan was ...
user avatar
0 votes
0 answers
268 views

Unable to add a mongod server in local network as shard

I'm trying to build a small cluster using mongodb sharding. In node1 and node2, mongod is started. In node1, config server and mongos are started. All listening 0.0.0.0 with exclusively allocated ...
user avatar
  • 101
0 votes
1 answer
129 views

Mongodb sharded cluster architecture

I have a mongodb cluster on aws. In order to reduce network latency I want to have multiple shards in different regions. In each region there will be the application nodes that connect to the ...
user avatar
4 votes
1 answer
3k views

MongoDB sharding: mongos and config servers together?

We want to create a MongoDB shard (v. 2.4). The official documentation recommends to have 3 config servers. However, the policies of our company won't allow us to get 3 extra servers for this purpose....
user avatar
0 votes
1 answer
2k views

mongodb sharding - launch mongos with mongos.conf

We are currently setting up a sharded cluster with MongoDB, config servers have been set up using a configuration file : mongod.conf pointing out alternative paths for data and logs folders : ...
user avatar
0 votes
1 answer
5k views

Authentication in a Mongo Sharded Cluster

I have a mongo shared cluster and replica sets: The ShardA Replica set: A1. 10.77.14.241 A2. 10.77.14.242 AA. 10.77.14.243 Arbiter Some parameters of /etc/mongod.conf: #bind_ip=127.0.0.1 auth=...
user avatar
  • 181
3 votes
1 answer
3k views

When and how should we shard MongoDB when we are bound to physical machines?

We maintain a search service that serves data from MongoDB. Our Mongo production instance is arranged in a 4 node replica set across four physical servers. The database is comprised of several small ...
user avatar
  • 31
1 vote
2 answers
267 views

Is it possible to shard postfix virtual mailboxes database?

I've managed to configure Postfix to read virtual mailboxes list out of a database. But what if my list is distributed among multi databases? Is there any way to set Postfix's database access ...
user avatar
  • 519
1 vote
1 answer
349 views

mongo shard has high number of faults, but top shows only 8% of memory used by mongod

I am investigating some performance issues in my mongo setup. I have 3 query controllers and 6 shards. I'm doing a large batch import/update, and on the 3 query controllers, I'm getting around 200 ...
user avatar
1 vote
0 answers
94 views

Merge 7 node mongodb (2.4) cluster into one standalone mongo shard

I'm running a 7 node mongodb sharded cluster (version 2.4...). I would like to merge this into a standalone shard with 7 times the disk-space. I've tried to create the single shard instance and ...
user avatar
0 votes
0 answers
428 views

Replicated MongoDB server slower than simple shards

I tried to compare the performance of a sharded configuration against a sharded and replicated configuration. The sharded configuration consists of 8 shards each running on three different machines ...
user avatar
0 votes
1 answer
2k views

How to recover a degraded MongoDB sharded cluster?

I've had problems with a MongoDB replica set of my sharded cluster. The current status of the nodes is: node1: primary (but apparently vetoed by node3 that needs to sync) node2: rollback (it was the ...
user avatar
  • 1,385
0 votes
2 answers
405 views

Restart tomcat on two different servers at same time

We are running the Apache solr on two servers. Unfortunately on heavy load of indexing (guess) servers going down. We are manually restarting the tomcat each time. When I google I find the way two ...
user avatar
  • 113
1 vote
2 answers
4k views

Rename Mongo Shard

Can I, and if I can, how can I rename a shard in Mongo? Like if I wanted to change the instances of rs0 to rep0 below: mongos> sh.status() --- Sharding Status --- sharding version: { "_id" ...
user avatar
0 votes
1 answer
123 views

MongoDB sharding authentication with multiple keyFile's

I have configured MongoDB with sharding something like below. Shard1 Shard2 Shard3 -------- -------- ------- rs1(Pri) rs1(Sec) rs1(Sec) rs2(Sec) rs2(Pri) rs2(Sec) rs3(Sec) ...
user avatar
  • 70
6 votes
2 answers
4k views

Mongo Sharding - Config Server and Mongo Authentication

I am needing to shard a database fairly soon, and am unclear on what the best practice is for enabling authentication on the mongos and config servers. I would like to have everything be secured with ...
user avatar
  • 163
1 vote
1 answer
168 views

MongoDB Balancer

So I have 2 sharded clusters with 3 servers each.My mongos is running, my config server is running and ive also added test data to 1 of the 2 shard clusters. I am having trouble getting that data to ...
user avatar
  • 228
0 votes
1 answer
495 views

How would I template an SLS using saltstack

I'm trying to do proof of concept with Mongodb(sharding) and Id like to run a command every time I spin up a new cluster without having to add lines in all my sls files. My current init is as follows: ...
user avatar
  • 228
1 vote
1 answer
1k views

Why mongo pecl module looses its connection after Primary stepDown?

I run an Apache 2.2.22-13 with pecl module "mongo 1.4.4". It connects to a MongoDB 2.4.6 sharded cluster. When a stepDown occurs, the mongo module seems to stay on the mongod node which was the ...
user avatar
  • 1,385
0 votes
1 answer
426 views

What's special in restoring a DB in a MongoDB sharded cluster?

I'd like to migrate a DB from a replica set to another which belongs to a sharded cluster. Till now, this cluster has got only 1 shard, and no collection or database is really sharded. I mongodumped ...
user avatar
  • 1,385
2 votes
1 answer
159 views

Add new shard - always best? [closed]

In our setup, we currently have 3 shard set sharded cluster, each shard being a replica set of 3. Our writes are about to go up significantly to implement a new feature, and we know the extra data ...
user avatar
  • 213
-3 votes
1 answer
183 views

Mysql Cluster Sharding/partitioning [closed]

Ok so i have mysql cluster setup but what i need is say table assets to be split among the nodes, not replicated. as the purpose of me setting up the cluster was due to lack of hard drive space so how ...
user avatar
0 votes
2 answers
2k views

remove a mongodb replica set when into a sharding cluster

I have this configuration: mongos> db.adminCommand('getShardMap') { "map" : { "RS13" : "RS13/mongo09-02:27003,mongo09-01:27001", "RS13/mongo09-02:27003,mongo09-01:27001" : "RS13/mongo09-02:...
user avatar
  • 313
0 votes
1 answer
167 views

SQl Server sharding

Can SQL Server be sharded for distributed data processing? Or are there any limitations that prevent relational databases from being sharded ?
user avatar
2 votes
1 answer
894 views

Mongodb Slave replication lag

We using standard mongo setup: 2 replicas + 1 arbiter. Both replica servers use same AWS m1.medium with RAID10 EBS. We experiencing constantly growing replication lag on secondary replica. I tried ...
user avatar
2 votes
4 answers
2k views

Find busiest databases and tables on MySQL server

I have a busy MyMQL server with quite a few databases. I'm trying to figure out the best way to find the busiest databases (and tables) on the server so I can migrate them to a separate machine. ...
user avatar
  • 677
4 votes
1 answer
2k views

What's the advantage of splitting a mysql table into smaller ones without physical distribution?

I've seen such design in my company. A big mysql table is split into tens of smaller ones of identical schema on the same server in the same db, without any physical distribution, no ...
user avatar
  • 225
5 votes
1 answer
389 views

Upgrading a mongo cluster

I am looking at adopting a Mongo cluster and am doing research. We hope to have a cluster that can scale from 10k updates per second to 100k updates per second with very write heavy traffic. The ...
user avatar
9 votes
3 answers
13k views

Run a MongoDB configuration server without 3GB of journal files

For a production sharded MongoDB installation we need 3 configuration servers. According to the documentation "the config server mongod process is fairly lightweight and can be ran on machines ...
user avatar
  • 240
1 vote
1 answer
402 views

ASP.NET, reaching the limits of DFSR, and file storage sharding

We run a popular web app, and our backend file storage exists on a file server, replicated to another file server for failover with DFSR. We are reaching the theoretical limits of DFSR, and therefore ...
user avatar
2 votes
1 answer
770 views

Mongos not using secondary for read in replica/sharding mongo setup

I have a Mongo DB setup of 4 DB (2 shards of 2 replica) and I noticed that the secondary servers of each shard is not being used for read queries. Both secondary servers received all inserts, updates ...
user avatar
  • 241
1 vote
1 answer
2k views

How to "unshard" an existant collection in mongo?

While setting up some collections in a mongo database to be sharded, I entered the name of one of them wrong (typo letter at the end). When I do db.printShardingStatus(), I see that inexistent ...
user avatar
  • 241
3 votes
1 answer
298 views

What is the safest way to change MongoDB server structure?

I currently have a MongoDB replica set configured as such: { "_id" : "ahspy_mongo_set", "version" : 13, "members" : [ { "_id" : 0, "...
user avatar
  • 241
3 votes
3 answers
3k views

Is it OK to shard mongodb without replica set?

Lets say I have 4 database servers. I want to add them to the "shard" cluster. That's it. No replica set, no nothing. Just 4 servers. If I turn "journaling" on all of them, is it OK?
user avatar
  • 8,031
1 vote
1 answer
161 views

Multiple databases on Mysql crashes the mysql server often

We are running Mysql on CentOS 5.5. And are using the database sharding approach. As it is an erp app, there are about 300 tables for each new Application/company. At the moment we have around 600 ...
user avatar
1 vote
3 answers
258 views

How to automatically move specific MySQL tables from one machine to another?

I have a MySQL database with tables in the form of "shard_0", "shard_1", "shard_2", etc. I want to add another DB server and move the even-numbered shards ("shard_0", "shard_2", "shard_4", ...) to ...
user avatar
  • 3,040
2 votes
1 answer
627 views

How to scale out by elvolving from database partitions to sharding?

Say I have a MySQL table: CREATE TABLE tweets ( tweet_id INT NOT NULL AUTO_INCREMENT, author_id INT NOT NULL, text CHAR(140) NOT NULL, PRIMARY KEY (tweet_id) ) PARTITION BY HASH(tweet_id) PARTITIONS ...
user avatar
  • 3,040
0 votes
3 answers
345 views

Splitting a MySQL DB in two may ease server from "Too many connetions"? I don't think so

I was requested to split a MySQL in two, it's kind of a horizontal partition, in which some rows correspond to one site, and some other correspond to another site. But they want to split it in two DBs ...
user avatar
  • 285
7 votes
4 answers
10k views

postgreSQL vs Cassandra vs MongoDB vs Voldemort?

Which database to decide upon? Any comparisions? Existing: postgresql Issues Not easily scalable horizontal. Needs sharding etc Clustering does not solve the data growth problem ...
user avatar
0 votes
2 answers
88 views

2 partitions in a sharded mysql database

the first one is running at 73% memory utilization. The second one runs at 16% utilization. how can i balance the 2 out?
user avatar
1 vote
5 answers
2k views

I have to shard a mysql database. I want to start with 12 shards on 2 machines. What is the best way to move some of them when I want to add a new server?

All tables are InnoDb. I would rather not use mysqldump, because the shard sizes will be about 200 GB (about 700 million rows), and that will take too long. I was hoping to just stop mysql for an ...
user avatar
  • 71