Questions tagged [sharding]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
81 views

MongoS refusing connections during network split

I am exploring sharded MongoDB clusters, and I have trouble understanding MongoS's behavior during a network split. My test environment works as expected until I simulate a network split. In the "...
Nick Turner's user avatar
0 votes
0 answers
33 views

Is it possible to create a big WindowsVPS from several DedicatedServer in openstack?

I need a WindowsVPS with a lot of hardware resources that are not available on a single DedicatedServer. I'm wondering if I can create a big WindowsVPS with three DedicatedServer resources using ...
pest's user avatar
  • 1
1 vote
1 answer
284 views

CouchDB deleted database shards

Currently running clustered CouchDB v3.1.1 on an AWS EC2 Instance q=2 n=3 Number of Nodes: 1 I have moved the database directory to another drive on my VM. But can't find any on the documents ...
Eliii's user avatar
  • 11
2 votes
0 answers
94 views

mongodb, remove one of the replica set's from a sharded cluster

I need to remove filed server01 (currently offline for a long time). The configuration looks like this: shards: { "_id" : "mongo1", "host" : "mongo1/server01:27017,...
iNio's user avatar
  • 55
0 votes
1 answer
2k views

Shards in Kubernetes

We have an application that we are running in kubernetes. However, this application only support a fixed number of people. It has multiple systems: DB Authentication system Web Application Media ...
AbrahamSustaita's user avatar
1 vote
0 answers
987 views

ElasticSearch not starting properly

On startup, our ElasticSearch server throws aJava exception I can see in the elasticsearch.log: [2019-02-27T15:51:59,908][WARN ][o.e.m.j.JvmGcMonitorService] [KibjtLy] [gc][young][263][19] duration [...
Alexis Dufrenoy's user avatar
3 votes
2 answers
262 views

Nosql autoscaling of compute

Most of the nosql autoscaling faces issue due to the fact the data have to be migrated during peak load. What if data is stored in a shared storage like CLVM which has less overhead(compared to NFS or ...
kalyan's user avatar
  • 249
0 votes
0 answers
66 views

Different sharding chunks size in MongoDB

I have a problem in a MongoDB Sharding deployment. I've got 4 replicasets, all of them with a very similar chunks. But the estimated data per chunk is very different in all of them. mongos> db....
RbT's user avatar
  • 1
0 votes
0 answers
75 views

MongoDB sharding replicaset overload when I stop the secondary

I've got a MongoDB Sharding with 8 servers, and 4 replicasets. Each replicaset has primary, secondary and arbiter. The distribution is something like this: - RS01 -> Server1A (primary), Server1B (...
RbT's user avatar
  • 1
1 vote
1 answer
24 views

On-prem DBs sync’d across multiple global sites?

My organization has two North American offices that are close enough to share a single web app server and database with minimal latency. However, we’re opening an office in India and that amount of ...
Scott's user avatar
  • 111
1 vote
1 answer
224 views

Advice on sharding the Parse Server _User collection?

The out-of-the-box _User collection in Mongo for Parse Server has a unique index on both username and email. Given that, what's the ideal strategy for sharding this collection? My initial plan was ...
andrew.w.lane's user avatar
0 votes
0 answers
276 views

Unable to add a mongod server in local network as shard

I'm trying to build a small cluster using mongodb sharding. In node1 and node2, mongod is started. In node1, config server and mongos are started. All listening 0.0.0.0 with exclusively allocated ...
Kun Ren's user avatar
  • 101
0 votes
1 answer
149 views

Mongodb sharded cluster architecture

I have a mongodb cluster on aws. In order to reduce network latency I want to have multiple shards in different regions. In each region there will be the application nodes that connect to the ...
Ciobanu Roxana's user avatar
4 votes
1 answer
3k views

MongoDB sharding: mongos and config servers together?

We want to create a MongoDB shard (v. 2.4). The official documentation recommends to have 3 config servers. However, the policies of our company won't allow us to get 3 extra servers for this purpose....
Pablo Torrecilla's user avatar
0 votes
1 answer
3k views

mongodb sharding - launch mongos with mongos.conf

We are currently setting up a sharded cluster with MongoDB, config servers have been set up using a configuration file : mongod.conf pointing out alternative paths for data and logs folders : ...
Irrational's user avatar
0 votes
1 answer
5k views

Authentication in a Mongo Sharded Cluster

I have a mongo shared cluster and replica sets: The ShardA Replica set: A1. 10.77.14.241 A2. 10.77.14.242 AA. 10.77.14.243 Arbiter Some parameters of /etc/mongod.conf: #bind_ip=127.0.0.1 auth=...
Dimaf's user avatar
  • 181
3 votes
1 answer
3k views

When and how should we shard MongoDB when we are bound to physical machines?

We maintain a search service that serves data from MongoDB. Our Mongo production instance is arranged in a 4 node replica set across four physical servers. The database is comprised of several small ...
Chris M's user avatar
  • 31
1 vote
2 answers
338 views

Is it possible to shard postfix virtual mailboxes database?

I've managed to configure Postfix to read virtual mailboxes list out of a database. But what if my list is distributed among multi databases? Is there any way to set Postfix's database access ...
Mehran's user avatar
  • 519
1 vote
1 answer
387 views

mongo shard has high number of faults, but top shows only 8% of memory used by mongod

I am investigating some performance issues in my mongo setup. I have 3 query controllers and 6 shards. I'm doing a large batch import/update, and on the 3 query controllers, I'm getting around 200 ...
user1130176's user avatar
1 vote
0 answers
103 views

Merge 7 node mongodb (2.4) cluster into one standalone mongo shard

I'm running a 7 node mongodb sharded cluster (version 2.4...). I would like to merge this into a standalone shard with 7 times the disk-space. I've tried to create the single shard instance and ...
Lasse Aagren's user avatar
0 votes
0 answers
434 views

Replicated MongoDB server slower than simple shards

I tried to compare the performance of a sharded configuration against a sharded and replicated configuration. The sharded configuration consists of 8 shards each running on three different machines ...
displayName's user avatar
0 votes
1 answer
3k views

How to recover a degraded MongoDB sharded cluster?

I've had problems with a MongoDB replica set of my sharded cluster. The current status of the nodes is: node1: primary (but apparently vetoed by node3 that needs to sync) node2: rollback (it was the ...
Icu's user avatar
  • 1,425
0 votes
2 answers
469 views

Restart tomcat on two different servers at same time

We are running the Apache solr on two servers. Unfortunately on heavy load of indexing (guess) servers going down. We are manually restarting the tomcat each time. When I google I find the way two ...
Gowri's user avatar
  • 113
1 vote
2 answers
4k views

Rename Mongo Shard

Can I, and if I can, how can I rename a shard in Mongo? Like if I wanted to change the instances of rs0 to rep0 below: mongos> sh.status() --- Sharding Status --- sharding version: { "_id" ...
HeySteve's user avatar
0 votes
1 answer
128 views

MongoDB sharding authentication with multiple keyFile's

I have configured MongoDB with sharding something like below. Shard1 Shard2 Shard3 -------- -------- ------- rs1(Pri) rs1(Sec) rs1(Sec) rs2(Sec) rs2(Pri) rs2(Sec) rs3(Sec) ...
vareda's user avatar
  • 70
6 votes
2 answers
4k views

Mongo Sharding - Config Server and Mongo Authentication

I am needing to shard a database fairly soon, and am unclear on what the best practice is for enabling authentication on the mongos and config servers. I would like to have everything be secured with ...
nakkor's user avatar
  • 163
1 vote
1 answer
173 views

MongoDB Balancer

So I have 2 sharded clusters with 3 servers each.My mongos is running, my config server is running and ive also added test data to 1 of the 2 shard clusters. I am having trouble getting that data to ...
secure212's user avatar
  • 228
0 votes
1 answer
513 views

How would I template an SLS using saltstack

I'm trying to do proof of concept with Mongodb(sharding) and Id like to run a command every time I spin up a new cluster without having to add lines in all my sls files. My current init is as follows: ...
secure212's user avatar
  • 228
1 vote
1 answer
1k views

Why mongo pecl module looses its connection after Primary stepDown?

I run an Apache 2.2.22-13 with pecl module "mongo 1.4.4". It connects to a MongoDB 2.4.6 sharded cluster. When a stepDown occurs, the mongo module seems to stay on the mongod node which was the ...
Icu's user avatar
  • 1,425
0 votes
1 answer
431 views

What's special in restoring a DB in a MongoDB sharded cluster?

I'd like to migrate a DB from a replica set to another which belongs to a sharded cluster. Till now, this cluster has got only 1 shard, and no collection or database is really sharded. I mongodumped ...
Icu's user avatar
  • 1,425
2 votes
1 answer
162 views

Add new shard - always best? [closed]

In our setup, we currently have 3 shard set sharded cluster, each shard being a replica set of 3. Our writes are about to go up significantly to implement a new feature, and we know the extra data ...
tonyl7126's user avatar
  • 213
-3 votes
1 answer
191 views

Mysql Cluster Sharding/partitioning [closed]

Ok so i have mysql cluster setup but what i need is say table assets to be split among the nodes, not replicated. as the purpose of me setting up the cluster was due to lack of hard drive space so how ...
Kristopher Therrien's user avatar
0 votes
2 answers
2k views

remove a mongodb replica set when into a sharding cluster

I have this configuration: mongos> db.adminCommand('getShardMap') { "map" : { "RS13" : "RS13/mongo09-02:27003,mongo09-01:27001", "RS13/mongo09-02:27003,mongo09-01:27001" : "RS13/mongo09-02:...
freedev's user avatar
  • 323
0 votes
1 answer
170 views

SQl Server sharding

Can SQL Server be sharded for distributed data processing? Or are there any limitations that prevent relational databases from being sharded ?
Maximusawesome's user avatar
2 votes
1 answer
935 views

Mongodb Slave replication lag

We using standard mongo setup: 2 replicas + 1 arbiter. Both replica servers use same AWS m1.medium with RAID10 EBS. We experiencing constantly growing replication lag on secondary replica. I tried ...
Leonid Bugaev's user avatar
2 votes
4 answers
3k views

Find busiest databases and tables on MySQL server

I have a busy MyMQL server with quite a few databases. I'm trying to figure out the best way to find the busiest databases (and tables) on the server so I can migrate them to a separate machine. ...
erikcw's user avatar
  • 697
4 votes
1 answer
2k views

What's the advantage of splitting a mysql table into smaller ones without physical distribution?

I've seen such design in my company. A big mysql table is split into tens of smaller ones of identical schema on the same server in the same db, without any physical distribution, no ...
Arrix's user avatar
  • 245
5 votes
1 answer
427 views

Upgrading a mongo cluster

I am looking at adopting a Mongo cluster and am doing research. We hope to have a cluster that can scale from 10k updates per second to 100k updates per second with very write heavy traffic. The ...
EnabrenTane's user avatar
9 votes
3 answers
14k views

Run a MongoDB configuration server without 3GB of journal files

For a production sharded MongoDB installation we need 3 configuration servers. According to the documentation "the config server mongod process is fairly lightweight and can be ran on machines ...
Thilo's user avatar
  • 240
1 vote
1 answer
413 views

ASP.NET, reaching the limits of DFSR, and file storage sharding

We run a popular web app, and our backend file storage exists on a file server, replicated to another file server for failover with DFSR. We are reaching the theoretical limits of DFSR, and therefore ...
Ken Randall's user avatar
2 votes
1 answer
782 views

Mongos not using secondary for read in replica/sharding mongo setup

I have a Mongo DB setup of 4 DB (2 shards of 2 replica) and I noticed that the secondary servers of each shard is not being used for read queries. Both secondary servers received all inserts, updates ...
Pierre's user avatar
  • 241
1 vote
1 answer
2k views

How to "unshard" an existant collection in mongo?

While setting up some collections in a mongo database to be sharded, I entered the name of one of them wrong (typo letter at the end). When I do db.printShardingStatus(), I see that inexistent ...
Pierre's user avatar
  • 241
3 votes
1 answer
301 views

What is the safest way to change MongoDB server structure?

I currently have a MongoDB replica set configured as such: { "_id" : "ahspy_mongo_set", "version" : 13, "members" : [ { "_id" : 0, "...
Pierre's user avatar
  • 241
2 votes
3 answers
3k views

Is it OK to shard mongodb without replica set?

Lets say I have 4 database servers. I want to add them to the "shard" cluster. That's it. No replica set, no nothing. Just 4 servers. If I turn "journaling" on all of them, is it OK?
Alex's user avatar
  • 8,591
1 vote
1 answer
196 views

Multiple databases on Mysql crashes the mysql server often

We are running Mysql on CentOS 5.5. And are using the database sharding approach. As it is an erp app, there are about 300 tables for each new Application/company. At the moment we have around 600 ...
Software Mechanic's user avatar
1 vote
3 answers
268 views

How to automatically move specific MySQL tables from one machine to another?

I have a MySQL database with tables in the form of "shard_0", "shard_1", "shard_2", etc. I want to add another DB server and move the even-numbered shards ("shard_0", "shard_2", "shard_4", ...) to ...
Continuation's user avatar
  • 3,130
2 votes
1 answer
630 views

How to scale out by elvolving from database partitions to sharding?

Say I have a MySQL table: CREATE TABLE tweets ( tweet_id INT NOT NULL AUTO_INCREMENT, author_id INT NOT NULL, text CHAR(140) NOT NULL, PRIMARY KEY (tweet_id) ) PARTITION BY HASH(tweet_id) PARTITIONS ...
Continuation's user avatar
  • 3,130
0 votes
3 answers
390 views

Splitting a MySQL DB in two may ease server from "Too many connetions"? I don't think so

I was requested to split a MySQL in two, it's kind of a horizontal partition, in which some rows correspond to one site, and some other correspond to another site. But they want to split it in two DBs ...
Petruza's user avatar
  • 285
7 votes
4 answers
10k views

postgreSQL vs Cassandra vs MongoDB vs Voldemort?

Which database to decide upon? Any comparisions? Existing: postgresql Issues Not easily scalable horizontal. Needs sharding etc Clustering does not solve the data growth problem ...
Ram on Rails's user avatar
0 votes
2 answers
88 views

2 partitions in a sharded mysql database

the first one is running at 73% memory utilization. The second one runs at 16% utilization. how can i balance the 2 out?
user avatar