Questions tagged [sharding]
The sharding tag has no usage guidance.
51
questions
1
vote
0
answers
78
views
MongoS refusing connections during network split
I am exploring sharded MongoDB clusters, and I have trouble understanding MongoS's behavior during a network split.
My test environment works as expected until I simulate a network split.
In the "...
- 11
0
votes
0
answers
31
views
Is it possible to create a big WindowsVPS from several DedicatedServer in openstack?
I need a WindowsVPS with a lot of hardware resources that are not available on a single DedicatedServer.
I'm wondering if I can create a big WindowsVPS with three DedicatedServer resources using ...
- 1
1
vote
1
answer
250
views
CouchDB deleted database shards
Currently running clustered CouchDB v3.1.1 on an AWS EC2 Instance
q=2
n=3
Number of Nodes: 1
I have moved the database directory to another drive on my VM. But can't find any on the documents ...
- 11
2
votes
0
answers
68
views
mongodb, remove one of the replica set's from a sharded cluster
I need to remove filed server01 (currently offline for a long time).
The configuration looks like this:
shards:
{ "_id" : "mongo1", "host" : "mongo1/server01:27017,...
- 55
0
votes
1
answer
2k
views
Shards in Kubernetes
We have an application that we are running in kubernetes. However, this application only support a fixed number of people.
It has multiple systems:
DB
Authentication system
Web Application
Media ...
- 113
1
vote
0
answers
961
views
ElasticSearch not starting properly
On startup, our ElasticSearch server throws aJava exception I can see in the elasticsearch.log:
[2019-02-27T15:51:59,908][WARN ][o.e.m.j.JvmGcMonitorService] [KibjtLy] [gc][young][263][19] duration [...
- 235
3
votes
2
answers
261
views
Nosql autoscaling of compute
Most of the nosql autoscaling faces issue due to the fact the data have to be migrated during peak load. What if data is stored in a shared storage like CLVM which has less overhead(compared to NFS or ...
- 249
0
votes
0
answers
64
views
Different sharding chunks size in MongoDB
I have a problem in a MongoDB Sharding deployment. I've got 4 replicasets, all of them with a very similar chunks. But the estimated data per chunk is very different in all of them.
mongos> db....
- 1
0
votes
0
answers
73
views
MongoDB sharding replicaset overload when I stop the secondary
I've got a MongoDB Sharding with 8 servers, and 4 replicasets. Each replicaset has primary, secondary and arbiter.
The distribution is something like this:
- RS01 -> Server1A (primary), Server1B (...
- 1
1
vote
1
answer
24
views
On-prem DBs sync’d across multiple global sites?
My organization has two North American offices that are close enough to share a single web app server and database with minimal latency. However, we’re opening an office in India and that amount of ...
- 111
1
vote
1
answer
220
views
Advice on sharding the Parse Server _User collection?
The out-of-the-box _User collection in Mongo for Parse Server has a unique index on both username and email. Given that, what's the ideal strategy for sharding this collection? My initial plan was ...
0
votes
0
answers
274
views
Unable to add a mongod server in local network as shard
I'm trying to build a small cluster using mongodb sharding. In node1 and node2, mongod is started. In node1, config server and mongos are started. All listening 0.0.0.0 with exclusively allocated ...
- 101
0
votes
1
answer
144
views
Mongodb sharded cluster architecture
I have a mongodb cluster on aws. In order to reduce network latency I want to have multiple shards in different regions. In each region there will be the application nodes that connect to the ...
4
votes
1
answer
3k
views
MongoDB sharding: mongos and config servers together?
We want to create a MongoDB shard (v. 2.4). The official documentation recommends to have 3 config servers.
However, the policies of our company won't allow us to get 3 extra servers for this purpose....
0
votes
1
answer
3k
views
mongodb sharding - launch mongos with mongos.conf
We are currently setting up a sharded cluster with MongoDB, config servers have been set up using a configuration file : mongod.conf pointing out alternative paths for data and logs folders :
...
- 35
0
votes
1
answer
5k
views
Authentication in a Mongo Sharded Cluster
I have a mongo shared cluster and replica sets:
The ShardA Replica set:
A1. 10.77.14.241
A2. 10.77.14.242
AA. 10.77.14.243 Arbiter
Some parameters of /etc/mongod.conf:
#bind_ip=127.0.0.1
auth=...
- 181
3
votes
1
answer
3k
views
When and how should we shard MongoDB when we are bound to physical machines?
We maintain a search service that serves data from MongoDB. Our Mongo production instance is arranged in a 4 node replica set across four physical servers.
The database is comprised of several small ...
- 31
1
vote
2
answers
321
views
Is it possible to shard postfix virtual mailboxes database?
I've managed to configure Postfix to read virtual mailboxes list out of a database. But what if my list is distributed among multi databases? Is there any way to set Postfix's database access ...
- 519
1
vote
1
answer
377
views
mongo shard has high number of faults, but top shows only 8% of memory used by mongod
I am investigating some performance issues in my mongo setup. I have 3 query controllers and 6 shards. I'm doing a large batch import/update, and on the 3 query controllers, I'm getting around 200 ...
- 173
1
vote
0
answers
101
views
Merge 7 node mongodb (2.4) cluster into one standalone mongo shard
I'm running a 7 node mongodb sharded cluster (version 2.4...).
I would like to merge this into a standalone shard with 7 times the disk-space.
I've tried to create the single shard instance and ...
- 11
0
votes
0
answers
434
views
Replicated MongoDB server slower than simple shards
I tried to compare the performance of a sharded configuration against a sharded and replicated configuration.
The sharded configuration consists of 8 shards each running on three different machines ...
- 101
0
votes
1
answer
3k
views
How to recover a degraded MongoDB sharded cluster?
I've had problems with a MongoDB replica set of my sharded cluster. The current status of the nodes is:
node1: primary (but apparently vetoed by node3 that needs to sync)
node2: rollback (it was the ...
- 1,425
0
votes
2
answers
456
views
Restart tomcat on two different servers at same time
We are running the Apache solr on two servers. Unfortunately on heavy load of indexing (guess) servers going down. We are manually restarting the tomcat each time.
When I google I find the way two ...
- 113
1
vote
2
answers
4k
views
Rename Mongo Shard
Can I, and if I can, how can I rename a shard in Mongo?
Like if I wanted to change the instances of rs0 to rep0 below:
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" ...
- 23
0
votes
1
answer
127
views
MongoDB sharding authentication with multiple keyFile's
I have configured MongoDB with sharding something like below.
Shard1 Shard2 Shard3
-------- -------- -------
rs1(Pri) rs1(Sec) rs1(Sec)
rs2(Sec) rs2(Pri) rs2(Sec)
rs3(Sec) ...
- 70
6
votes
2
answers
4k
views
Mongo Sharding - Config Server and Mongo Authentication
I am needing to shard a database fairly soon, and am unclear on what the best practice is for enabling authentication on the mongos and config servers.
I would like to have everything be secured with ...
- 163
1
vote
1
answer
171
views
MongoDB Balancer
So I have 2 sharded clusters with 3 servers each.My mongos is running, my config server is running and ive also added test data to 1 of the 2 shard clusters. I am having trouble getting that data to ...
- 228
0
votes
1
answer
506
views
How would I template an SLS using saltstack
I'm trying to do proof of concept with Mongodb(sharding) and Id like to run a command every time I spin up a new cluster without having to add lines in all my sls files. My current init is as follows:
...
- 228
1
vote
1
answer
1k
views
Why mongo pecl module looses its connection after Primary stepDown?
I run an Apache 2.2.22-13 with pecl module "mongo 1.4.4". It connects to a MongoDB 2.4.6 sharded cluster.
When a stepDown occurs, the mongo module seems to stay on the mongod node which was the ...
- 1,425
0
votes
1
answer
429
views
What's special in restoring a DB in a MongoDB sharded cluster?
I'd like to migrate a DB from a replica set to another which belongs to a sharded cluster. Till now, this cluster has got only 1 shard, and no collection or database is really sharded.
I mongodumped ...
- 1,425
2
votes
1
answer
162
views
Add new shard - always best? [closed]
In our setup, we currently have 3 shard set sharded cluster, each shard being a replica set of 3. Our writes are about to go up significantly to implement a new feature, and we know the extra data ...
- 213
-3
votes
1
answer
191
views
Mysql Cluster Sharding/partitioning [closed]
Ok so i have mysql cluster setup but what i need is say table assets to be split among the nodes, not replicated. as the purpose of me setting up the cluster was due to lack of hard drive space so how ...
0
votes
2
answers
2k
views
remove a mongodb replica set when into a sharding cluster
I have this configuration:
mongos> db.adminCommand('getShardMap')
{
"map" : {
"RS13" : "RS13/mongo09-02:27003,mongo09-01:27001",
"RS13/mongo09-02:27003,mongo09-01:27001" : "RS13/mongo09-02:...
- 323
0
votes
1
answer
169
views
SQl Server sharding
Can SQL Server be sharded for distributed data processing? Or are there any limitations that prevent relational databases from being sharded ?
- 111
2
votes
1
answer
929
views
Mongodb Slave replication lag
We using standard mongo setup: 2 replicas + 1 arbiter.
Both replica servers use same AWS m1.medium with RAID10 EBS.
We experiencing constantly growing replication lag on secondary replica.
I tried ...
- 143
2
votes
4
answers
3k
views
Find busiest databases and tables on MySQL server
I have a busy MyMQL server with quite a few databases. I'm trying to figure out the best way to find the busiest databases (and tables) on the server so I can migrate them to a separate machine. ...
- 697
4
votes
1
answer
2k
views
What's the advantage of splitting a mysql table into smaller ones without physical distribution?
I've seen such design in my company.
A big mysql table is split into tens of smaller ones of identical schema on the same server in the same db, without any physical distribution, no ...
- 235
5
votes
1
answer
415
views
Upgrading a mongo cluster
I am looking at adopting a Mongo cluster and am doing research. We hope to have a cluster that can scale from 10k updates per second to 100k updates per second with very write heavy traffic.
The ...
- 255
9
votes
3
answers
14k
views
Run a MongoDB configuration server without 3GB of journal files
For a production sharded MongoDB installation we need 3 configuration servers.
According to the documentation "the config server mongod process is fairly lightweight and can be ran on machines ...
- 240
1
vote
1
answer
409
views
ASP.NET, reaching the limits of DFSR, and file storage sharding
We run a popular web app, and our backend file storage exists on a file server, replicated to another file server for failover with DFSR. We are reaching the theoretical limits of DFSR, and therefore ...
- 143
2
votes
1
answer
781
views
Mongos not using secondary for read in replica/sharding mongo setup
I have a Mongo DB setup of 4 DB (2 shards of 2 replica) and I noticed that the secondary servers of each shard is not being used for read queries. Both secondary servers received all inserts, updates ...
- 241
1
vote
1
answer
2k
views
How to "unshard" an existant collection in mongo?
While setting up some collections in a mongo database to be sharded, I entered the name of one of them wrong (typo letter at the end).
When I do db.printShardingStatus(), I see that inexistent ...
- 241
3
votes
1
answer
299
views
What is the safest way to change MongoDB server structure?
I currently have a MongoDB replica set configured as such:
{
"_id" : "ahspy_mongo_set",
"version" : 13,
"members" : [
{
"_id" : 0,
"...
- 241
2
votes
3
answers
3k
views
Is it OK to shard mongodb without replica set?
Lets say I have 4 database servers. I want to add them to the "shard" cluster.
That's it. No replica set, no nothing. Just 4 servers.
If I turn "journaling" on all of them, is it OK?
- 8,481
1
vote
1
answer
185
views
Multiple databases on Mysql crashes the mysql server often
We are running Mysql on CentOS 5.5. And are using the database sharding approach. As it is an erp app, there are about 300 tables for each new Application/company. At the moment we have around 600 ...
1
vote
3
answers
267
views
How to automatically move specific MySQL tables from one machine to another?
I have a MySQL database with tables in the form of "shard_0", "shard_1", "shard_2", etc.
I want to add another DB server and move the even-numbered shards ("shard_0", "shard_2", "shard_4", ...) to ...
- 3,080
2
votes
1
answer
630
views
How to scale out by elvolving from database partitions to sharding?
Say I have a MySQL table:
CREATE TABLE tweets (
tweet_id INT NOT NULL AUTO_INCREMENT,
author_id INT NOT NULL,
text CHAR(140) NOT NULL,
PRIMARY KEY (tweet_id)
)
PARTITION BY HASH(tweet_id)
PARTITIONS ...
- 3,080
0
votes
3
answers
376
views
Splitting a MySQL DB in two may ease server from "Too many connetions"? I don't think so
I was requested to split a MySQL in two, it's kind of a horizontal partition, in which some rows correspond to one site, and some other correspond to another site.
But they want to split it in two DBs ...
- 285
7
votes
4
answers
10k
views
postgreSQL vs Cassandra vs MongoDB vs Voldemort?
Which database to decide upon? Any comparisions?
Existing: postgresql
Issues
Not easily scalable horizontal. Needs sharding etc
Clustering does not solve the data growth problem
...
- 209
0
votes
2
answers
88
views
2 partitions in a sharded mysql database
the first one is running at 73% memory utilization. The second one runs at 16% utilization.
how can i balance the 2 out?
ming yeow