System that handles large amounts of data in an organised fashion

learn more… | top users | synonyms (2)

17
votes
1answer
2k views

How do you do load testing and capacity planning for databases?

This is a canonical question about capacity planning for databases. Related: Can you help me with my capacity planning? How do you do Load Testing and Capacity Planning for Web ...
11
votes
2answers
8k views

Third-party SSD solutions in ProLiant Gen8 servers

I was wondering if anyone had any specific experience using Intel DC3700 SSDs (or similar) in the HP (DL380p) Gen8 servers? I'm upgrading a set of database servers that use direct-attached storage. ...
37
votes
7answers
30k views

What is the best Linux filesystem for MySQL (InnoDB)?

I tried to look for benchmark on the performances of various filesystems with MySQL InnoDB but couldn't find any. My database workload is the typical web-based OLTP, about 90% read, 10% write. Random ...
1
vote
1answer
1k views

How do I set my.cnf in Mysql so that there are no limits to connections?

Unlimited connections, how do I edit that setting?
10
votes
3answers
504 views

What types of systems have to “scale up” rather than “scale out”?

I have been wondering for a long time if there are systems that have to "scale up" (onto a more powerful, more expensive server) rather than "scale out" by being split across many smaller servers. ...
7
votes
9answers
2k views

How do you usually back up your database?

I am using mysql How often do you back up your database? How do you normally backup your database? Export all data into sql or cvs format and keep it in a folder??
16
votes
4answers
20k views

How do I load a sql.gz file to my database? (importing)

is this right? mysql -uroot -ppassword mydb > myfile.sql.gz
10
votes
4answers
3k views

Should I backup and restore the `mysql` database?

In the process of creating an automated solution for backing up and restoring an entire MySQL server, I've come across the mysql database which seems to contain user accounts, permissions, metadata, ...
20
votes
4answers
2k views

Can MySQL effectively take advantage of 64 GB RAM?

We have been running into a problem where querying a table that has approximately 50 million rows, and has an index size of 4 GB (table size of about 6 GB) results in the database server swapping ...
4
votes
12answers
17k views

SQL Server database on an external hard disk drive

Due to some security problems, My boss has asked me to store all sensitive data in external/removable storages like USB stick or external HDD and this specially includes the MDF/NDF/LDF files of SQL ...
1
vote
6answers
2k views

MySQL replication out of sync

I have a master-master replication system. However, due to an auto-increment issue, I got an error in replication...and it stopped replicating. Someone told me to do: stop slave; SET GLOBAL ...
-1
votes
6answers
247 views

backing up large database with large amount of data in a small time limit then restoring it within a small time of limit? [closed]

Backing up large database with large amount of data in a small time limit then restoring it within a small time of limit? Any useful ideas?
2
votes
1answer
178 views

Microsoft SQL alternative for Oracle's Imp and Exp command

I'm looking a command line tool or SQL command to export/import a table or complete database. Much like the imp and exp commands of Oracle databases. Any suggestions?
1
vote
1answer
4k views

how to connect to mongodb server via ssh tunnel

It was easy for me to connect to my remote mysql server on AWS using a sequelpro, however I'm struggling with doing the same thing with mongodb. I tried setting up an ssh tunnel via command line ...
1
vote
2answers
252 views

Database Connection Error

Recently this has been happening every so often...say once a day for about 10 minutes. All the pages on my website load up as a blank page with big text DATABASE CONNECTION ERROR. I try to login to ...
1
vote
1answer
1k views

Deleting huge chunks of data from mysql innodb

I need to delete a huge chunk of my data in my production database, which runs about 100GB in size. If possible, i would like to minimize my downtime. My selection criteria for deleting is likely to ...
0
votes
1answer
248 views

Binding MySQL to run from the public or private LAN IP address - which one is faster

So we have 2 servers all running at the same web host. We have bind MySQL to listen on the public ip-address of the database server and the web server connects to it from the public ip. Both servers ...
0
votes
2answers
365 views

Windows Server 2008 Web vs Standard

Please tell me what choice will be best between Windows Server 2008 R2 Web edition and Windows Server 2008 R2 Standard edition for NoSQLDatabase/BackendComputing scaling farm? I am pretty sure that ...
0
votes
4answers
63 views

back-up database in Linux partition from windows partition!

Hello I am in a very embarrassing situation, I was trying to upgrade to ubuntu 10.04 from ubuntu 9.10 yesterday and while updating I forgot to put my laptop on charger and the battery ran out. Since ...
0
votes
2answers
575 views

How do I sync a subset of tables between two databases on the same mysql database server

would like to be able to sync a subset of tables between two mysql databases that are running on the same server. One of the databases acts as the master where inserts, updates and deletes can be ...
30
votes
7answers
18k views

Is it safe to have SQL Server auto-shrink turned on?

There are many SQL Server options that can be enabled for databases, and one of the most misunderstood ones is auto-shrink. Is it safe? If not, why not?
12
votes
2answers
14k views

Best Mysql cache settings for 8gb RAM dedicated mysql server using only Innodb (5gb database)

I'm a pretty big noob when it comes to setting up mysql for performance. And honestly I'm not worried about the fine tuning to squeeze every last bit of performance out of mysql, but I do know that ...
9
votes
3answers
9k views

How does one list warnings from the 'mysqlimport' utility?

To start off, this is not about loading data from within mysql itself, but using the command-line tool "mysqlimport". I am using it to load a CSV directly into a table and need to see the warnings ...
4
votes
8answers
3k views

MS SQL Layout for best performance

We have purchased a new server to act as a MS SQL backend. I am curious to know what the best setup would be for it. The server is a Dell R710 it has 6 hard drives 2x 74GB 15k and 4x 146GB 15k ...
7
votes
3answers
2k views

OpenSSH with public keys from database

Is it possible to fetch the public keys from a database instead of the authorized_keys file? I would like to use such a setup to manage ssh access to things like git repositories for multiple users ...
5
votes
5answers
2k views

Best way to make sure a MySQL database is fully in UTF8

After some problems with UTF8 and none-UTF8 strings, we're standardising on UTF8. One thing I need to do is check that everything is in UTF8 in the MySQL database? What do I need to check? Server ...
1
vote
2answers
1k views

Debugging mysql too many connections problem

I get a "too many connections" error from time to time with my mysql database, if I do a show status like 'Conn%'; I get a result like this +---------------+-------+ | Variable_name | Value | ...
22
votes
6answers
3k views

Scaling databases with cheap SSD hard drives

I hope that many of you are working with high traffic database-driven websites, and chances are that your main scalability issues are in the database. I noticed a couple of things lately: Most large ...
5
votes
5answers
2k views

What is “The” book for database design? [closed]

In programming, there is often a canonical book for a particular topic, like the dragon book for compilers, K&R for C, etc. Is their a book regarding modern database design that simply must be ...
4
votes
7answers
2k views

MySQL database backup over network

Server #1 is a MySQL database server running on Debian which contains numerous tables and one particular table that is over 100GB. Server #2 is used as a MySQL slave for replication, but now it needs ...
3
votes
5answers
1k views

mysqldump for sql server

I have used mysqldump on a mysql database lots of times and am familiar with how it works. Does anyone know if there is a similar tool that could be used on a sql server database. I know sql server ...
2
votes
5answers
20k views

How can I synchronize MS SQL Server databases?

How can I synchronize MS SQL Server databases? Is there any tool, preferably open source?
2
votes
4answers
10k views

How to copy & move MySQL database

I'm very new to servers and databases, my experience spanning only a few installations of XAMPP for easy website testing and exploration. So I have XAMPP installed on my laptop, and I just put it on ...
2
votes
5answers
436 views

Mysql high availability

I have a application that receives data from approximately 1000 different sources every couple of minutes. This data needs to be saved into a OSS database (most likely MySQL - but depending on answers ...
6
votes
1answer
1k views

Does anyone have any experience when it comes to speed of Amazon RDS vs MySQL on a Small server?

EC2 Small vs Amazon RDS - which is faster for MySQL? (I have one database, well-indexed and tuned. About 10,000 rows on each table.) How's the performance like?
6
votes
5answers
534 views

Virtualizing Microsoft SQL Database server

My company is looking to virtualize our Microsoft SQL server. I read on another question that databases on VM's suffer from a IO bottleneck which is not very good for us. But I was wondering if ...
6
votes
2answers
3k views

Is there a secure way to allow IIS 7 in a DMZ to access a DB server behind the firewall?

Our network admins are adamant that it is insecure for our web servers, which are hosted in the DMZ, to access the DB server behind our firewall. To get round the problem, we access the data via web ...
5
votes
4answers
5k views

MySQL, disk I/O and SSD drives

We have a LAMP box with 2x mirrored 1 TB WD Black Caviar disks running the whole OS and MySQL. 8 GB / RAM, 2x quad core CPUs. We're really taxed on disk I/O, and I've been thinking of ...
3
votes
5answers
665 views

Alternative to multiple WordPress installs (and not using WordPress-mu)

I run several instances of WordPress on a server for managing blogs across multiple domains (also hosted on the same server). Some domains are even utilizing multiple WordPress instances just as ...
3
votes
7answers
1k views

soft/fake raid in linux

original I don't have much experience with Linux, but this is a good oportunity to learn I'm mounting a simple database server and i would like to know if ubuntu server 9.10 (what do you guys ...
2
votes
3answers
1k views

Amazon EC2 + EBS:: Regular backup plan?

I have read some articles about amazon ec2 instances and ebs. I found out that, the data on the instance are not persistent, which means, if the hardware fails, instance shut down, I will lost my ...
2
votes
4answers
995 views

Mysql replications: slave is not readonly

After setting up mysql replications with master/slave db scheme, I noticed that the slave db server is not read-only. Of cause, after that I configured it manually in /etc/my.cnf file. I just can't ...
2
votes
1answer
194 views

What database is easy to maintain and manage in a cluster?

I'm looking for a database (DBMS) that is easy to scale out. I would like to have high availability so I need a multi-master cluster, where the data is replicated to two or more physical computers. I ...
1
vote
2answers
2k views

MySQL Table not repairing

Table info: Database name: user_motiva Table name: wp_options.frm wp_options.MYD wp_options.MYI wp_options.TMD when I do a mysqlcheck -r --all-databases it gets hung on that table even if you ...
1
vote
2answers
410 views

What is the standard system architecture for MongoDB

I know this question is too vague, so I would like to add some key numbers to give insights about what the scenario is Each Document size - 360KB Total Documents - 1.5 million Document created/day ...
1
vote
2answers
4k views

Any way to copy the schema from one MySQL database to another?

I have a MySQL database that contains almost 100 tables. I want to set up N additional MySQL databases on the same server, each running on a different port. And I want each additional database to ...
0
votes
1answer
544 views

MySQL keeps restarting due to running out of memory

how can we prevent this from happening? it generally causes down time of about 30 minutes, which is extremely disruptive for service...
0
votes
1answer
8k views

How do I uninstall MySQL completely?

remove ALL settings. remove ALL databases. remove ALL replication settings. WHat command do I run to remove that?
5
votes
3answers
4k views

Dell SSD options and 3rd party SSDs in Dell hardware

Anyone have any experience with Dell's SSDs? We're looking at using them for a machine running a key value database like Tokyo Tyrant. They're expensive as well as being pretty small. Only 25/50GB ...
3
votes
2answers
384 views

Good Books About Scaling Up Databases/Servers/etc.? [closed]

I've applied for an internship at a startup company that expects its user base to grow by a large factor in a small amount of time, and so part of their project is to scale everything up so that ...