Questions tagged [database]
System that handles large amounts of data in an organised fashion
309
questions with no upvoted or accepted answers
4
votes
2
answers
5k
views
Large lag on mysql replication (Relay_Log_Pos and Exec_Master_Log_Pos does not increase)
Today my two slave's (one mysql 5.1 and second MariaDB 5.5, master is mysql 5.1) started lagging. Similar situation are quite often with lags rises to even 10000 seconds, because slaves have worse ...
- 5,258
3
votes
0
answers
1k
views
How to access database on Docker swarm without exposing ports to internet?
I had a really hard time coming up with a short and descriptive title for this... Let me explain:
In classic Docker (no swarm mode) I had my system set up in a way that my PostgreSQL database would ...
- 295
3
votes
0
answers
2k
views
Determining Rails max threads and database pool size using Puma and NGINX
I noticed the default pool size for a Rails 5 App using Puma is equal to:
<%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
Which means that you can establish a max number of threads as an ...
- 131
3
votes
0
answers
8k
views
postgresql invalid command \
When trying to import an .out file back into a postgres database, I get many (thousands) of rows like this: psql:G:/Backup/main_db_backup.out:251875: invalid command \
With a different number where "...
- 31
3
votes
1
answer
2k
views
Galera cluster - cannot start MariaDB (CentOS7)
I'm trying to install a Galera cluster with 3 mariadb VM's (all of them CentOS 7). I've never done this before, so I was following this guide:
http://tunnelix.com/mariadb-galera-cluster-installation/
...
- 39
3
votes
0
answers
49
views
How to handle point in time restore without binlogs/How to split database
I thought about this 3 days already and asked in IRC but I wanna consult your expertise too.
Basically it is about a High Traffic Site that uses Drupal (don't get me started, I have no choice in this ...
- 65
3
votes
1
answer
11k
views
Recover mysql database innodb 'table doesn't exist'
Our server crashed last week and we lost a lot of information. We were able to recover .frm files for the tables, ib_logfile0, ib_logfile1 and ibdata1 but no .ibd.
I've tried the innodb-force-...
- 81
3
votes
1
answer
2k
views
mongodb keep looping while trying to sync data
I'm converting my my standalone mongodb into a replica set. I've added one more member (and I want to add two more members later, and to shutdown the primary server).
My primary mongodb is running 2....
- 43
3
votes
0
answers
74
views
Barracuda SSL VPN Managing Users
Not really sure if this is the right site to be posting on, but I have a question about my Barracuda SSL VPN. I know there is a built-in user database on the VPN.
Is there some way (in my asp.net ...
- 131
3
votes
1
answer
237
views
How do I remove an old SQL Server 2008 Express Database Instance
I am trying to restore a database over the existing one. This worked fine in the past, until all of a sudden, it doesnt.
I have clearly, in the past, attached the same database file under a different ...
- 309
2
votes
1
answer
2k
views
Should the maximum number of request threads be similar to the maximum number of database connections?
I have a spring boot application with an embedded tomcat server. The max-threads setting of tomcat is currently set to 250. Next to that the spring boot application also uses a MySql database as the ...
- 137
2
votes
0
answers
158
views
Why is my mongodb setup not reaching EBS IOPS limit?
I have an application that is constantly pumping data to MongoDB. MongoDB instance is running with 2-replicas, each with 3TB gp2 EBS volume.
As you can see on the following graph of
irate(...
- 173
2
votes
1
answer
489
views
How are Prometheus Snapshots stored on disk?
I am trying to understand how to scale my Prometheus and look at the storage mechanic for this.
Lets assume the following:
Prometheus data storage Directory: 20GB in size
Snapshots amount: 3
Snapshot ...
- 121
2
votes
2
answers
712
views
ZFS performance with databases and more than 80% space used
We have Oracle 11.2 databases on SAN storage (fibre channel on EMC) with Solaris 11.3.
For the development environments, the space used on the filesystems is over 80% most of the time.
How important ...
2
votes
0
answers
807
views
MongoDB Very slow queries since yesterday
Since yesterday I am having huge trouble with my MongoDB. I have a 100GB collection with over 50M documents. Making a find on them and a .count() was never an issue until yesterday.
Somehow, since ...
2
votes
0
answers
3k
views
MariaDB Cluster is having timeout issue between nodes
I'm having some hard time diagnosing MariaDB Cluster issue that I would like to have some advices.
We're running a 3-nodes MariaDB Cluster, each node is on a dedicated ESXi server, and connected by ...
user472241
2
votes
0
answers
1k
views
How to associate existing RDS instance to Elastic Beanstalk, with the right storage size?
I found the post How to associate an existing RDS instance to an Elastic Beanstalk environment? and tried to associate my existing RDS instance to Elastic Beanstalk by following it:
First, creating a ...
2
votes
1
answer
151
views
Azure elastic database pool storage limit briefly drops to zero
Yesterday and today I my azure web app experienced bunch of timeouts and 502's for few minutes. While investigating I found out that database storage limit and storage used dropped to zero at that ...
- 123
2
votes
0
answers
2k
views
pg_restore changing object owner not working ,
I just try the solution found here : https://stackoverflow.com/questions/31469008/postgresql-backup-database-and-restore-on-different-owner
And I still have an error on the inexistent object owner.
...
- 161
2
votes
1
answer
2k
views
How can I restart a server and connect it to an existing cluster?
I have three Nginx servers + php7 + memcache + Cluster Galera + Mariadb + Monit.
Everything works and all three are connected.
But if I now want to edit the my.cnf file to play the utilized RAM or ...
- 41
2
votes
0
answers
6k
views
How can I browse the contents of a Berkeley DB file?
I'd like to see what's in a bdb file in my Thunderbird profile:
$ file key3.db
key3.db: Berkeley DB 1.85 (Hash, version 2, native byte-order)
I've tried this and it doesn't seem to work:
$ db_dump -...
- 1,115
2
votes
0
answers
1k
views
Huge oplog size after full sync on one secondary causing performance issues - MongoDB
I'm a MongoDB newbie. I have a replica set with 3 members (1 primary and 2 secondaries) on CentOS 6 and Mongo 2.6.8.
One of the secondaries crashed because of high consumption of memory and I couldn'...
2
votes
1
answer
86
views
Managing data created with an API in multiple environments
I have a staging and production environment for my application and store Stripe::Customer ids in my database. Occasionally, I want to copy production data to my staging database, but the Stripe::...
- 191
2
votes
1
answer
1k
views
nodejs postgresql database connection string
Is there a way where I can specify(include) both the master and slave server in nodejs db (postgresql) connecetion string in a replicated envirnment?
My aim is 'if master is down then the read/write ...
- 1,262
2
votes
1
answer
239
views
What does -fixcolnames do in log parser 2.2?
I'm using Logparser 2.2 to try to bring a .csv filled with Performance Counter information into a SQL database, and I'm using the following command line command:
logparser "Select * into IISCounters ...
- 463
2
votes
0
answers
2k
views
Corrupt mysql system tables
I am having issues with the columns_priv table in the mysql system database. I cannot add new users currently.
I have tried repairing it using mysqlcheck --auto-repair --all-databases --password but ...
- 235
2
votes
2
answers
280
views
Creating a test database with copied data *and* its own data
I'd like to create a test database that each day is refreshed with data from the production database.
BUT, I'd like to be able to create records in the test database and retain them rather than ...
- 1,310
2
votes
1
answer
315
views
Strategies for protecting third party PHP web app config files
I'm interested in comments and strategies for protecting PHP config files, particularly those with Db credentials.
The stock advice, set the file extension to *.php, is fine as far as it goes, but it ...
- 331
2
votes
0
answers
3k
views
Firebird 2.5 Database Corrupt
We have an issue where a database hangs the server when:
a backup is performed (Hangs on a specific table)
selecting * or count(1) from a specific table
or viewing data that is related to the table (...
BrendanH
2
votes
3
answers
938
views
What are performance limits of a database?
What are some rough performance limits (read/s, write/s) for a single database server (no master-slave architecture), assuming storage on disk? How many read/s, write/s, depending on the kind of disk? ...
Tommy
2
votes
3
answers
914
views
SQL 2008 - db mail issue
I have two instances of SQL Server 2008. One was upgraded from SQL Server 2000 and one was a clean, new install. The instances are running on different nodes of the same cluster, although I have ...
Chris
2
votes
1
answer
1k
views
Delay index build until SQL Server table load is complete with SSIS
I have a large table that I am updating. Is it possible to disable index updates on the destination table until the load is complete? It seems like a waste for it to be constantly updating the index ...
- 163
1
vote
0
answers
69
views
How to enable SQL trace for AWS Oracle RDS
To enable sql trace under Oracle you would typically run the following via sqlplus after login:
ALTER SESSION SET SQL_TRACE=TRUE
On Oracle 19 RDS this results in the following:
Error report -
ORA-...
- 11
1
vote
0
answers
90
views
Large WordPress E-Com Site - Very slow performance (High volume of Post Meta data and Order Data)
I have an e-com store running on WordPress (Together with using WooCommerce) - We've operated the site for around 4 years now, and in that time, our website has grown heavily in sales and customer ...
- 11
1
vote
1
answer
87
views
Logged in users cannot store visualizations within their OpenSearch tenant space
We have recently switched 1 of our 3 clusters from Elasticsearch to OpenSearch. The cluster is receiving data into 30+ data streams and the data is searchable in Dashboards using index patterns etc. ...
- 111
1
vote
0
answers
92
views
Haproxy 2 domain ssl crt conflit
Haproxy load balancer - I configured 2 different domain SSL certificates and get a conflict. For example I have two domains api.xxx.com and api.yyy.com. If try to get api.xxx.com its showing yyy cert ...
- 11
1
vote
0
answers
625
views
Postgresql - Master\Slave Async Replication - How to quickly fail back and forth?
I have two hosts set up as very simple master\slave. All servers connect to the floating IP 10.1.1.10.
Host A:
Designation: Master
Floating IP - 10.1.1.10
Standard IP - 10.1.1.11
relevant pg_hba....
1
vote
0
answers
1k
views
Can I recover a db password?
For our project in GCP, the app has a production Postgres db instance, we don't have the current password, therefore we can't access the db other than through the app. If we change the password, then ...
google-cloud-platform
1
vote
1
answer
245
views
Idle database connection lost
I am able to connect to the database server (Firebird), run queries, all those fun things, but after an undetermined period of inactivity, the next query attempt generates the following error
Unable ...
- 111
1
vote
1
answer
2k
views
Only rdsadmin account on AWS RDS with full master permissions
Good afternoon all,
I'm hoping someone is able to help with the following.
On our main live database it's just been brought to our attention that we need to be able to kill user processes that are ...
1
vote
1
answer
244
views
Oracle database - solaris 10 9/10 where is i can find
We have Oracle tech login but cannot find the elusive Solaris 10 9-10 SW load iso file.
We have just about everything else for Sol 10 .... and can't restore a an ORACLE DATABASE backup without it to ...
1
vote
0
answers
2k
views
Mysql server won't start after changing bind-address
I have two dropplets at digitalocean, one is for http server and another is for database.
In mysql config /etc/mysql/mysql.conf.d/mysqld.cnf I have set bind-address to digitalocean private ip and ...
- 11
1
vote
0
answers
1k
views
PostgreSQL cannot work when opens SSL on Windows 10
I am not sure this topic will fit here but I don't know ask it where.
I am trying to open the SSL of PostgreSQL 10.16 on Windows10.
I read a lot of documents about creating an SSL certificate to ...
- 11
1
vote
0
answers
94
views
Cost-Effective Database Architecture for 2000-3000 concurrent users | AWM Cloud
We have conducted the exam of 2000 concurrent users this week. And it went smoothly.
Our Architecture was, we used an AWS AutoScaling setup with 5 servers running at the start (each has 2CPU, 8GB RAM)....
- 11
1
vote
0
answers
96
views
Would a flat or tree-like file structure allow for faster fetching of files by Path?
Would a flat file structure allow for faster locating/fetching of files by Path in an NTFS (Windows 10 File System) or would it essentially be as efficient as a highly structured tree with a bunch of ...
- 191
1
vote
0
answers
20
views
Why would files appear corrupt but then be fine days later?
The Setup: Dedicated CentOS 7 web server hosting a CRM database on a company’s internal network. Everyday, the server backs up to itself and then the backups are copied offsite.
The Problem: This ...
- 121
1
vote
0
answers
114
views
Experimental setup of PostgresXL on Kubernetes with Local volumes
I'm making a Gedankenexperiment about deploying PostgresXL on Kubernetes (k8s) where each datanode uses local (Directly Attached) storage.
Imagine we have the following nodes:
2x highend machines ...
- 111
1
vote
0
answers
366
views
MySQL server has gone away
MySQL is running on Ubuntu 18.04. We have many databases running on this single server and many sites running from those databases.
All sites working fine except one site. That site frequently goes ...
- 11
1
vote
0
answers
133
views
Setup Local Persistent Volume on physical machines with Kubernetes
Let me make a statement first: I'm new to Kubernetes, please take it easy if I'm asking wrong questions.
Ok, here is what I'm gonna do. I'm planning to build a Kubernetes for my project using some ...
- 131
1
vote
0
answers
182
views
How to collect historical data using smartctl?
I'm trying to do some analysis on the SMART stats of disk drives. Unfortunately, I'm not collecting/storing smart stats data daily for the analysis. I will be writing some procedure to collect it from ...
- 111