Your post doesn't describe much in the way of specs (memory on the LS indexer, log volume or much else) but I'll try and answer your questions best I can first. Disclaimer: I'm one of the logstash devs -
Apache going nuts was likely a side effect of the logstash process acting up. I'd put that aside for now.
The sane way to make ES f/b/s is add more ES ...
Elasticsearch does not shrink your data automagically. This is true for any database. Beside storing the raw data, each database has to store metadata along with it. Normal databases only store an index (for faster search) for the columns the db-admin chose upfront. ElasticSearch is different as it indexes every column by default. Thus making the index ...
you are almost there, It does not matter if is a virtual machine or a physical machine, those settings are always changeable.
I'll show 3 methods.
1) It's better to execute as root, if possible.
2) /proc on unix is not a real filesystem, it's a in-memory kernel file system, but it appears to be like a normal disk file system. You ...
The answer is: it depends.
A blog post by Adrien Grand, who works on Elasticsearch did some benchmarking with this from the Lucene standpoint. It looks like he was getting about 2x improvement.
He also mentions LUCENE-4226, where some of this underlying compression work was done. There's a few benchmarks listed within, comparing the various compression ...
The way we handle this is by creating multiple groups of servers in a layered stack (even if a group currently only needs one instance). The first layer is your Elastic Load Balancer, clearly.
The second layer is an Auto Scaling Group of web servers (multi-availability zone). These boot a custom AMI designed be in a proper ready-state for this task on ...
You will have to delete the cluster for billing to stop. However, if you want to backup the data for later experiments, you can take manual snapshots of the indices to your S3 buckets.
The next time you spawn a cluster, just restore the snapshot :)
The settings in /etc/security/limits.conf usually get applied by the pam_limits PAM module. Services started by init don't run under a PAM session however, and so these settings are not applied. The only reliable way I've found to set this is to run ulimit from the initscript itself. Edit the ElasticSearch initscript and add something like this near the top:
You have to mirror the parts you want searched in elastic search.. you just don't tell elastic search to index a mysql database.
Basically whatever gets put/changed/deleted in mysql also has to do the same in elastic search. You don't have to put all the info just what you want to search on and a way to link it to the mysql info.
I had an issue where I deleted my index in ElasticSearch, then recreated it. After that nothing appeared in Kibana.
The solution: Simply delete the kibana index pattern on the Settings tab, then create it again. Same name same everything, but now it gave me data.
Elasticsearch is a search and index kind of database, build and optimized for searching and analytics, not for caching.
For a caching server you would use Memcached. But these days there are other alternatives such as Redis (Key-Value database) which is much faster and supports more complex solutions. Carl Zulauf did a great job describing the differences ...
If someone will be interested how I resolved an issue with logs...
After some investigation, I found that you can actually set the amount of logs to store in logging.yml which is by default lie in /etc/elasticsearch, by adding:
line like this:
This is a known issue with Elasticsearch on Ubuntu 15.04. There is a GitHub issue. The temporary fix:
As a workaround you can uncomment the paths in the /etc/default/elasticsearch file and it should work.
From unix exchange.
This is happening because by default, AF_INET6 sockets will actually work for both IPv4 and IPv6. See section 3.7 - Compatibility with IPv4 Nodes of RFC 3493 - Basic Socket Interface Extensions for IPv6
But as you've figured out, firewalld is enabled out of the box.
create this file to your /etc/firewalld/services/elasticsearch.xml
If you use later versions of Logstash with Kibana:
I deploy Kibana into a virtual host in an Apache at /kibana/ and route the Elasticsearch API through a reverse proxy such that is available at /elasticsearch/:
You need to adapt Kibanas ...
When were you hit? Could it be the leap second problem that hit on both July 1st and recently on certain servers on August 1st, causing both Linux kernel crashes and Java CPU load problems? Here's one of the main references for the July leap, and a thread about the August leap.
Short answer is no it isn't asynchronous. Longer answer is "Not unless you wrote the backend yourself to do so."
If you're using XHR, each request is going to have a different worker thread on the backend which means no request should block any other, barring hitting process and memory limits. While XHR presents an event-based interface, it's still a live ...
This is a difficult question to answer, because there's generally not a single thing that you should do. Rather you collect info from a variety of sources, see what you've got and follow those leads further. Still here's some ideas:
significant info I see in what you've given us includes:
a cron job has been installed, and you know some times when it ran
It is enough to rename the folder holding the data.
The folder created by default for a standalone server is called elasticsearch, renaming it to the cluster name (mycluster in the context of the question) makes the previous data available.
I think your "virtual machine" is actually an OpenVZ container (which you can verify this by running virt-what).
In this case, you can't change vm.max_map_count sysctl or many others. The values are fixed.
This is a well known issue with elasticsearch (issue #4978). It's not just Elasticsearch. Java apps are well known to perform poorly on various OpenVZ ...
Give a name to your cluster and the issue will be solved.
Edit config/elasticsearch.yml in your elasticsearch directory.
Uncomment cluster.name: elasticsearch and change it to something like cluster.name: your_hostname
Then try to start elasticsearch again.
Assuming your elasticsearch service is running on port 443. You need to specify keystore path of elasticsearch along with a password. Check this link for all available options for logstash elasticsearch output.
PS: Make sure elasticsearch service is accessible from your logstash machine. Confirm using telnet from logstash to elasticsearch.
In an Elasticsearch cluster, must all nodes be identical i.e. having
or is it allowed for the nodes to be different?
No, they do not need to be identical. Depending on use case of the cluster, and the roles assigned to each node, hardware differences are to be expected.
We use HAproxy to balance between two redundant clusters. During normal operation each receives ~50% of traffic; each is provisioned to take 100% when necessary.
We experienced a fault recently based on a failure case we had not planned for: all client and master nodes stayed up, so our cluster was responsive to REST–but all data nodes were temporarily ...
Even tho Resizing Persistent Volumes using Kubernetes from Kubernetes 1.11, there seems to be some issues with it.
As discussed in GitHub: StatefulSet: support resize pvc storage in K8s v1.11 #68737
Due to this limitation, many database Operators for Kubernetes don't support PVC resizing.
It is a critical issue because when your database becomes bigger ...
Unfortunately, you are just not going to get the same level of performance with the MySQL fulltext search as you will from the dedicated search solutions, but that begs the question--do you care?
sphinxsearch will give you faster indexing, more advanced queries, incremental updates, and it will work out of the box with MySQL--but it is still a separate ...