Questions tagged [etcd]
A highly-available key value store for shared configuration and service discovery
76
questions
0
votes
1
answer
271
views
Kubernetes component status reports etcd as unhealthy
I have a 3 node kube control network setup with etcd, kube-apiserver, kube-control-manager and kube-scheduler running. When I check the status of the components it says etcd is unhealthy, but etcd ...
- 113
0
votes
0
answers
58
views
CoreDNS not recognizing queries with unempty "text" keys in etcd records
I've spent my last 2 days trying to figure out what's wrong with my setup and couldn't find a resonable answer or solution, so I want to share it with you.
What happened: I've deployed a new etcd ...
1
vote
0
answers
289
views
Changing out all Control Plane Nodes safely - Best Practice
I have a 3 control plane node HA kubeadm cluster that I need to completely replace. I have already replaced the worker nodes. How do I completely replace the control plane without downtime?
I use a LB ...
- 11
0
votes
1
answer
238
views
Patroni interconnection failover
3 Data Centres:
Patroni version: 2.1.4
PostgreSQL version: 14.4
Etcd version: 3.3.11
DC
Server
Name
Host
Status
1st
Patroni
patroni-s11
172.16.0.2
Leader
1st
Patroni
patroni-s12
172.16.0.3
Sync ...
- 1
1
vote
1
answer
417
views
What exactly is Kubernetes API datastore?
THis page mentioned Kubernetes API datastore -- https://projectcalico.docs.tigera.io/getting-started/kubernetes/flannel/flannel -- and it seems to suggest that Kubernetes API datastore is a ...
- 121
1
vote
3
answers
2k
views
kubernetes: how to check pod IP range allocated to node
Every node in a kubernetes cluster has a dedicated range of IP addresses that it can allocate to the pods.
How can one check what that IP range is?
4
votes
2
answers
2k
views
All kube-system pods keep crashing, etcd receives sigterm
I'm trying to set up an on-premises Kubernetes cluster. For starters I used kubeadm to initialize my control plane on one node. But all kube-system pods constantly crash. I took a deep look into the ...
- 143
1
vote
2
answers
236
views
ETCD restore granularity
I'm studying some backup and restore solutions for Kubernetes and Openshift, typically with Velero. So, I'm not familiar with ETCD backups, but I would like to ask about the granularity level of etcd ...
- 11
0
votes
0
answers
791
views
Changed IP addresses of etcd nodes. How to restart cluster?
I have a k8s cluster set up using 'hardway' a while ago. I've recently changed my network around and needed to set new IPs for the controller nodes (which are also the clustered etcd hosts).
I've ...
- 354
1
vote
0
answers
3k
views
Error from server: etcdserver: request timed out - error after etcd backup and restore
I have done the etcd backup and then a restore on the same cluster and now I'm having these issues where I can list resources but I can't create or delete.
It's a 1 master and 2 workers setup , ...
- 121
1
vote
0
answers
934
views
ETCD snapshots causing etcdserver: leader changed
For a while now we've experienced regular errors from operations on kube API in AKS resulting in etcdserver: leader changed message. From what we've learned there is an ETCD snapshot performed every ...
0
votes
0
answers
574
views
Clean install of Kubernetes on Raspberry Pi doesn't work
I'm trying to do a clean install of Kubernetes 1.23.x on a cluster of four Raspberry Pis, each running the x64 version of Raspberry Pi OS, however I am running into a major snag as soon as I try and ...
- 719
0
votes
1
answer
2k
views
etcdctl endpoint health --> unhealty?
If I run etcdctl it hangs. If I use --debug=true I see:
root@k8scp:~# kubectl exec -n kube-system -it etcd-k8scp sh
sh-5.1# ETCDCTL_API=3 etcdctl --debug=true endpoint health
{"level":&...
- 3,573
0
votes
1
answer
161
views
etcdctl: unknown command "save" for "etcdctl"
I entered the etcd container:
kubectl -n kube-system exec -it etcd-k8scp -- sh
The I try to backup the container like explained in the K8s docs
ETCDCTL_API=3 etcdctl --endpoints $ENDPOINT snapshot ...
- 3,573
0
votes
1
answer
85
views
How to detect traffic change and trigger GitLab pipeline?
We have two etcd clusters running in Kubernetes one act as primary(A) and the second one act as backup(B). We also have etcdctl make-mirror in place between these two clusters, now the problem is if ...
- 101
2
votes
0
answers
291
views
Kubernetes on Flatcar Linux will not start up kube API server after a reboot of the master node
Thank you for reading and taking your time to review this problem.
I have a problem using my Kubernetes cluster.
It is running Flatcar Linux, made by Kinvolk, recently acquired by Microsoft. I have ...
- 101
0
votes
0
answers
2k
views
How to fix etcd within a kuberentes cluster?
I have a bare-metal (kubeadm) kubernetes cluster that's really unstable, and I traced it back to an etcd issue.
From the etcd pod's description I get:
Image: k8s.gcr.io/etcd:3.4.13-0
Liveness: ... #...
- 281
1
vote
1
answer
520
views
systemd terminates etcd service started with podman - reception only permitted for main PID
I try to start etcd as a systemd service run in a container with podman.
After start I get this error log from systemd:
systemd[1]: etcd.service: Got notification message from PID 4696, but reception ...
- 1,193
1
vote
1
answer
1k
views
How to start etcd in docker from systemd?
I want to start etcd (single node) in docker from systemd, but something seems to go wrong - it gets terminated about 30 seconds after start.
It looks like the service starts in status "...
- 1,193
1
vote
0
answers
497
views
How to spin up a ETCD docker container and restore a backup obtained from a kubernetes ETCD Cluster node?
I am trying to spin up a ETCD node in docker and restore a backup obtained from the Kubernetes Cluster node.
ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --user=root:'root' snapshot save ...
2
votes
0
answers
739
views
Kubernetes V1.19.13 - kubeapi servers not able connecting to different etcd database
I have upgraded Kubernets cluster ( 3 master, 3 etcd servers database) from 1.18 to v1.19.13 and ETCD to 3.4.13. since than API servers are not stable, keep switching different etcd server, because of ...
- 243
1
vote
1
answer
838
views
ETCD Cluster configuration for Kubernetes: Which one should be considered?
I would like to know how to deploy a ETCD cluster for Kubernetes.
It seems like there are two different documentation and I don't know which one must be considered or the impact of each of them.
From ...
- 121
0
votes
1
answer
5k
views
Kubernetes: kubeadm join fails in private network
I'm trying to set up a HA Kubernetes cluster on Hetzner Cloud following this guide. I've created 6 servers, 3 control plane hosts and 3 workers. When trying to use kubeadm to join the second server to ...
- 11
2
votes
1
answer
1k
views
How can we modify the heartbeat synchronization time of the etcd cluster on Kubernetes
I want to modify heartbeat-interval of embedded etcd cluster running on kubernetes, I know while installation we can do with kubeadm extraargs but how to do in running cluster?
Before modifying I want ...
- 150
0
votes
1
answer
652
views
Centos 7, HA postgresql12, patroni with etcd v3.4
I followed this document but dont know how to enable v2 so that patroni can work with, can anyone help?
https://computingforgeeks.com/setup-etcd-cluster-on-centos-debian-ubuntu/
- 51
-2
votes
1
answer
576
views
etcd: Max Storage Size Update [closed]
Can anyone confirm whether the improvements mentioned How Does Alibaba Ensure the Performance of System Components in a 10,000-node Kubernetes Cluster? have actually been included in etcd 3.4.0?
Docs ...
- 1
0
votes
2
answers
3k
views
ETCD database cluster certificate renewal for Kubernets external database setup
I have deployed 3 node external ETCD database (etcdctl version: 3.4.7) cluster for my Kubernetes v1.18.6 cluster using etcdadm tool. my certificate is expring in couple of months.
I believe kubeadm ...
- 243
1
vote
3
answers
10k
views
Kubernetes OCI runtime exec failed - starting container process caused "exec: \"etcdctl\": executable file not found in $PATH": unknown
Background
Created a fresh Kubernetes cluster using kubeadm init --config /home/kube/kubeadmn-config.yaml --upload-certs and then joining the 2nd control plane node by running the below.
kubeadm join ...
- 71
0
votes
1
answer
675
views
Kubernetes API: Compare and update config map key
Etcd has a concept of Atomic Compare-and-Update by comparing the key's value before executing an update. I'd like to use this feature for updating a ConfigMap in my Kubernetes cluster. I'd like to ...
- 267
0
votes
2
answers
119
views
Upgrade multi etcd cluster running inside docker container
Currently my k8s cluster is on v1.16.x and I want to upgrade it to v1.17.x for which ETCD has to be upgraded to 3.4 (currently 3.3). My setup is bit complex as I'm running ETCD outside the master ...
- 101
0
votes
0
answers
902
views
Kubernetes etcd panic: store.keyindex: put with unexpected smaller revision
After an ungraceful shutdown of our Kubernetes cluster, the API server didn't start working. After some investigation, I found the following errors in each etcd member on each node.
How can I recover ...
- 103
2
votes
2
answers
5k
views
Kubernetes - kubeadm join - Connection refused after new control plane joined
Issue
I am trying to join a second control-plane node to a K8S cluster. The first node has a IP of 10.1.50.4 where the second node has a IP of 10.1.50.5. There is a load balancer Virtual IP for the ...
- 301
3
votes
0
answers
811
views
k3s without HA: how to switch master node?
Rancher documentation on k3s is quite nice and its HA support (both with external DB or embedded etcd) look nice, but I don't want/need an HA setup.
In case my master node fails, I don't mind having ...
- 123
0
votes
1
answer
3k
views
Error response from daemon: {“message”:“No such container: kubelet”}
When adding a new node to a Kubernetes cluster I end up with this error :
+ docker start kubelet
Error response from daemon: {"message":"No such container: kubelet"}
Error: failed ...
- 1
0
votes
0
answers
538
views
ETCD client connection to etcd servers from k8s api server issue
We expect the following issue with etcd cluster. We generated certificate with the following configuration:
{
"CN": "client",
"hosts": [""],
"...
1
vote
1
answer
3k
views
TLS handshake issues with etcd
We are using external etcd cluster for k8s cluster. We connected master to this etcd server but receive
"tls: first record does not look like a TLS handshake"
How to fix this issues? (for ...
0
votes
2
answers
4k
views
How to recover from master failure in kubernetes
I have three nodes multi-master kubernetes(1.17.3) cluster(Stacked control plane and etcd nodes),
11.11.11.1 - master1
11.11.11.2 - master2
11.11.11.3 - master3
So before going to productions, I am ...
- 150
2
votes
1
answer
312
views
Why do kubernetes kube-api server need etcd-keyfile and kubelet-client-key
As I understand kube-api server acts as a client when communicating with ETCD and Kubelet. Both ETCD and Kubelet act as servers for kube-api. With secure environment (two way SSL authentication), kube-...
- 123
0
votes
1
answer
1k
views
Include own/external etcd cluster in kubeadm init
I have a running etcd cluster with five members (Own etcd cluster for Kubernetes). How can I include this in the kubeadm init command? My idea is that I generate the configuration, edit it manually ...
- 534
1
vote
2
answers
407
views
Own etcd cluster for Kubernetes
I want to build my own Kubernetes cluster across two locations (300 km distance) and integrate it into GitLab.
Let me list my ideas. My question is if I have a mistake in my thinking somewhere and ...
- 534
0
votes
1
answer
692
views
Installing kubernetes with external etcd - calico problems
I faced multiple problems during installation of k8s multimaster cluster with external etcd. I did it before twice, on other sites, successfully, but this time I need help.
calico was installed from ...
- 125
0
votes
1
answer
844
views
kube-calico bird: Mesh_172_26_178_195: Socket error: bind: Address not available
Issue appears when kube-calico service on node trying to connect master: Dec 17 12:03:41 a docker[27052]: bird: Mesh_47_105_189_58: Socket error: bind: Address not available Dec 17 12:03:41 a docker[...
3
votes
1
answer
4k
views
missing '=' etcd when defining service file
I'm struggling while following Kelsey Hightower's "Kubernetes the Hard Way" tutorial. I've gone off script, because I'm trying to bootstrap k8s on a local server.
I've got the point where I'm ...
- 61
0
votes
1
answer
545
views
Is there a way to determine to which etcd server the Kubernetes-apiserver is communicating with?
In the following scenario, is there a way to determine to which etcd server the Kubernetes-apiserver is communicating with??
Let's say we have 3 Master nodes with an external load balancer and 3 etcd'...
0
votes
1
answer
219
views
Does etcd resources are encrypted on gke master nodes?
I do not find any information on etcd database managed by google on kubernetes master nodes. I see how resources (pod/nodes/...) communicates, but I don't know if the data are encrypted before being ...
0
votes
1
answer
396
views
etcd on GCE-load balancing and health checks
I'm trying to set up a secure etcd cluster on CoreOS on top of GCE.
I want to distribute load between several etcd nodes with some kind of load balancing. I want a load balancer that would have an ...
google-compute-engine
2
votes
1
answer
491
views
CoreOS 1353.4.0 - How should I configure Ignition for an etcd cluster member with dynamic discovery?
Using CoreOS 1353.4.0, on DigitalOcean, how should I write an Ignition config for a machine that is to join an etcd cluster with dynamic discovery?
I've written the following Ignition config (...
- 2,105
2
votes
0
answers
446
views
Weighting an etcd instance to have two quorum votes
For obvious reasons, best practice in building clusters that require distributed consensus is to use 3, 5, or another node count that has a tiebreaker in place.
Sometimes, however, one has two ...
- 946
13
votes
1
answer
2k
views
Graceful shutdown of Kubernetes cluster
We have a 4 node Kubernetes cluster, installed using Juju 2.0 and the official canonical Kubernetes charm. The sole purpose of this very local and private installation is development and testing.
One ...
- 131
1
vote
0
answers
901
views
Etcd: machines can't connect to cluster
I reconfigurated my etcd cluster for working with certificates and now when i trying to start i'm getting
First machine:
Nov 05 16:24:43 Dm3Ch-Server-Node1 etcd[13999]: 21b9ef35413b514 received vote ...
- 111
Related Tags
kubernetes × 46
coreos × 20
docker × 5
linux × 4
kubeadm × 3
azure × 3
cluster × 3
systemd × 3
ubuntu × 3
cloud-init × 2
containerd × 2
debian × 2
kubectl × 2
patroni × 2
zookeeper × 2
systemctl × 1
ssl × 1
ssh × 1
shutdown × 1
rkt × 1
rhel7 × 1
rancher × 1
rackspace-cloud × 1
more related tags
Hot Network Questions
- Does the sentence imposed by God, during wilderness wanderings, fit the crime?? The crime: wood gathering on the Sabbath-Numbers 15:32-36
- Do most conference papers get written last minute?
- Airy disks and pixel pitch on circle of confusions?
- Why do Belugas keep executing a go-around?
- tcolorboxes in beamer
- How to validate and reference digraph /special character in Vim?
- C# and C++ Inheritance and Performance - Shouldn't Compilers Handle this Issue?
- The serpent told Eve should would not die from eating the forbidden fruit; how does Eve know what death is?
- Uzzah's sin of touching the ark was actually meant to be in reverence for same, with no time for aforethought, or consequence, so why death? 2 Sam 6:7
- Undefined opcodes in bitcoin core defined as "Synonyms for OP_RETURN" in rust-bitcoin
- Command(s) to know If the current RAM installed by slot is ECC/Non-ECC and Buffered/Unbuffered
- How to check if a raster in PostgreSQL table is tiled
- Faster chess engine in python (with move ordering algorithm?)
- Show positions of duplicated matrix elements
- Why can't I use `defconst` to define a new rx?
- I am not Jewish - can I wish you a Shabbat Shalom?
- Recommended way to sightread grand staff
- Piece identification
- What does this Peter Sellers sentence mean?
- How to handle quiz question with no correct answer?
- "XXXX it is, then" with plural nouns?
- To find rate of change of area of triangle when rate of change and value of length of base and height are 3cm/min, 5cm/min and 8cm,10cm respectively.
- The reason why Semitic languages are written right to left
- Infinite regress in answering 'Why G-d created the world"