Installation Troubleshooting

1. Cluster not connecting

Possible solutions

The Helm command didn’t run

Make sure to copy the code in the cluster connection dialog and run it in your cluster.

Kollector did not start

The ARMO portal flags a cluster as connected only after the Kollector microservice creates a connection. Make sure that the Kollector is up and running:

kubectl -n kubescape get pods | grep kollector

The Kollector should be running:

NAME                          READY   STATUS    RESTARTS   AGE
kollector-0                   1/1     Running   0          1m38s

Kollector did not connect

The ARMO portal flags a cluster as connected only after the Kollector microservice creates a connection. Make sure that the Kollector managed to connect to the ARMO Portal. Run:

kubectl -n kubescape logs -f $(kubectl -n kubescape get pods | grep kollector |  awk '{print $1}')

You should see the following log:

"msg":"connected successfully","URL":"wss://report.armo.cloud/k8s/cluster-reports?

2. Scanning results are not available

After running the Helm install and seeing that the cluster is connected, the UI does not forward you to the results page

Possible solutions

Refresh the page

Sometimes, the best solution is to simply refresh your webpage :)

The Kubescape or kubevuln pods are not running

You may find that one or more pods are not running after installation. Run the following:

kubectl -n kubescape get pods

Below you can see that only one pod is not running:

NAME                          READY   STATUS    RESTARTS   AGE
gateway-5c8f486dcf-hsbpq      1/1     Running   0          5m38s
kollector-0                   1/1     Running   0          5m38s
kubescape-744bc74f8c-z9drg    0/1     Pending   0          5m38s
kubevuln-7648c9c88f-6q7zd     0/1     Pending   0          5m38s
operator-5c86b8f97d-59nw5     1/1     Running   0          5m38s

Check the status of the pods that aren't running as follows:

kubectl -n kubescape describe pod <podname>

Below, the pod kubescape-744bc74f8c-z9drg has insufficient CPU in the Events section, at the bottom.

Name:         kubescape-744bc74f8c-z9drg
Namespace:    kubescape
Priority:     0
Node:         cluster-8260/192.168.58.2
Start Time:   Tue, 07 Mar 2023 12:33:33 +0200
Labels:       app=kubescape
              app.kubernetes.io/instance=kubescape
              app.kubernetes.io/name=kubescape
              helm.sh/chart=kubescape-cloud-operator-1.10.3
              helm.sh/revision=2
              pod-template-hash=5cff758d74
              tier=ks-control-plane
Annotations:  <none>
Status:       Running
IP:           172.17.0.9
IPs:
  IP:           172.17.0.9
Controlled By:  ReplicaSet/kubescape-5cff758d74
Containers:
  kubescape:
    Container ID:  docker://d6f8a3c613f2ea774e5f8458554b7d70a0b4694370ea6cb522ad5a96c777fdef
    Image:         quay.io/kubescape/kubescape:v2.2.4
    Image ID:      docker-pullable://quay.io/kubescape/kubescape@sha256:91d1ad9bf22757b972324a3909da05dbcd9bff9ed4544d04e85a47e7069b65eb
    Port:          8080/TCP
    Host Port:     0/TCP
    Command:
      ksserver
    State:          Running
      Started:      Tue, 07 Mar 2023 12:33:52 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     600m
      memory:  800Mi
    Requests:
      cpu:      250m
      memory:   400Mi                   
    Mounts:
      /home/ks/.kubescape/config.json from kubescape-config-volume (rw,path="config.json")
      /home/ks/.kubescape/host-scanner.yaml from host-scanner-definition (rw,path="host-scanner-yaml")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-4fh8f (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kubescape-config-volume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      kubescape-config
    Optional:  false
  host-scanner-definition:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      host-scanner-definition
    Optional:  false
  kube-api-access-4fh8f:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason             Age                From                Message
  ----     ------             ----               ----                -------
  Warning  FailedScheduling   6m1s               default-scheduler   0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
  Warning  FailedScheduling   52s                default-scheduler   0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
  Normal   NotTriggerScaleUp  1s (x6 over 5m2s)  cluster-autoscaler  pod didn't trigger scale-up:

Solution
The cluster had a single node assigned. Increasing the node count addressed the problem and the remaining Kubescape pods automatically started once the additional nodes were available.


3.The Kubescape pod restarted

You may find that the Kubescape pod restarted, this means Kubescape probably did not complete the scan. Run the following command:

kubectl -n kubescape get pods | grep kubescape

You should see the following output:

NAME                          READY   STATUS    RESTARTS   AGE
kubescape-744bc74f8c-z9drg    1/1     Running   1          5m38s

Possible solutions

Typically the restart is caused due to memory issues.

Solution 1

We recommend increasing the memory request and CPU (by editing the Deployment):

kubectl -n kubescape edit deployment kubescape

After the kubescape pod is ready, restart the operator pod:

kubectl -n kubescape rollout restart deployment operator

Solution 2

Re-install the Helm chart and manually set the requested memory/CPU (add this to the helm upgrade --install command):

--set kubescape.resources.requests.memory=800Mi \
--set kubescape.resources.limits.memory=1500Mi \
--set kubescape.resources.requests.cpu=500m \
--set kubescape.resources.limits.cpu=1000m