.. _co-troubleshooting:

===============
Troubleshooting
===============

The following sections provide information about troubleshooting your |cp|
deployment.

Logs
----

Logs are sent directory to STDOUT for each pod. Use the command below to view the logs for a pod:

::
  
  kubectl logs <pod-name> -n <namespace>

Metrics
-------

* JMX metrics are available on port 7203 of each pod.
* Jolokia (a REST interface for JMX metrics) is available on port 7777 of each pod.

Debugging
---------

There are two types of problems that can go wrong while using Operator: 

* A problem exists at the infrastructure level. That is, something has gone wrong at the Kubernetes layer.

* A problem exists at the application level. This means that the infrastructure is fine but something has gone wrong with |cp| itself, usually in how something is configured.

You should look for `Kubernetes issues <https://kubernetes.io/docs/tasks/debug-application-cluster/determine-reason-pod-failure/>`__ first. 

Check for potential Kubernetes errors by entering the following command:

::
   
   kubectl get events -n <namespace>
   
Then, check for a specific resource issue, enter the following command (using
the resource type example *pods*):

::
   
   kubectl describe pods <podname> -n <namespace>
   
If everything looks okay after running the commands above, check the individual
pod logs using the following command:

::
   
   kubectl logs <pod name> -n <namespace>
   
|cp| containers are configured so application logs go straight to STDOUT. The
logs can be read directly with this command. If there is anything wrong at
the application level, like an invalid configuration, this will be evident in
the logs.

.. note::
   
   If a pod has been replaced because it crashed and you want to check the
   previous pod's logs, add ``--previous`` to the end of the command above.


Troubleshooting problems caused by the datacenter infrastructure, such as
virtual machine (VM) firewall rules, DNS configuration, etc., should be resolved
by infrastructure system administrator.

Testing the deployment
-----------------------

See the following sections for information about testing cluster communication.

Internal validation
^^^^^^^^^^^^^^^^^^^

.. include:: includes/internal-validation.rst

External validation
^^^^^^^^^^^^^^^^^^^

.. include:: includes/external-validation.rst


Common CLI commands
--------------------

The following provides common commands that you may find useful when managing
the cluster.

Helm commands
^^^^^^^^^^^^^

.. include:: includes/helm-commands.rst

kubectl commands
^^^^^^^^^^^^^^^^

.. include:: includes/kubectl-commands.rst