> For the complete documentation index, see [llms.txt](https://docs.cloudcyte.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.cloudcyte.com/troubleshooting/proxy-troubleshooting/portal-server.md).

# Portal Server

Each article describes a different situation.

* Solution for: "failed to create fs.notify watcher: too many open files" error, execute this command on the master node machine:
  1. sudo sysctl -w fs.inotify.max\_user\_watches=2099999999
  2. sudo sysctl -w fs.inotify.max\_user\_instances=2099999999
  3. sudo sysctl -w fs.inotify.max\_queued\_events=2099999999
  4. sysctl -p
* Solution for authentication: If sec.yaml keys and values are the same on the UI and yaml, redeploy the sec component. But this time please only use yaml file instead of adding keys & values by hand, Rancher encrypts secrets in his own way. If key & value fields are given by hand, Rancher's encryption is not working and an authentication problem will happen.
* If the onprem.cloudcyte.com gives "cannot get/" error, please check the auth pod. Sometimes it needs to be restarted.
* If the servers have issue with connecting to the internet, please check:
  1. certificates -> "Storage -> Secrets" on rancher
  2. configmap -> "Storage -> ConfigMaps" on rancher, there should be no\_proxy/NO\_PROXY and http(s)\_proxy/HTTP(s)\_PROXY fields
  3. pods -> check the pods that have config maps proxy variables
  4. /etc/resolv.conf
  5. /etc/resolvconf/resolv.conf.d/head
  6. /etc/dnsmasq.conf
  7. /etc/network/interfaces
* Check DNS Port: 
  1. ss -lptn 'sport = :53'
* Show k8s container logs of the rancher:
  * kubectl logs -f \<pod\_full\_name>
* For applying the proxy settings again or resetting the rancher, please use rke2-manager.sh on the portal server, it is a builtin script in the portal that CyberCyte provides. With the script, many confiurations can be applied with minutes.
* If the servers have issue with communicating with the other servers, please check:
  1. /etc/resolv.conf
  2. /etc/resolvconf/resolv.conf.d/head
  3. /etc/dnsmasq.conf
  4. /etc/netplan/\<file>.yaml
* Check DNS Port via executing these commands:&#x20;
  * ss -lptn 'sport = :53'
* Check services via executing these commands:
  * systemctl status rke2-server
* Check server resources via executing these commands:
  * df -h
  * top
  * btop (if exists)
  * lsblk
  * ncdu (if exist)