Integrate External Nodes with Omnia Kubernetes Cluster

Omnia provides you a way to integrate external nodes (with pre-installed OS) into an existing Omnia Kubernetes (K8s) cluster.

Note

This feature is currently validated only on nodes running Ubuntu 24.04 OS.

Prerequisites

Before integrating external nodes, ensure the following requirements are met:

OpenSSH: Ensure OpenSSH is enabled during the Ubuntu OS installation process.
Enable root user: Edit the /etc/ssh/sshd_config file and set PermitRootLogin yes (instead of PermitRootLogin prohibit-password).
Root password: Ensure that all servers use the same root password. The root password should have minimum 8 characters and should not contain -, \, ', or ".
Fully Qualified Domain Name (FQDN): Set the FQDN for each node (Example: compute1.omnia.test). Ensure that the same is also updated in the etc/hosts file on the OIM.
IP Configuration: Assign either a static or dynamic IP from the PXE network (Example: 10.5.0.x). Ensure that the same is also updated in the etc/hosts file on the OIM.
Internet Access: The node should have internet access. To test the internet connection, execute:
```
wget https://github.com/opencontainers/runc/releases/download/v1.2.3/runc.amd64
```

Steps to Integrate External Nodes

Ensure that the OIM server is set up and running with Ubuntu 24.04 OS.
Activate the Omnia Python virtual environment using the following command:
```
source /opt/omnia/omnia171_venv/bin/activate
```
Create an inventory file in the following format:
```
[compute]
10.5.0.101
10.5.0.102
```
Note

Only IP addresses are supported in the inventory file. Ensure that these IPs align with the PXE subnet.
Run the connect_external_server.yml playbook using the following command to connect the external servers, where <inventory> is the path to your inventory file:
```
cd utils
ansible-playbook connect_external_server.yml -i <inventory>
```
While execution, the playbook will prompt for the root password. Enter the root password that has been configured on all servers.
Navigate to the input/omnia_config.yml file and set the k8s_offline_install variable to false. For more information, click here.
Note

If the input/omnia_config.yml file is encrypted, run the following command to decrypt it:
```
ansible-vault edit omnia_config.yml --vault-password-file .omnia_vault_key
```

Navigate to the input/software_config.json and remove all software entries except k8s and nfs. Omnia supports adding external nodes with only these two entries in the input/software_config.json. Here’s a sample:

{
  "cluster_os_type": "ubuntu",
  "cluster_os_version": "24.04",
  "repo_config": "always",
  "softwares": [
      {"name": "nfs"},
      {"name": "k8s", "version":"1.31.4"}
  ]
}

Run the omnia.yml to deploy a Kubernetes cluster with the new nodes, where <inventory> is the path to your inventory file consisting of the external nodes:
```
ansible-playbook omnia.yml -i <inventory>
```

Note

To build a Kubernetes cluster with a mix of Omnia-provisioned nodes and external nodes (with a pre-loaded OS), do the following:

First, deploy Kubernetes on the Omnia-provisioned nodes by running omnia.yml with k8s_offline_install: true.
Once that’s complete, add the pre-loaded OS nodes by re-running omnia.yml with k8s_offline_install: false.

If you have any feedback about Omnia documentation, please reach out at omnia.readme@dell.com.