Skip to content

Architecture of OpenNebula

Assignment

"Before you install anything, I want to understand what we are running," Simone Wyss writes. "What runs where, and who talks to whom? If something fails at night, the platform team has to know."

Learning goals

After this chapter you can:

  • Name the building blocks of OpenNebula and assign them to the front-end or the nodes
  • Explain how a request travels from the web interface to a running VM
  • Tell the objects of OpenNebula apart
  • Name the three ways you operate OpenNebula

OpenNebula consists of a front-end that manages the cloud and nodes that run the VMs. The nodes need no OpenNebula service of their own. The front-end controls them over SSH with small driver programs.

The building blocks of OpenNebula The building blocks of OpenNebula in three areas stacked on top of each other. At the top Sunstone, the CLI and OpenTofu reach oned through the API on port 2633. In the middle oned on the front-end manages all objects, stores them in MariaDB, lets the scheduler pick nodes and receives states from onemonitord. The drivers for hosts, VMs, transfer, networks and datastores carry out the jobs. At the bottom the two nodes work with libvirt and KVM, and their datastore 0 holds the disks of running VMs. Notice that no OpenNebula service runs on the nodes. The drivers reach them over SSH. ACCESS FRONT-END NODE 1 NODE 2 API 2633 SSH Sunstone port 2616 CLI onevm, onehost OpenTofu from part 4 oned manages all objects MariaDB database Scheduler picks nodes Monitoring onemonitord Drivers IM, VMM, TM, VNM, DS libvirt VMs with KVM Datastore 0 disks of running VMs libvirt VMs with KVM Datastore 0 disks of running VMs

The core is the service oned. It accepts all requests through its API, manages the state of every object and stores it in a database, MariaDB in the course. The scheduler is a driver of oned and picks a node with enough free capacity for every new VM. onemonitord regularly queries the state of the nodes and VMs.

Drivers do the actual work. oned calls them for each kind of job:

DriverJobIn the lab
IM (Information Manager)record the state and capacity of the hostsKVM nodes
VMM (Virtual Machine Manager)start, stop and move VMslibvirt with KVM
TM (Transfer Manager)bring disks to the node and backssh, copies over SSH
VNM (Virtual Network Manager)set up networks on the nodesvxlan from chapter 2.3
DS (Datastore)store and manage imagesfs on the front-end

When you start a VM in Sunstone, Sunstone sends the request to the API of oned. oned creates the VM in the database, and the scheduler picks a node. The TM driver copies the image over SSH to the datastore on that node, and the VNM driver sets up the network there. Finally the VMM driver has libvirt start the VM with KVM. From then on onemonitord keeps reporting its state. Chapter 2.2 shows which services and ports run on the front-end for this.

In OpenNebula you work with objects. Each has a number, a name, an owner, a group and permissions.

ObjectMeaningCLI
Hosta node on which VMs runonehost
Clustera group of hosts with their networks and datastoresonecluster
Datastorea place to store images or the disks of running VMsonedatastore
Imagea disk that VMs boot fromoneimage
Virtual networka network with an address range for the VMsonevnet
Templatethe description of a VM from which you start VMsonetemplate
VMa running or stopped virtual machineonevm
User and groupwho may do what and who owns itoneuser, onegroup

You access OpenNebula in three ways, and all three talk to the same API of oned. The web interface Sunstone is good for exploring and for the members. The CLI on the front-end is the fastest way for the platform team and for scripts. Tools such as OpenTofu use the API directly. From part 4 on you create VMs from code with OpenTofu.

Exercise Classifying

Assign the building blocks oned, KVM, MariaDB, scheduler, libvirt, Sunstone, onemonitord and datastore 0 with the disks of running VMs to the front-end or the nodes.

Solution

oned, MariaDB, the scheduler, Sunstone and onemonitord run on the front-end. KVM and libvirt run on the nodes. Datastore 0 with the disks of running VMs is on the nodes as well, because the transfer driver ssh copies the disks there.

Exercise The path of a VM

Describe in five steps what happens between clicking "Create" in Sunstone and the running VM. Name the building block involved in each step.

Solution
  1. Sunstone sends the request to the API of oned.
  2. oned creates the VM in MariaDB.
  3. The scheduler picks a node with enough capacity.
  4. The TM driver copies the image to the node, the VNM driver sets up the network.
  5. The VMM driver has libvirt start the VM with KVM, and onemonitord reports its state.

Text and figures are licensed under CC BY-SA 4.0 and the code under the MIT License. The sources are on GitHub.