Network Management automation for VM

Multiple virtual machines (VMs) created in a single physical platform greatly improve the efficiency of data centers by enabling more work from less hardware. VMs have their lifecycles from VM creation, VM startup to VM deletion. The VMs may also move across the participating virtualization hosts (e.g., the virtualization server or hypervisor). One example is, as the workload on one physical server increases or physical server needs upgrade, VMs can be moved to other available lightweight-workload servers to ensure that service level agreement and response time requirements are met. We call this VM movement or relocation as VM migration. When the workload decreases, the VMs can be moved back, allowing the unused server powered off to save cost and energy. Another example is as one tenant moves, VMs associated with this tenant may also move to the place that is more close to the tenant and provides better user experience (e.g., larger bandwidth with lower latency). We call such movements as VM mobility. VM migration refers to the transfer of a VM image including memory, storage and network connectivity while VM mobility refers to sending data to the moving tenant associated with the VM and emphasizes service non-disruption during a tenant’s movement. This document advocates the distinction between VM mobility and VM migration, both important notions in VM management. The implication is that different signaling or protocols for VM mobility and VM migration might be chosen to automate Network Management for VM Movement, thus possibly reusing the existing protocols or schemes to manage VM migration and VM mobility separately. Unfortunately we sometimes mixed them up or don’t distinct VM migration management from VM mobility management and intend to utilize one common protocol to support both VM migration and VM mobility, which seems to simplify the overall protocol design but it is difficult or impossible to run one such protocol across both VM mobility management system that manages VM mobility and VM management platform that manages VM attributes. This document discusses how VMs are managed, signaling for VM management and argues VMs need management functionality support but can be managed without VM mobility functionality support.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Virtual machines usually operate under the control of a server virtualization software residing on the physical compute server. The server virtualization software is commonly referred to as ‘hypervisor’. The hypervisor is the container of the VM and provides shared compute/memory/storage and network connectivity to each VM that it hosts. Therefore the hypervisor or the virtualized server MUST be aware of VMs that it hosts. However it should not be aware of VMs that it doesn’t host. When VMs hosted in different virtualization servers need to communicate each other, packets from one VM will be forwarded by a virtual switch within the virtualization server or the hypervisor to other VMs on another virtualization server. Since the virtual switch resides within the hypervisor or virtualization server, the rule on VM awareness applied to the hypervisor should apply to virtual switch too. Unlike VM awareness, VM movement awareness is the capability of knowing the location update of the VM. For example, when a VM moves out of the hypervisor and goes to another host, the original hypervisor that hosts the VM is aware of VM movement or location changing but may not be able to keep track of the new location after the VM moves. Therefore one external party that maintains the mapping between the VM’s identity and the VM’s current location is needed which keeps track VM movements.

VM migration refers to VM movement within a virtual environment in response to events,conditions or based on requirements. The events or conditions could be, for example, very high workloads experienced by the VMs or upgrades of the physical server or virtualization server, load balancing between virtualization servers. The requirements could be, for example,low power and low cost requirements or service continuity requirement. When a VM is moved without service disruption, we usually call this VM movement as VM mobility. However it is very difficult to provide transparent VM mobility support since it not only needs to keep connection uninterrupted but also needs to move the whole VM image from one place to another place, which may take a long down time (e.g., more than 400 ms) and can be noticed by the end user. Fortunately, VMs may be migrated without VM mobility support. For example, a server manager or administrator can move a running virtual machine or application between different physical machines without disconnecting the client or application if the client or application supports VM suspending and resuming operation or stopped at the source before the movement and restart at the destination after movement. In some case when VM mobility is really needed, it is recommended that one copy of VM SHOULD be replicated from the source to the destination and during VM replication, thus the VM running on the source should not be affected. When VM replication to the destination completes and the VM on the destination restarts, the VM on the source can be stopped. However how the VM on the destination coordinates with the VM on the source to know whom the latter is communicating with is a challenging issue.

To ensure the quality of applications (e.g., real-time applications) or provide the same service level agreement, the VM’s state(i.e., the network attributes and policies associated with the VM ) should be moved with the VM as well when the VM moves across participating virtualization hosts (e.g., virtualization server or hypervisor). These network attributes associated with VM should be enforced on the physical switch and the virtual switch corresponding to VM to avoid security and access threats. The hypervisor or the virtualization server may maintain the network attributes for each VM. However when VMs are moved from the previous server to the new server, the old server and the new server may have no means to find each other. Therefore one external party called VM network management system (e.g., Cloud Broker) is needed and should get involved to coordinate between the old server and the new server to establish the association between network attributes/policies and the VM’s identity. If the VM management system does not span across data center and the VM is moved between data centers, the VM network management system in one data center may also need to coordinate with VM network management system in another data center.

VM grouping significantly simplifies the administration tasks when managing large numbers of virtual machines, as new VMs are simply added to existing groups. With grouping, similar VMs can be grouped together and assigned with the same networking policies to all members of the group to ensure consistent allocation of resources and security measures to meet service level goals. Members of the group retain the group attributes wherever they are located or move within the virtual environment, providing a basis for dynamic policy assignments. VM groups can be maintained or distributed on the virtualization server or can be maintained on a centralized place, e.g., the VM management platform that manages all the virtualization servers in the data center. VM groups maintained on each virtualization server may change at any time due to various VM operations (e.g., VM adding, VM removing, VM moving). Therefore VM groups need to be synchronized with the central VM management platform. Profiles containing network configurations such as VLAN, traffic shaping and ACLs for VM groups can be automatically synchronized to the central VM management platform as well. This way, consistent network policies can be enforced regardless of the VM’s location.

The ability to identify VMs within the physical hosts is very important. With the ability to identify each VM uniquely, the administrator can apply the same philosophy to VMs as used with physical servers. VLAN and QoS settings can be provisioned and ACL attributes can be set at a VM level with permit and deny actions based on layer 2 to layer 4 information. In the VM environment, a VM is usually identified by MAC or IP address and owned by the tenant. Typically, the tenant not only owns the VM but also owns one or several virtual networks, while one VM only belongs to one tenant. Furthermore, one tenant may own a group of VMs in one virtual network or several groups of VMs distributed in multiple virtual networks. On the request of the tenant, a VM can be added, removed and moved by the virtualization server or the hypervisor. When the VM moves, the network attributes or configuration attributes associated with the VM should also be moved with the VM as well to ensure that the service level agreement and response times are met. These network attributes include access and tunnel policies and (L2 and/or L3) forwarding functions and should be part of VM information. We use Virtual Network Instance ID to represent those network attributes. One virtual network has at least one Virtual Network Instance ID. Therefore each VM should at least include the following information: VM MAC/IP address Virtual Network Instance ID Virtual Network Identifier Tenant ID Additional information optionally includes Group ID and Server ID. Editor Notes: It is more desirable to use TLV or a figure to describe the structure of VM information. The current structure seems not representing the embededness relationship of a VM within a Server or Virtual Network,nor indication of server ID. How many occurrences of each field could be. Also which combination uniquely identifies a VM? We should make it clear here.

VM can be moved within the virtual environment in response to events or conditions. An issue here is who triggers and controls VM movement? In a small scale or large scale data center, the server administrator is usually not aware of VM movement and may respond quickly to system fault or server overload and move a virtual machine or a group of VMs to different physical machines. However it is hard for the server administrator to response to dynamic VM movement and creation since he doesn’t keep track of VM movements. In large scale data centers, the server administrator or VM network administrator may be more hesitated to utilize VM movements because of the time demands of managing the related networking requirements. Therefore automated solutions that safely create and move virtual machines and free VM network or Server administrators from their responsibilities is highly required. The external party (i.e., the VM management platform) is needed to play the role of server administrator and should support keeping track of VM movement and response quickly to dynamic VM creation and movement. When one tenant moves from one place to another place, VM movement associated tenant should be informed to the VM management platform. When one tenant requests to improve the quality of application and shorten the response time, the VM management platform can trigger VM being moved to the server that is closer to the user.

In order to sort out bad VMs, VM monitoring is very important. The VM monitor mechanism keeps track of the availability of VMs and their resource entitlements. It ensures that there is no overloading of resources whereby many service requests cannot be simultaneously fulfilled due to limited resource available. VM monitor is also useful for server administrations and report thstatus information of VMs or VM groups in each server to the VM management system.

We envision the VM management reference model to consist of vServers (virtualization servers) and vCenters (the aforementioned VM management platform). One vCenter may be placed in each data center and manage all the vServers in that data center. Also, one vCenter may be placed in the central place; it either directly manages a large number of vServers distributed in each data center, or indirectly manages the vServers distributed in each data center by coordinating with local vCenter in each datacenter. The vServer connects to NVE Edge in its own data center either directly or via a switched network (typically Ethernet).

,---------. ,' `. |--------- ( VCenter ) | `. ,' | `-+--+---+' | ,---------. ,' `. +--------------( VCenter )---------------+ | `. ,' | | `-+--+---+' | Management Management Management interface interface interface | | | .........|............ .........|............ .........|............ . | . . | . . | . . +--------+ . . +--------+ . . +--------+ . . | | . . | | . . | | . . |vServer1| . . |vServer2| . . |vServerm| . . | | . . | | . . | | . . +--------+ . . +--------+ . . +--------+ . . |VM VM VM| . . |VM VM VM| . . |VM VM VM| . . ---------+ . . .---------+ . . .---------+ . . Tenant End Systems . . Tenant End Systems . . Tenant End Systems . .........|............ ..........|........... .........|............ | | | +---+----+ +----+---+ +---+----+ |NVE Edge| |NVE Edge| |NVE Edge| +--------+ +--------+ +--------+

The vServer is also referred to as “the virtualization server” or hypervisor and creates profile containing network configuration such as VLAN, ACL, QoS and maintains VM configurations for each tenant end system.

The vCenter manages vServer and maintains not only vServer configurations such as vServer name, vServer MAC/IP address but also VM configurations for each tenant end system associated with that vServer.

When a VM is created in the vServer, the vServer may create profile for this VM containing VM identity, ACL and QoS parameter and registers the VM configuration associated with the tenant to the vCenter. Upon receiving such a registration request, vCenter should check if it has already established profile for the corresponding VM: if yes, vCenter should update the existing profile for that VM, otherwise vCenter should create a new profile for the VM.

When a VM is removed from the vServer, the vServer may remove profile for this VM containing VM identity, ACL and QoS parameter and deregisters the VM configuration associated with the tenant to the vCenter. Upon receiving such a deregistration request, vCenter should check if it has already established profile for that VM: if yes, vCenter should remove the existing profile for that VM, otherwise other vCenter should report alert to the vServer.

When a large number of VMs are created in one vServer and share the same networking policy, the vSever may create a profile for a group of these VMs and send a bulk registration request containing the group identifier and associated profile to the vCenter. Upon receiving such a bulk registration request, vCenter should create or update the profile for a group of these VMs.

When a VM is moved from one vServer and added to the current vServer, the current vServer should check with vCenter based on VM identity to see if the profile for that VM already exists and which server maintains that VM configuration. If yes, vCenter should reply to the current vServer with the vServer’s address or name from which the VM is moved.

When vCenter is triggered to move one VM or a group of VMs from one source vServer to another destination vServer, the vCenter should send a VM relocation request to both vServers and updates its profile to indicate the new vServer that maintains the VM configuration for that VM or a group of those VMs. The relocation request will trigger the VM image to be moved from the source vServer to the destination vServer.

One tenant moves between vServers or between data centers and may, as the internet user, want to access applications via the VM without service disruption. In order to achieve this, he can choose to access applications via the same VM without moving the VM when he moves. However, the VM he is using may be far away from where he stays. In order to provide better user experience, the tenant may request vCenter to move VM to the vServer that is more close to where he stays and keeps the service uninterrupted. In such case, the tenant may request vServer that hosts the original VM to interact with the vCenter, chooses one vServer that is closer to him and moves one copy of the VM image to the destination vServer.

When one VM is created, moved, added, removed from the vServer, the VM monitor should be enabled to report the status information and resource availability of that VM to the vCenter. In this case, vCenter can know which server is overloaded, which server is unused or least used.

When one vCenter cannot interact with the vServers distributed in multiple data centers, vCenter in the central place may interact with local vCenter in each location and signaling VMs operation that is applied between vServer and vCenter to the corresponding vServers through local vCenter in each data center.

For sigaling between vServers helps moving VM from one physical server to another, one way is to let vServers interact with the corresponding Network Virtualization Edges (NVE)[ I.D-ietf-nvo3-framework] that are connecting to vServers and copy VM memory to the destination vServer through the tunnel established between NVEs.

Threats may arise when VMs move into a hostile VM environment, e.g., when the VM identity is exploited by adversaries to launch denial of service or Phishing attacks[Phishing]. Further details are to be explored in the future version of this document.

This document has no actions for IANA.