Networking many qemu-kvm on one hosting machine or Virtual machines with Virtual networking
VDE - Virtual Distributed Ethernet Create a virtual switch in your machine with software.
QEMU - is a generic and open source machine emulator and visualizer
QEMU-KVM - QEMU with Linux Kernel virtual machine, using hardware in processors that support emulations (very fast).
I have been using Qemu-kvm for the last 2 years and the biggest pain is setting up networking for a group of virtual machines, there is a way to set up a tun and bridge interface for each virtual machine but it is a pain and when you display all of the network machines it is a mess to take in. VDE seems like a way to simplify this networking mess, with VDE you can set up a virtual switch that has one tun to connection and many QEMU-KVM connections to VDE. All of this is not perfect and may require some bridge creation and or FW rules.
Setting up VDE
There is a VDE daemon that must be started to bring up a switch:
vde_switch -tap tap0 -daemon
This daemon must be running to use vde to run qemu, you can bring it up when you start up your machine or with a script as you start up your qemu-kvm machines. Please note that this will create a pipe file in /tmp/ or /var/tmp , depends on installation of vde and parm overrides.
Creating a Linux bridge interface
To get the vm machines to communicate out you can set up routing in linux with a FW SNAT or you can set up a bridge interface over you active network interface. The bridge option is a bit easier and better if you want to connect to your virtual machines from the outside of your host machine. You can create a br0 device through your network start up (Using your normal distro startup) or by hand (using brctl utils) then attach your active network interface to it and the tap0 device and you are done with the network setup, well almost. Note: Distro networking is notorious confusing and worthless, yes people (in most cases people running old distros that google search works well for) do get it to work , but for the rest of us it's not worth the pain since setting it up requires us to understand the start scripts written by some bash hacker of dubious intentions. The doc from brctl is easy to understand thus why bother with anything else.
Using vde with qemu-kde
To put all of this together you need to run qemu-kde under vde and the best way is with this format of a command:
vdeq qemu-kvm -net vde,vlan=0 -net nic,model=virtio,vlan=0,macaddr=52:54:00:00:AA:01 -curses -drive file=/space/images/linux-amd64-2.img,snapshot=off,cache=none,if=virtio,boot=on -smp 2 -m 1G -monitor telnet:127.0.0.1:9221,server,nowait
When running qemu-kvm under vdeq you can add network options IE "-net vde". With this option qemu will connect to a free port on the vde_switch which is now bridged with the br0 device and has free access to the internet/lan . The one trick here is defining the mac address "macaddr=
, this needs to be done for each VM you inter connect to the switch, else all machines will get the same default mac address, the rest of the command is pure qemu and I leave that to the reader to figure out.
, this needs to be done for each VM you inter connect to the switch, else all machines will get the same default mac address, the rest of the command is pure qemu and I leave that to the reader to figure out.
Running multiple network interfaces
One last trick is running qemu-kde with vde and having eth[1-100]. The standard way of doing this is bring up multiple vde_switch daemons and connect to the different UNIX sockets of the multiple switch daemons and set up multiple bridges to connect to which switches and etc . This can get very complex very fast and in most cases it is not needed. The quick way to do this is to use the vlan=X option in vde and use the same vde_switch for each vlan option , a example :
vdeq qemu-kvm -curses -drive file=/space/images/linux-amd64-1.img,cache=none,if=virtio,snapshot=off,boot=on -smp 2 -m 1G -monitor telnet:127.0.0.1:9222,server,nowait -net vde,vlan=0 -net nic,model=virtio,vlan=0,macaddr=52:54:00:00:AA:03 -net vde,vlan=1 -net nic,model=virtio,vlan=1,macaddr=52:54:00:00:AA:04
Note the vlan=0 and vlan=1, this creates a different interface for the qemu-kvm and rides the networks on the same vde_switch, IE you networks can not conflict for this to work address/subnets/netmasks but if that is clean , as it should be, you will be running two/3/4 different networks on the same switch ( There may be security issues with this and thus you may be forced not to do this ) . The vlan= option gives you a easy way to simulate multiple networks between multiple VM's .
Summay
Using vde with qemu-kvm will simplify putting many qemu-kvm machines up on the same machine and make networking very straight forward for the VM's and the hosting machine. For me its a breath of fresh air for networking simple and direct. Anyone with any other networking gems for Linux ? Any other experiences with vde/qemu/kvm ?
