[ad_1]
If your OpenStack hosted virtual instances need network connectivity you’re going to have to create a network. There are multiple kinds of networks and in order to make the right choice you will need to understand at least two very important network attributes: ‘router:external’ and ‘shared.’ Unless you know what these attributes and their combinations mean, it will be difficult to make the optimal network choice.
In this article, we’ll explain the four types of networks these attributes dictate and how to configure them and briefly explain some typical use cases. Some of these exercises apply to tenants and others are for administrators only. Examples are given to work through this exercise on a KVM-hosted Red Hat or RDO all-in-one OpenStack instance. This has been tested on Red Hat OpenStack Platform 8, 9, 10 and RDO Mitaka and Newton.
The four types explained
The table below gives an overview of the types of networks we’re going to create and their names.
| router-external | shared | network type | name in this exercise | description |
| false | false | vxlan | private1-demo | Typical tenant network, which is only usable by members of the tenant. Typically an overlay (vxlan, gre). |
| false | true | vxlan | admin1-shared | This can be shared by multiple tenants with RBAC on who can use it. Typically an overlay (vxlan, gre). |
| true | false | flat | external1 | Typical external network, where the scope is all tenants. Can only be created by administrators. Tenants connect their router for external access. Typically a ‘flat’ or ‘vlan’ network. |
| true | true | flat | external2-shared | Scope is all tenants. Can only be created by administrators. Tenants can connect directly to it. Typically known as a ‘provider’ network and is ‘flat’ or ‘vlan.’ |
Logical network topology
OpenStack all-in-one build
An all-in-one instance is handy for understanding new concepts where separate roles are not required. Troubleshooting and configuration are considerably simplified, and that is what we’re going to use for our examples. If you have a Red Hat subscription, you can follow these instructions. You can also sign up for a 60 day Red Hat OpenStack Platform evaluation.
For an RDO instance, go here for a quick installation guide, and be sure to enable the additional repositories on this page.
When your packstack build is complete, log in as root and run the following commands to verify your OpenStack deployment is working:
. ./keystonerc_admin openstack catalog list neutron agent-list nova service-list nova boot --flavor m1.tiny --image cirros --nic net-name=private admin01Once you have established the health of OpenStack, delete the virtual instance you just created along with the ‘Public’ and ‘Private’ networks (and their subnets) and ‘router1’ created by the packstack installation.
KVM host configuration
The KVM host for this exercise will have three networks:
| Linux bridge name | KVM network name, VM nic | Purpose |
| virbr0 | default, eth0 | Direct access the OpenStack virtual host. |
| virbr1 | external1, eth1 | External access for instances, either through SNAT or floating IPs. |
| virbr2 | external2, eth2 | Eirect external access. |
Your KVM host probably already comes with a default network on eth0 that with bridge virbr0. We’ll define and then create the other two external networks. Be sure to do this as root:
cat > /tmp/external1.xml << EOF <network> <name>external1</name> <forward mode='nat'> <nat> <port start='1024' end='65535'/> </nat> </forward> <bridge name='virbr1' stp='on' delay='0'/> <mac address='52:54:00:73:a0:8e'/> <ip address='172.16.0.1' netmask='255.255.255.192'> </ip> </network> EOF cat > /tmp/external2.xml << EOF <network> <name>external2</name> <forward mode='nat'> <nat> <port start='1024' end='65535'/> </nat> </forward> <bridge name='virbr2' stp='on' delay='0'/> <mac address='52:54:00:61:98:8c'/> <ip address='172.16.0.65' netmask='255.255.255.192'> </ip> </network> EOFNow create these networks and start them:
virsh net-define /tmp/external1.xml; virsh net-define /tmp/external2.xml virsh net-autostart external1; virsh net-autostart external2 virsh net-start external1; virsh net-start external2Now add a VNIC to the OpenStack all-in-one instance for each external network. Do this from the KVM host:
dom=<your Red Hat Enterprise Linux or RDO instance name from ‘virsh list’> virsh attach-interface --domain $dom --type network --source external1 --model virtio --config --live virsh attach-interface --domain $dom --type network --source external2 --model virtio --config --liveVerify the KVM host
$ virsh net-list Name State Autostart Persistent ---------------------------------------------------------- default active yes yes external1 active yes yes external2 active yes yes $ brctl show bridge name bridge id STP enabled interfaces virbr0 8000.525400ce5983 yes virbr0-nic virbr1 8000.52540073a08e yes virbr1-nic virbr2 8000.52540061988c yes virbr2-nic $ route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.16.0.0 0.0.0.0 255.255.255.192 U 0 0 0 virbr1 172.16.0.64 0.0.0.0 255.255.255.192 U 0 0 0 virbr2 192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0OpenStack all-in-one configuration
We need to make some changes to the packstack delivered Neutron configuration to remove the single Linux bridge ‘br-ex’ and re-add new bridges br-ex and br-ex2 and the flat networks we’ll attach them to:
yum install crudini -y crudini --set /etc/neutron/l3_agent.ini DEFAULT external_network_bridge #blank this crudini --set /etc/neutron/l3_agent.ini DEFAULT gateway_external_network_id #blank this crudini --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 type_drivers vxlan,flat crudini --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 network_vlan_ranges physnet1,physnet2 crudini --set /etc/neutron/plugins/ml2/openvswitch_agent.ini ovs bridge_mappings physnet1:br-ex,physnet2:br-ex2Note that it is up to you what you want to call your network_vlan_ranges when using ‘flat’ networks. These will be the ‘–provider:physical_network’ names when you create your external networks. External bridge names (br-ex*) are also your choice.
Now create the interface configurations for Open vSwitch:
cat > /etc/sysconfig/network-scripts/ifcfg-br-ex << EOF DEVICE=br-ex ONBOOT=yes HOTPLUG=no NM_CONTROLLED=no PEERDNS=no DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=172.16.0.1 NETMASK=255.255.255.192 EOF cat > /etc/sysconfig/network-scripts/ifcfg-eth1 << EOF DEVICE=eth1 ONBOOT=yes HOTPLUG=no NM_CONTROLLED=no PEERDNS=no DEVICETYPE=ovs TYPE=OVSPort OVS_BRIDGE=br-ex BOOTPROTO=none EOF cat > /etc/sysconfig/network-scripts/ifcfg-br-ex2 << EOF DEVICE=br-ex2 ONBOOT=yes HOTPLUG=no NM_CONTROLLED=no PEERDNS=no DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=172.16.0.65 NETMASK=255.255.255.192 EOF cat > /etc/sysconfig/network-scripts/ifcfg-eth2 << EOF DEVICE=eth2 ONBOOT=yes HOTPLUG=no NM_CONTROLLED=no PEERDNS=no DEVICETYPE=ovs TYPE=OVSPort OVS_BRIDGE=br-ex2 BOOTPROTO=none EOFVerify Open vSwitch external bridge
Once you have completed those configuration steps you will want to reboot your virtual host. The bridges created should look like the following after the reboot:
ovs-vsctl list-ports br-ex eth1 phy-br-ex ovs-vsctl list-ports br-ex2 eth2 phy-br-ex2Neutron networks, subnets, and routers
Finally, we’re ready to get down to business and create the networks we’ve been talking about. The following series of commands will create networks, subnets, routers, security group rules, instances and floating IPs. When this is complete you’ll have the four different types of networks ready for use.
External networks
The following series creates our external networks. Note that both networks use ‘router:external=true’ and they are both ‘network_type=flat’, but each network is on a different physical_network and the 2nd one has the ‘–shared’ attribute:
. ~/keystonerc_admin neutron net-create external1 --router:external=True --provider:network_type=flat --provider:physical_network=physnet1 neutron subnet-create --name external1-subnet --disable-dhcp external1 172.16.0.0/26 --allocation-pool start=172.16.0.2,end=172.16.0.62 neutron net-create external2-shared --shared --router:external=True --provider:network_type=flat --provider:physical_network=physnet2 neutron subnet-create external2-shared --name external2-shared-subnet --allocation-pool start=172.16.0.66,end=172.16.0.126 --gateway 172.16.0.65 172.16.0.64/26Tenant network and router
Tenant networks are where you can create your own private rfc 1918 address spaces. Tenants can provision any combination of networks, subnets, and routers to meet their needs. They can define their own broadcast domains and determine which instances are externally accessible through floating IP addresses.
The following creates a private or ‘tenant’ network and subnet. It also creates a router to attach it to external network external1 Note that we are switching to the ‘demo’ project for these networks:
. ~/keystonerc_demo neutron net-create private1-demo neutron subnet-create private1-demo 10.0.1.0/24 --name private1-demo-subnetNote that this network was created with these by default:
neutron net-show -c 'router:external' -c shared private1-demo +-----------------+-------+ | Field | Value | +-----------------+-------+ | router:external | False | | shared | False | +-----------------+-------+ neutron router-create router1-demo neutron router-gateway-set router1-demo external1 neutron router-interface-add router1-demo private1-demo-subnetThe router gateways should be pingable. Perform this command and ping the gateway IPs it returns:
neutron router-list -c external_gateway_info | grep -o "172.16.0.[0-9]*"Security group rules, virtual instances, and floating IPs
Before creating virtual instances, let’s open the ports for ICMP and SSH in the default security group so we will be able to connect to them:
. ~/keystonerc_demo neutron security-group-rule-create --direction ingress --ethertype IPv4 --protocol tcp --port-range-min 22 --port-range-max 22 default neutron security-group-rule-create --direction ingress --ethertype IPv4 --protocol icmp default nova boot --flavor m1.tiny --image cirros --nic net-id=$(neutron net-show private1-demo -c id -f value) demo01-private1 nova floating-ip-create external1 nova floating-ip-associate demo01-private1 $(neutron floatingip-list -c floating_ip_address -f value) nova list #the floating IP should be pingable and you should be able to ssh to this instance through it.What did you just do? You created external networks and an instance on a private tenant network with the following characteristics:
Outbound access: This instance can access the external1 network through its router (router1-demo). Its source address will be ‘SNATed’ to that of its floating IP address. That ‘SNATing’ takes place in the router namespace. If this instance did not have a floating IP, however, it would still have external1 network access. The router, in this case, will SNAT the source address to be that of its own public IP address.
Inbound access: This instance is known to the outside world by it’s floating IP address, which router1-demo will respond to ARP requests for. That IP is DNAT’ed to that of its local IP address in the router namespace.
Note: to see these NATing rules dump them from the router namespace:
. ~/keystonerc_demo ip netns exec qrouter-$(neutron router-show router1-demo -c id -f value) iptables -S -t natTenant network access: This instance is in the same broadcast domain as any other instance on the private1-demo network. Because the vxlan overlay network abstracts the underlying physical network, this broadcast domain extends across compute nodes where these instances are hosted.
View the network topology from the Horizon console
. ~/keystonerc_admin echo $OS_AUTH_URL echo $OS_USERNAME echo $OS_PASSWORDPoint your browser at the URL above. Log in with the OS_USERNAME and OS_PASSWORD from ~/keystonerc_admin. From the ‘Project’ tab select the ‘demo’ project (you will need to add admin as a member of the demo project first). Now select ‘Network/Network Topology.’ Here you’ll see the networks we created; be sure to try the two views – topology and graph. Keep this view in front of you as we try to understand what we are doing.
Shared tenant network
A tenant network can be created with the ‘–shared’ attribute which allows other tenants to attach their own instances to it. By default, only the admin tenant can create a shared tenant network but it is possible for other tenants to do so with RBAC. See Role-Based Access Control for networks. This type of network can be useful when two or more projects have instances that benefit from being in the same broadcast domain, thus bypassing the need to share with floating IP addresses.
. ~/keystonerc_admin nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0 #allow icmp through default security group nova secgroup-add-rule default tcp 22 22 0.0.0.0/0 #allow ssh through default security group neutron net-create --shared --router:external=false admin1-shared neutron subnet-create admin1-shared 10.0.9.0/24 --name admin1-shared-subnetNow create an instance on the shared tenant network from the admin project:
nova boot --flavor m1.tiny --image cirros --nic net-id=$(neutron net-show admin1-shared -c id -f value) admin01-admin1-sharedNow create an instance on the shared tenant network from the demo project:
. ~/keystonerc_demo nova boot --flavor m1.tiny --image cirros --nic net-id=$(neutron net-show admin1-shared -c id -f value) demo02-admin1-sharedWhat did you just do? If you log into either of these instances using the instance/console option from Horizon you will notice that they are on the same subnet, even though they are in different projects. This type of network can be useful for sharing instances between projects; the only other choice for doing so is with floating IPs. The shared network (admin1-shared) can be seen and joined by any tenant by default. Note that there is no external network access from this network unless you add a router and set the gateway to external1.
Shared external network
On a shared, external network tenants can connect their instances directly to an external network and get a floating IP address assigned automatically. We’ll use the external2-shared network already created above.
. ~/keystonerc_demo nova boot --flavor m1.tiny --image cirros --nic net-name=external2-shared demo03-external2-shared nova console-log demo03-external2-sharedYou will probably notice from the console log output that the instance failed to contact the metadata service:
checking http://169.254.169.254/2009-04-04/instance-id failed 1/20: up 3.03. request failed failed 2/20: up 8.09. request failed failed 3/20: up 11.09. request failed etc…You may be able to ping the instance, but if you want instances on external2-shared to have access to the metadata service then in /etc/neutron/dhcp_agent.ini set:
crudini --set /etc/neutron/dhcp_agent.ini DEFAULT enable_isolated_metadata True systemctl restart neutron-dhcp-agentThis is necessary because you are not connecting to this network through a router, which is normally where metadata service access comes from. Now reboot the instance and check the console log again:
nova reboot demo03-external2-shared nova console-log dem03-external2-sharedNow you should see this in the log:
checking http://169.254.169.254/2009-04-04/instance-id successful after 1/20 tries: up 2.04. iid=i-000000efWhat did we just do? By creating an instance on this shared, external network you bypass the need for using a router for external access. Additionally, you get what amounts to a floating IP by default. Lastly, the broadcast domain is the external subnet. So, any instance on this subnet is in the same broadcast domain regardless of the tenant, the same characteristic of the shared tenant networks we looked at previously. If a tenant is accessing instances by floating IPs only on their private network then they might be better off with a shared, external network.
Use the right network attributes for the right job
By understanding the characteristics of these four types of networks you’ll be off to a good start in harnessing the flexibility of Neutron. Floating IP’s can get to be a scarce IPv4 resource, for example. As a workaround, you could rely on NAT, or use shared private networks to save yourself some headaches. By understanding your use-cases you’ll be able to select the right network type for yourself and/or your customers.
This article was originally published on the Red Hat blog as a part of a series from Red Hat Technical Account Managers (TAM).
[ad_2]
Source link