I have been discussing external network provisioning with a small customer, who wants to utilise their network in a particular way to connect their external vCloud networks to the Internet. I'm looking for thoughts / opinions on their design.
At the physical layer, there is a DC network tail that connects directly to a firewall. That then connects to a pair of access switches, to which the cloud compute nodes have their data connections (management, storage, vMotion etc all go to a different switch stack). The data connections are split between each switch, 2 x pNics per switch.
From a vCloud perspective, the hosts use vLAN backed network pools, with the physical switch ports all tagged with the vLAN IDs of the assigned ranges. 'The Internet' is a tagged port group on a DvSwitch, and is defined as a single external /24 network for all the Organisations to use. The concept for the network is for the 'Internet' vLAN to be a transit network between vCloud organisations and the physical firewall. The Organisation vLANs only reside on the switches and the uplink to the firewall. The only vLAN on the firewall is the 'Internet' transit vLAN where it's gateway resides.
A double NAT scenario presents itself. Because the customer wants to be able to terminate seperatie VPNs to Organisations, they want Edge devices to connect routed Organisation networks to the 'Internet' external vCD network, then for the individual Edge devices to connect to external public IPs through NATs in the physical firewall to the Internet. For example (not real IPs):
Org network: 192.168.200.0/24 (vLAN 200)
External network: 192.168.250.0/24 (vLAN 250)
Organisation network GW: 192.168.200.1
Edge external sub-allocated IP: 192.168.250.70
NAT 1 (Edge): 192.168.200.0/24 to 192.168.250.70
NAT 2 (firewall): 192.168.250.70 to <public IP>
So, the idea the customer wants is traffic from the routed organisation goes through the edge device, is translated by the edge to the external network, then immediately goes through the second NAT across the physical firewall to the Internet.
Except, something is not quite right. VMs inside the organisation can ping both the internal and external interfaces of the edge device (both VSE and NAT), and even the GW of the external network, but they cannot get out to the Internet. Interestingly, when connecting the VMs to a directly connected network (I.e. they get an IP from the external network pool) connectivity is fine and works well.
(For those wondering, I have allocated the first sub-allocated IP to the edge device in vShield Manager in vSphere, then created the edge NAT as seems to be required in vCD 5.1).
My expectation was that all Organisation traffic would transit the edge devices and use the external network gateway to access the Internet, and the second physical NAT would only be used for terminating a VPN on the edge endpoint, but the customer disagrees.
any thoughts on how we might get this working? Any thoughts gratefully received!
Cheers.
Jeremy.