Please don't use RDP as a network speed test. Here are some other tools:
Next, get rid of the E1000 driver. Install the latest VMware Tools package and use vmxnet3 whenever you can.
Try guest to guest on your host and see what you're getting in and out of each guest. Then try guest to your desktop to see what the changes are to help you figure out where your bottleneck is.