The article by Ethan44 eludes to the "feature", however the following great article by Frank Denneman may explain better the SIOC behaviour on VMs with 5.0 and 5.1 :-
I have gone through this pain also. Basically, it is by design ! The v5.0 SIOC is meant to "throttle" VM I/O to reduce high I/O on the Datastores. I suppose the best advice is "if it aint' boke, don't fix it" !
Options :-
1. Disable SIOC. If there has been no issue in your environment, then no issues.
2. Upgrade to v5.1 (Frank recommends it) and look at Storage DRS. However, you need to investigate how this impacts your environment as a whole as I know others have had issues.
3. If you want to reduce I/O on your Datastores, investigate any Storage specific solutions with your NetApp.
Hope this helps.