I dug out my old phone, running Cyanogenmod 7 and with 1.7 of the VMware client, and tried it. It was similar to my Nexus 4 where I could sign on and seemingly produce non-alphanumerics, but my password still fails.
I restored 1.6.1 from backup and it didn't help. On my computer I connected, opened a window, the logged in via the Android client and tried putting a ? on the command line--nothing. Return worked so I know it was typing.
This sounds like a bug that was never fixed. I tried reporting it from the Google Play store, I was referred to here. Is there some place I can report a bug [where it will be read and acted on]?