My original CSV was set up as follows...
I'd map "bob" to the username, "Password123" to the password, "Bob Robertson" to the Display As, and ignore the reply address. This would import the username/password but leave the Display As blank.
I then re-did the CSV to just have the username,password,displayas columns...
username,password,displayas
bob,Password123,Bob Robertson
And would map to the matching fields in the drop box. This seems to work as now the Display Name shows in the account details (but still doesn't show in the full list of users). This should be fine for what I'm doing. If the end users want to put in a different reply address, they can do some themselves (will send out a message to all new users to check out their profiles and fill in any missing data).