Hi Jeremy!
The way Cherry Picking was designed is really meant for only the initial ticket submission. That's why when your agents take a ticket, the ticket remains assigned to them and any customer responses go to the agent, not the queue. However, by making your agents unavailable to receive tickets in the group you can achieve what you're looking for.
I'll assume you have the Manager or Administrator role, so you should be able to do this, but if not, I'll give you another way of accomplishing this. You can use the Dashboards button in the top right corner of the Management Interface to change your agent's group availability. Click on Dashboards and choose User. Double click on an agent to open their Active Groups window. Make sure the Tickets column is unchecked for any group that should have tickets routed to the queue. Keep in mind, this will need to be done for any agent in the group that should not be automatically assigned ticket responses.
Let me know if this achieves what you're asking for.