4
Web Services Migrations - sequential vs. simultaneous
Question asked by kevind - 7/10/2015 at 8:43 AM
Unanswered
Hello.
 
Using web services svcUserAdmin -> QueueMailBoxMigration to move mailboxes to SmarterMail.  It takes forever because the migration queue goes sequentially, one at a time.
 
For example, if we queue 50 mailbox migrations in web services, it takes 20 hours or more.  If we enter them manually through the web interface, it only takes a couple hours because they all run simultaneously.
 
Any suggestions for speeding this up without hiring a bunch of data entry people? Looking for a way to create multiple queues or process 4-5 migrations at a time.
 
Thanks,
Kevin

5 Replies

Reply to Thread
0
kevind Replied
First of all, thank you very much for incorporating this request into 14.1.5675.
 
Unfortunately, it created another problem. Submitting 20 or more migrations brings the server to its knees since all migrations run simultaneously.  This affects all users as webmail stops responding, the spool can't keep up, etc.
 
Looking to find some balance.  Ideas:
  • Limit the number of simultaneous threads to 10 or so. Leave the rest in queue.
  • Add a parameter in web services for queue #.  Then I could submit 5 for queue #1, 5 for queue #2, etc.
  • Create queues based on time. Example: all migrations submitted at 6:01am go into queue #1, all submitted at 6:02am go into queue #2, etc.
  • Give each web services userID a separate queue. I could submit migrations to web services using multiple admin accounts each having their own queue.
  • Watch CPU and don't start any migrations when server is busy???
  • Any other idea you can offer to balance the load between 20+ hours (single queue) or crushing the server (all simultaneous).
I realize that you probably don't want to spend a lot of dev time here, so any small adjustment you can make would be appreciated.
 
Thanks,
Kevin
0
Matt Petty Replied
Employee Post
They shouldn't be running simultaneously there is code in place to prevent that, but I can look into it further.
 
Are you submitting them one by one or all at once?
Matt Petty Software Developer SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
kevind Replied
Matt, the functionality was changed in the latest build to run simultaneously.

I don't know how to answer '1x1 or all at once' because there's only one way to do it AFAIK. We submit them individually, one after another, but the entire submission is done in less than 30 seconds.

Previously they went into a queue and took forever. Now they all process simultaneously and crash the server. We're hoping for something in between.

Thanks.
0
Matt Petty Replied
Employee Post
Ahh, I know exactly why it's doing it then. Easyish fix. May eventually add a field for it but does 3 at a time sound good?
Matt Petty Software Developer SmarterTools Inc. (877) 357-6278 www.smartertools.com
0
kevind Replied
How about 5 at a time? That would be a good balance that should work for just about everyone. On a fast server, we can get 10-20 going without a problem. Over 20 it really hits the wall. Thanks!!!

Reply to Thread