| Subject | Re: Fastest method of moving records between databases? |
|---|---|
| Author | cantak3 |
| Post date | 2011-05-19T19:43:45Z |
I made a couple more adjustments.
- I disabled all indices while the copy takes place (except for PK generated indices of course).
- I optimized some of my parallel threading.
I can now run many tables at once without too much of a slowdown. In other words, I start my biggest table and am getting 115 records per second. I then start 25 separate threads all processing another table, and still get roughly 80 records per second on each. When one finishes it moves on to the next table. 20 minutes later, all tables have completed except the largest table which goes back to processing 115 records per second.
This single table continues for two more hours. It's far better than before, and it's workable at this rate. I just wish I could get decent throughput on a single table.
- I disabled all indices while the copy takes place (except for PK generated indices of course).
- I optimized some of my parallel threading.
I can now run many tables at once without too much of a slowdown. In other words, I start my biggest table and am getting 115 records per second. I then start 25 separate threads all processing another table, and still get roughly 80 records per second on each. When one finishes it moves on to the next table. 20 minutes later, all tables have completed except the largest table which goes back to processing 115 records per second.
This single table continues for two more hours. It's far better than before, and it's workable at this rate. I just wish I could get decent throughput on a single table.