| Subject | Re: [firebird-support] Restore performance (gbak service_mgr) whilst activating indices |
|---|---|
| Author | |
| Post date | 2017-09-08T07:13:09Z |
Hello Thomas,
Thank you for your reply. The VM I was using only has one core allocated to it. It was definitely CPU bound for part of the time (as expected), but there were substantial portions of time where the 1 CPU core was under 5% utilisation and the I/O should have had plenty of headroom.
There is AV but FDB is excluded (at least is supposed to be, I don't have visibility to the exclusion list). Temp files are going to be caught I guess. That said, the relevant AV service was mostly on 0% CPU and wasn't doing much I/O in resource monitor. It seemed to be behaving itself reasonably by AV standards.
The linked ticket is referring to a CPU bound performance limitation. It would be nice if it could use all CPUs for sure, but that isn't the bottleneck in this weird case.
Thanks
Adam
---In firebird-support@yahoogroups.com, <ts@...> wrote :
Thank you for your reply. The VM I was using only has one core allocated to it. It was definitely CPU bound for part of the time (as expected), but there were substantial portions of time where the 1 CPU core was under 5% utilisation and the I/O should have had plenty of headroom.
There is AV but FDB is excluded (at least is supposed to be, I don't have visibility to the exclusion list). Temp files are going to be caught I guess. That said, the relevant AV service was mostly on 0% CPU and wasn't doing much I/O in resource monitor. It seemed to be behaving itself reasonably by AV standards.
The linked ticket is referring to a CPU bound performance limitation. It would be nice if it could use all CPUs for sure, but that isn't the bottleneck in this weird case.
Thanks
Adam
---In firebird-support@yahoogroups.com, <ts@...> wrote :
> Hello Group,The restore process is bound to a single physical core. I've seen something similar in the past, where basically no resource (CPU, disk I/O - throughput + IOPS) being exhausted, thus not bound to available hardware. Do you have an Antivirus solution running affecting the restored Firebird database and/or temporary created files during restore?
>
> Sorry if this double posts, I received an error when Sending.
>
> I was provided with a largish (~27GB) Firebird 2.5 backup file (gbak)
> yesterday. I have been restoring this on Windows 2012R2 running 2.5.6 Classic
> using gbak with the -service_mgr switch. This particular VM only has a single
> CPU core but it is connected to the SAN and isn't doing anything else. I am
> remote desktop'd into the machine.
>
> I have been tracking the restore through several mechanisms:
> * -v switch of gbak
> * How large the file is on disk
> * CPU utilisation (Resource Monitor)
> * Disk I/O (Resource Monitor)
>
> The first 40GB or so of the restored FDB took about an hour or so. The I/O for
> fb_inet_server.exe was consistently in the 30MB/s range (usually about 10 read
> from the fbk and 20 write to the fdb file)
> The next 10 GB took 10 hours. Now obviously when it gets to "activating and
> creating deferred index xyz" it slows down, but I cannot see where the
> bottleneck is. I have seen extended periods of about 1MB/s reads corresponding
> with 5% CPU utilisation whilst activating some of those indices. I copied an
> unrelated 20GB file in the same folder and it happily copied at 50MB/s so there
> is definitely capacity that isn't being used.
>
> I can see that when it activates indices on larger tables it is creating temp
> files and these are at least being written to at 8+ MB/s.
>
> Although I get that activating the indices is going to be slower than restoring
> the data, I was expecting to see it either CPU and/or disk bound at any moment
> in time. (Plenty of headroom for memory and network utilisation is very low).
You may also vote for:
http://tracker.firebirdsql.org/browse/CORE-2992
--
With regards,
Thomas Steinmaurer
http://www.upscene.com
Professional Tools and Services for Firebird
FB TraceManager, IB LogManager, Database Health Check, Tuning etc.