Subject Re: [Firebird-Java] Are there plans for streaming backup support?
Author Ivan Arabadzhiev
Hi again,
Just a quick status update. I'm now almost happy with how the code looks but I have come to a spot where I could use a second opinion on InputStream - I chased my restore issues down to the fact that my OpenJDK (1.8.0_72, OpenSUSE x64) would start rereading the final block over and over and over again if the size the input file happens to be a multitude of the buffer. In that same environment, .available() behaves as expected so I moved to that. I can (and will) test on every JDK I can get my hands on within a few days but I'm open to ideas (or a preferred way to do things). Also not sure what would happen with other streams (GZIPInputStream also works), which may or may not make sense - other compression streams, other sockets and so on.

As pushed, it has been tested to backup, restore (with and without verbose output), with a small database on 
Linux x64, LI-V2.5.4.26856
Windows x64, WI-V2.5.5.26952
Windows x86, WI-V2.5.5.26952
Linux armv7, LI-V2.5.2.26540

I'll try to upload a .jar later to make it easier for anyone interested to test for possible client-side issues with some OS/JVM combo.

2016-02-16 14:37 GMT+02:00 Ivan Arabadzhiev <intelrullz@...>:
So far so good but it is terribly slow so I reworked the backing up to enable a bigger buffer - will push as soon as I find out what I broke in restore ...

I'm thinking multi-file split can be done simply by closing fileN at X bytes and opening fileN+1. Might not be 100% what gbak does but the API shouldn't really care if I have a single file on disk or I store every byte on a different media. That being said, I've never needed the functionality myself and don't know if people actually use it often enough to need a default implementation.

As for the change - perhaps I should've given to begin with. addArgument(isc_info_svc_line, bufferToBeAdded); throws an exception otherwise.

2016-02-16 14:25 GMT+02:00 Mark Rotteveel mark@... [Firebird-Java] <>:

On Tue, 16 Feb 2016 03:42:35 +0200, "Ivan Arabadzhiev intelrullz@...
[Firebird-Java]" <> wrote:
> Hi again,
> My insomnia is getting the better of me and I now have a proof of
> at
> . Haven't made a PR because it doesn't feel good enough yet, but it does
> backup and restore successfully at x86_64 linux with fb 2.5.4(which
> it should be less work than I initially thought).

I am going to look at it in more detail later this week, but here are some
quick notes

> I have a 25GB db to play with after sunrise, and I will try to test on
> 32/64 Windows/Linux this week but :
> 1 Are we ok with the 'hack' I'm using for backing up with by temporarily
> changing the logger? Now that I have some idea what exactly is
happening, I
> think doing a backup like that should be ok even with my earlier idea
> currently released Jaybird.

I think for now it is ok. That said I'm not entirely happy with how the
service managers work in Jaybird (that whole logger feels a bit clunky),
and in a future version of Jaybird I wanted to look at changing this (but
no firm plans yet.

> 2 Are we ok with some code being duplicated in the new class? I
> suppose FBServiceManager could be reworked so all the handling remains
> it, but I'm not really sure it's worth it - right now there is just the
> file and one other changed line.

Well, and FBStreamingBackupManager can probably share
a common super-class (or one might extend the other), but I suggest you
first get it working ok, then we can think about extracting the

> 3 Currently the implementation expects InputStream/OutputStream, which
> means no multi-fbk operations. I'm thinking that part could be left to
> users to handle, but providing some default stream-like wrappers won't
> that much more work to do, if it seems necessary.

I'm not actually sure if split backupfiles is supported with streamed
backups/restores. In any case, Java already has, for writing there is no equivalent. I don't
know if split backupfiles contain additional headers, etc, and I don't
really see a need to support split backup files for this.

> 4 I'm not sure if the change in ByteArrayArgument can cause any problems
> but if it does - how should I add the byte[] parameter
> for isc_info_svc_stdin?

I'm not sure what you mean could you link to the specific code and
describe this in more detail?

> 5 Any other thoughts on the generic idea?