Subject Re: [Firebird-Java] Are there plans for streaming backup support?
Author Ivan Arabadzhiev
Post date 2016-02-16T12:37:55Z
Hi,
So far so good but it is terribly slow so I reworked the backing up to enable a bigger buffer - will push as soon as I find out what I broke in restore ...

I'm thinking multi-file split can be done simply by closing fileN at X bytes and opening fileN+1. Might not be 100% what gbak does but the API shouldn't really care if I have a single file on disk or I store every byte on a different media. That being said, I've never needed the functionality myself and don't know if people actually use it often enough to need a default implementation.

As for the change - perhaps I should've given https://github.com/ls4f/jaybird/commit/d62a2417110a3ba2477f8139dfc6c83d5db1b226 to begin with. addArgument(isc_info_svc_line, bufferToBeAdded); throws an exception otherwise.

2016-02-16 14:25 GMT+02:00 Mark Rotteveel mark@... [Firebird-Java] <Firebird-Java@yahoogroups.com>:
 

On Tue, 16 Feb 2016 03:42:35 +0200, "Ivan Arabadzhiev intelrullz@...
[Firebird-Java]" <Firebird-Java@yahoogroups.com> wrote:
> Hi again,
> My insomnia is getting the better of me and I now have a proof of
concept
> at
>
https://github.com/ls4f/jaybird/blob/streaming-br/src/main/org/firebirdsql/management/FBStreamingBackupManager.java
> . Haven't made a PR because it doesn't feel good enough yet, but it does
> backup and restore successfully at x86_64 linux with fb 2.5.4(which
means
> it should be less work than I initially thought).

 I am going to look at it in more detail later this week, but here are some
quick notes

> I have a 25GB db to play with after sunrise, and I will try to test on
> 32/64 Windows/Linux this week but :
> 1 Are we ok with the 'hack' I'm using for backing up with by temporarily
> changing the logger? Now that I have some idea what exactly is
happening, I
> think doing a backup like that should be ok even with my earlier idea
and
> currently released Jaybird.

 I think for now it is ok. That said I'm not entirely happy with how the
service managers work in Jaybird (that whole logger feels a bit clunky),
and in a future version of Jaybird I wanted to look at changing this (but
no firm plans yet.

> 2 Are we ok with some code being duplicated in the new class? I
> suppose FBServiceManager could be reworked so all the handling remains
in
> it, but I'm not really sure it's worth it - right now there is just the
new
> file and one other changed line.

 Well, FBBackupManager.java and FBStreamingBackupManager can probably share
a common super-class (or one might extend the other), but I suggest you
first get it working ok, then we can think about extracting the
commonalities.

> 3 Currently the implementation expects InputStream/OutputStream, which
> means no multi-fbk operations. I'm thinking that part could be left to
the
> users to handle, but providing some default stream-like wrappers won't
be
> that much more work to do, if it seems necessary.

 I'm not actually sure if split backupfiles is supported with streamed
backups/restores. In any case, Java already has
java.io.SequenceInputStream, for writing there is no equivalent. I don't
know if split backupfiles contain additional headers, etc, and I don't
really see a need to support split backup files for this.

> 4 I'm not sure if the change in ByteArrayArgument can cause any problems
> but if it does - how should I add the byte[] parameter
> for isc_info_svc_stdin?

 I'm not sure what you mean could you link to the specific code and
describe this in more detail?

> 5 Any other thoughts on the generic idea?

 Mark