Subject Re: TIBDataSet and Filter?
Author Jason Wharton
> That all sounds really good (in theory). :)
>
> But first of all IBO cost money and IBX not :)

That's a pretty tender spot you just punched!

(Speaking in general) Let's not promote the awful tide of crap replacing
much higher quality options because they are free. Market niches are being
destroyed right and left because of this. IBO has barely survived in spite
of the advent of IBX. If everyone used your logic, what will be left when
every commercial endeavor to produce high quality software has been
undermined and destroyed by "free beer" software? That's not a pretty
picture in my mind. The programming profession should not be made into a
commodity item! That is what "free beer" software is doing. There's a better
way...

Have you read the trustware terms IBO is distributed under? In short...
It only costs for commercial use and if you don't have the money to afford
IBO comfortably it is free until you do have the money. Trustware is about
generating revenue surrounding good products to keep them professional
quality. It should be easy to see the difference between IBO and IBX in this
regard if you look beneath the crust. Trustware is the best of both
commercial and open source models.

IBX only survives because Jeff Overcash is willing to surrender his precious
free time to rescue it. With open source, generally you get what you pay for
or someone like Jeff is practically selling his soul for a thing of naught.
Or, employers are having their company time squandered from them. I suppose
people rack up some nerd kudos but that's about it. In Jeff's case I really
don't think he is being fair to himself and certainly Borland shouldn't be
trained into the mindset that the world is full of people like Jeff who are
willing to prostrate themselves for Borland's commercial benefit.

Open source is a good thing when a company wants to responsibly dispose of a
software product and not support it any longer. That is really the only case
where I see it having any real benefit. Perhaps there are some benefits in
the open standards areas too for things that truly are a commodity in the
industry. Like an OS, network protocols, databases, security algorithms,
etc. But, niche things should not be "free beer"...

I suppose that is enough of a rant there.... This is all just my personal
and very biased opinions. I intend no offense on anyone's part and am just
frankly speaking my mind.

> The main reason I was looking to filtering is that I am writing one thin
> client application which is going to work with Dial-Up connection so code
> need to be optimized the best way possible.

If that is so, using OnFilterRecord in IBX is going to hurt you badly.

> In few words, the task I am trying to resolve now is following:
> I have one query and Data Grid and I am selecting some fields like Unit
and
> Name from remote database which's already takes 5-7 seconds. Then they
need
> to be able to search by Unit or/and Name and be able to filter records by
> some criteria too. So I was thinking instead of using difference select
> statements like
> select Unit,Name from Table where Unit = condition and/or Name = condition
> which's will take 5-8 seconds each time and not acceptable generally, to
use
> Filtering. Because I was thinking that probably Filtering was implemented
> the way that if I already call select and fetched all records I will not
> need to fetch data from remote database anymore because it already kind of
> in memory and I will just filter them depends on conditions. But my
testing
> show me that each time I am changing Filtered from FALSE to TRUE it
fetches
> all records again and it takes 5-8 seconds like select statement which
> already not acceptable.
>
> I am almost sure that it is not a right way to do that, and probably Jeff
> needs to find better solution to that. Or may be I am not understand
> conception enough well or doing something wrong. :)
>
> So do you know if IBO worked in the way I explained above or it is doing
the
> same things IBX does.

What IBO will allow you to do with filtering mechanisms is pretty powerful.
First of all, what you put in the Filter property is resolved to SQL so that
on the first level you are determining what records are in the buffer. This
is how you trim down what records cross the wire. Then, you use the
OnFilterRecord event in IBO to suppress records in the buffer from being
seen by the user. If the filter criteria changes you can call
RefreshFilteredRows and it will reset the filtered flags of the records in
the buffer and intelligently refresh your datasets to show the newly visible
records. Thus, you can work with the records you have in memory without
having to fetch records from the server again. IBO also properly integrates
the Filtered property and the FindXXXX methods that accompany them.

> And in general how can I get some real information on IBO vs. IBX
> performance issue.

If you simply do a client buffer saturation test they are going to be
sufficiently equivalent that there is nothing worth distinguishing. In
short, for tiny datasets IBX is slightly faster but as datasets grow in size
its performance begins to sorely degrade. IBO's performance is far more
leveled and remains excellent regardless of dataset size. Also, IBO has been
designed with optimal performance for low bandwidth remote connections and
almost everyone who posts benchmarks were simply using local IB. So, as a
general rule you just have to see it for yourself to believe it.

Also, many dataset operations IBX either doesn't support (like Filter) or
when you attempt to use them creatively it bogs down or chokes up. IBO
totally supports 100% of what TDataset implements and even goes the extra
mile to implement every nice feature you used with TDatabase, TQuery and
TTable via the BDE. Only with IBO the quirks and traps are not there and it
is a lot more flexible about what you can mix and match feature-wise. For
example, you can mix cached updates with refreshes and it will maintain the
record pointer accurately and merge new records with the cached updates.
With IBX you can't even refresh to start with but if you do the close and
open you lose your update buffers. Same is true with master/detail That's
just one example of many.

IBX redefines a lot of standard dataset behaviors into its own mutated
versions. e.g. Refresh refreshes only a single row instead of the dataset.
RecordCount is a count of fetched records in the buffer instead of an
accurate count of records in the dataset. Bookmarks are pretty much
worthless since they are just arbitrary integers. IBX doesn't know how to
deal with inserts because its buffering strategy is so rigid. They are
tacked onto the end of the buffer regardless of where the record pointer
was. And, don't get me started on transactions. IBX's transaction model is a
total mess. Let's just leave it at that.

Mostly IBO kills IBX in performance because there is a whole lot more
intelligence built into IBO that allows you to do things not possible in IBX
without writing hordes of complex code in your applications. So, you either
have the choice of rewriting all the complex stuff in your application or
you can use a solid and reliable engine that is 5 years old that thousands
of people are using and that is virtually void of bugs. Not to mention there
isn't a major rewrite of IBO scheduled in its near future as there is for
IBX if, I heard correctly.

The most convincing argument not to use IBO comes from Craig Stuntz. He
said, and I am paraphrasing, "Using something like IBO will get you used to
having all these features and then when you need to use something else you
are in trouble because what you did with IBO isn't going to work well in
other places." This is very true because I don't know of any other data
access architecture/implementation that is as complete and refined as IBO
is. Its no wonder ever since Delphi Informant introduced the Best Database
Connectivity category in their awards IBO stands undefeated (for two
years<g>).

Another negative for IBO is I am not finished with my Kylix port. But, when
it is finished I will have goodies for all levels of Kylix, including the
open edition. Then, when you rack up costs, IBO will actually be the most
cost effective solution for working with InterBase over IBX.

FWIW,
Jason Wharton
CPS - Mesa AZ
http://www.ibobjects.com