Subject Re: [firebird-support] suggestion for a db scenario
Author Thomas Steinmaurer
> Hi all, it's been a while since i last used FB (not that i used
> anything else :) but i have not been involved with DBs :))
> Now we're planning a project with hundreds of clients (think 5-600)
> that will do insert queries every minute or so (to save status data)
> over WAN and one or two clients that will do select queries to plot
> this information, think of: select avg(myval) from table1 where
> "timestamp<15min ago"
> Since the stored data will grow quite fast (we need to store a few
> months worth of data) have you any recommendation on which kind of
> architecture (both HW and FB) would be suitable?

What platform do the clients use? Are these rich client applications or
just dumb value providers (e.g. sensors)?

For a DWH project collecting measurement value from devices in the
field, we did something along the lines:

A dedicated "data collector process" in the main office, which receives
data (compressed XML in our case) from the devices in the field,
transforms that into a relational interpretation and inserts this data
into a Firebird database.

An ETL job then transforms this daily business data into the OLAP/DWH
world and inserts data into a separate OLAP Firebird database. There we
have a custom pre-aggregate strategy in place (I described that stuff in
this article
http://www.ibphoenix.com/resources/documents/general/doc_1), which is a
must have for having an acceptable response time on adhoc aggregate
queries with a growing data volume. In our architecture, we also use an
OLAP server (Mondrian), which is capable to access aggregate tables
transparently for the end-user, so the end-user doesn't need to know
what aggregate tables with what pre-aggregated data is available.

In regard to hardware. Start with something you already have, because,
usually, with a well-designed database, you don't need THAT fast
hardware, although it won't of course. I guess something along a
quad-quore, 8GB and acceptable IO would be fine. You'd best splitting up
OS, database files and temp files onto dedicated physical drives.

All, without knowing a lot about your project, so, don't charge/sue me
if something goes wrong. ;-)



--
With regards,

Thomas Steinmaurer
Upscene Productions
http://www.upscene.com
http://blog.upscene.com/thomas/

Download LogManager Series, FB TraceManager today!
Continuous Database Monitoring Solutions supporting
Firebird, InterBase, Advantage Database, MS SQL Server
and NexusDB!