| Subject | RE: [firebird-support] Re: Out-of-memory problem |
|---|---|
| Author | Alan McDonald |
| Post date | 2003-08-14T09:25:52Z |
well if you are using tables, let's follow this logic..
the same sort of thing can happen if you select * from table (as a query)
with no limiting clause...
you are storing 1.1 million records on disk in the gdb file.
you ask the serve to serve up all these records... you (I think) are also
running this server on the same machine as the client is running.
So the server starts to deliver these records to the client (i.e. PC
memory). The pc memory becomes starts to fill and the c starts to then use
virtual memory.. i.e. it starts to write these same records out to disk
since it cannot hold all of them in memory. By the time you get all records
out of the server environment you now have written most of the records back
to disk in the form of virtual memory (disk pageing). You now have two
copies of each record... the database copy inside the gdb file and the
cached client copy some in memory and some in virtual disk paging. For the
same reason that windows occasionally warns you that you are running low on
resources when too many application are running... you are now running low
on pc memory since all of it is now taken up with as many of the 1.1 million
records you hacve requested.
One improvement will be had by putting the server or client app on a
separate machine. This will not totally avoid the issue that the server will
need to rapidly buffer these records off disk in memory as fast (faster)
than it can deliver them via the NIC to your client app. So you are still
asking a lot of your server to deliver 1.1. million records in one hit.
A remarkable improvement will be had by asking for pages of data from the
server, lets imagine 500 records at a time. Dealing with these records then
asking for 500 more. There are always ways of dealing with parcels of data
rather than asking for all records in one hit.
That is why you are getting out of memory problems. 256Mb of server memory
is pretty basic. Don't demand this much from you server - its simple.
Alan
the same sort of thing can happen if you select * from table (as a query)
with no limiting clause...
you are storing 1.1 million records on disk in the gdb file.
you ask the serve to serve up all these records... you (I think) are also
running this server on the same machine as the client is running.
So the server starts to deliver these records to the client (i.e. PC
memory). The pc memory becomes starts to fill and the c starts to then use
virtual memory.. i.e. it starts to write these same records out to disk
since it cannot hold all of them in memory. By the time you get all records
out of the server environment you now have written most of the records back
to disk in the form of virtual memory (disk pageing). You now have two
copies of each record... the database copy inside the gdb file and the
cached client copy some in memory and some in virtual disk paging. For the
same reason that windows occasionally warns you that you are running low on
resources when too many application are running... you are now running low
on pc memory since all of it is now taken up with as many of the 1.1 million
records you hacve requested.
One improvement will be had by putting the server or client app on a
separate machine. This will not totally avoid the issue that the server will
need to rapidly buffer these records off disk in memory as fast (faster)
than it can deliver them via the NIC to your client app. So you are still
asking a lot of your server to deliver 1.1. million records in one hit.
A remarkable improvement will be had by asking for pages of data from the
server, lets imagine 500 records at a time. Dealing with these records then
asking for 500 more. There are always ways of dealing with parcels of data
rather than asking for all records in one hit.
That is why you are getting out of memory problems. 256Mb of server memory
is pretty basic. Don't demand this much from you server - its simple.
Alan
> -----Original Message-----
> From: Gary Jones [mailto:nonesuch1960@...]
> Sent: Thursday, 14 August 2003 7:11 PM
> To: firebird-support@yahoogroups.com
> Subject: [firebird-support] Re: Out-of-memory problem
>
>
> --- In firebird-support@yahoogroups.com, Daniel Rail <daniel@a...>
> wrote:
> > What are you using to make the connection? BDE? IBX? other
> > component? How much memory do you have on that computer and what
> is
> > the size of the paging file?
> >
> > --
>
> Connection being made with IBX.
> RAM on PC: 256Mb.
> Paging file: 4096 bytes
>
> Thanks for any help.
>
> (In response to the other reply; as I said in my original post, I
> know that using tables in this way in C/S situations is a Bad Idea.
> I would just like to know what's actually causing the problem in
> this case.)
>
>
>
> To unsubscribe from this group, send an email to:
> firebird-support-unsubscribe@yahoogroups.com
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>