| Subject | Re: [IBO] Calling Commit on TIBODatabase forces Post on all components in edit or insert mode without explicit transactions |
|---|---|
| Author | |
| Post date | 2019-10-31T20:25:44Z |
Our code did work correctly when we were using BDE. We moved to IBO because it has replacements for BDE components. We would not have been able to move IBO had we had to change the thousands queries / table objects in our system to all be done the "right way" with the small staff that we have. It doesn't matter to me if what the BDE was doing was "wrong" if that wrong functionality worked and now with IBO it is broken. What I need out of the TDataset components in IBO is for them to be able to work like BDE did as much as possible so that the more than 200 screens I have to maintain work properly. I prefer it to work like BDE out of the box, but I am also ok with having to opt in if need be. For the most part stuff has worked like BDE. There have been edge cases and other various issue and I have been able to work with Jason to find the easiest solution possible.
As to the way the BDE worked. There was only 1 transaction in the system, the "global transaction". You could either explicitly start and commit it so that you could batch operations into one transaction or not. If you performed an action without starting the transaction then that action automatically started the transaction before the operation then committed if after. If the system's transaction was already started before the operation was performed then it was assumed you wanted that operation to be part of a larger set of operations inside a transaction and it would treat it as such. When you performed a commit it wouldn't go to every query on every screen and call Post on them if they were not in browse mode. If a transaction was not started when you went in to insert / edit mode on a TTable it wouldn't start a transaction, but rather the transaction logic only came in to play during the Post operation.
IBO is treating these TIBOTables that don't have a transaction assigned like they are all a part of the global transaction if one is started while they are already in insert / edit mode. If you call Post on them while you are in the middle of a transaction then sure, add them to the transaction. With the global transaction functionality there is really no other option. But if you don't call Post on them then they shouldn't be considered to be a part of that transaction.
This code has worked fine in the past with BDE because when we explicitly start a transaction, the only Post calls on components are done within a single block of code. The user is not able to interact with the GUI causing other queries or table objects to Post during this operation. In the failure scenario we are seeing, a user clicks a New button to create a new record, but before the hitting the Save button they go to another screen and perform some action that involves an explicit transaction. On the original screen, the New button caused a TIBOTable to have have Append called on it, which put it in insert mode. When you perform an operation on another unrelated screen that requires an explicit transaction because you need to batch multiple database operations together, that TIBOTable gets Post called on it. Previously not only did this not happen, but there was no way for the Post to be called on that TIBOTable without the user clicking the Save button which caused other necessary logic to be called before it called Post. This IBO logic is causing important logic to be bypassed and data to be corrupted in the database.
If I was writing a new application in Delphi with IBO I wouldn't use the global transactions. I also would not use TIBOTable or TIBOQuery with the RequestLive property set to true. I would connect data-aware controls to in-memory datasets only and I would manually populate those datasets with data from manually crafted queries and I would transfer their values to a manually crafted query when data needed to be persisted to the database. I would have full control and explicitly execute transactions around every database operation. That is not the world I live in unfortunately. And to get to this scenario we would need to perform a complete rewrite for more than 700000 lines of code. That is just not in the cards for us. We are too small of a company and there is too little to gain for us to decide to devote many years to that project. I need the screens to continue to work like they did before IBO came in to the picture. Hopefully with as little work as possible. We manually fixed a single screen and it took about a week because of how complex the code is and how many different classes it utilizes that needed to be changed to support explicit transactions. We can change the remaining 60 places in our code that need this change. Some will be as painful as this screen and some won't. What I am hoping is that Jason may already have a way to have IBO act like BDE and not call Post on all queries in Insert / Edit mode through some property setting etc. If not I am hoping that there may be a change that can be safely made to IBO to be able to mimic BDE possibly through some opt in setting. It is not totally crazy to ask for the TDataset based components to be able to work like the they did when they were BDE TDataset based components. The BDE might not be your favorite thing in the world. It certainly isn't mine. That is one of the reasons to leave it behind and use IBO. But for a codebase our size we need some consistency between the two products so that we can keep our software working while we slowly transition to a better codebase.
---In IBObjects@yahoogroups.com, <listas@...> wrote :
Re: [IBO] Calling Commit on TIBODatabase forces Post on all components in edit or insert mode without explicit transactions
Hi!
I can't see how what you are asking for would work correctly, since the records visibility for the queries are determined by the transaction isolation and the "time" that the transaction started. BDE transaction handling sucks, even more for RDBMS like Firebird, which uses MVCC/Versioning and needs transactions even for simple readonly selects.
If you commit the transaction, the concurrency control for that "transaction context" is gone, so how do you expect the other datasets associated to that transaction to handle ongoing edits, etc. if the transaction context doesn't exists anymore?
Whatever BDE is doing in this case, I think it is wrong and inconsistent.
I would go and refactor the transaction control of the application. Also, keep an eye in the TransForUpdate property. IBO allows you to associate a ReadOnly+ReadCommit transaction for browsing, and a writable transaction to handle the DML.
[]s
Carlos H. Cantu
eBook Guia de Migração para o FB 3 - www.firebase.com.br/guiafb3.php
www.FireBase.com.br - www.firebirdnews.org - blog.firebase.com.br
This code has worked fine in the past with BDE because when we explicitly start a transaction, the only Post calls on components are done within a single block of code. The user is not able to interact with the GUI causing other queries or table objects to Post during this operation. In the failure scenario we are seeing, a user clicks a New button to create a new record, but before the hitting the Save button they go to another screen and perform some action that involves an explicit transaction. On the original screen, the New button caused a TIBOTable to have have Append called on it, which put it in insert mode. When you perform an operation on another unrelated screen that requires an explicit transaction because you need to batch multiple database operations together, that TIBOTable gets Post called on it. Previously not only did this not happen, but there was no way for the Post to be called on that TIBOTable without the user clicking the Save button which caused other necessary logic to be called before it called Post. This IBO logic is causing important logic to be bypassed and data to be corrupted in the database.
If I was writing a new application in Delphi with IBO I wouldn't use the global transactions. I also would not use TIBOTable or TIBOQuery with the RequestLive property set to true. I would connect data-aware controls to in-memory datasets only and I would manually populate those datasets with data from manually crafted queries and I would transfer their values to a manually crafted query when data needed to be persisted to the database. I would have full control and explicitly execute transactions around every database operation. That is not the world I live in unfortunately. And to get to this scenario we would need to perform a complete rewrite for more than 700000 lines of code. That is just not in the cards for us. We are too small of a company and there is too little to gain for us to decide to devote many years to that project. I need the screens to continue to work like they did before IBO came in to the picture. Hopefully with as little work as possible. We manually fixed a single screen and it took about a week because of how complex the code is and how many different classes it utilizes that needed to be changed to support explicit transactions. We can change the remaining 60 places in our code that need this change. Some will be as painful as this screen and some won't. What I am hoping is that Jason may already have a way to have IBO act like BDE and not call Post on all queries in Insert / Edit mode through some property setting etc. If not I am hoping that there may be a change that can be safely made to IBO to be able to mimic BDE possibly through some opt in setting. It is not totally crazy to ask for the TDataset based components to be able to work like the they did when they were BDE TDataset based components. The BDE might not be your favorite thing in the world. It certainly isn't mine. That is one of the reasons to leave it behind and use IBO. But for a codebase our size we need some consistency between the two products so that we can keep our software working while we slowly transition to a better codebase.
---In IBObjects@yahoogroups.com, <listas@...> wrote :
I can't see how what you are asking for would work correctly, since the records visibility for the queries are determined by the transaction isolation and the "time" that the transaction started. BDE transaction handling sucks, even more for RDBMS like Firebird, which uses MVCC/Versioning and needs transactions even for simple readonly selects.
If you commit the transaction, the concurrency control for that "transaction context" is gone, so how do you expect the other datasets associated to that transaction to handle ongoing edits, etc. if the transaction context doesn't exists anymore?
Whatever BDE is doing in this case, I think it is wrong and inconsistent.
I would go and refactor the transaction control of the application. Also, keep an eye in the TransForUpdate property. IBO allows you to associate a ReadOnly+ReadCommit transaction for browsing, and a writable transaction to handle the DML.
[]s
Carlos H. Cantu
eBook Guia de Migração para o FB 3 - www.firebase.com.br/guiafb3.php
www.FireBase.com.br - www.firebirdnews.org - blog.firebase.com.br
First off I know using the global transaction functionality is not ideal, but this is we have a lot of legacy code using it since this was formerly a BDE base product and the global transaction was the only option. We have a log of legacy code using queries and table components that don't explicitly utilize transactions. When they post changes the transaction is automatically started and committed. We also have a number of places in legacy code where transactions were required to ensure consistency in the case of a failure. This old (originally BDE based) code uses the StartTransaction and Commit methods on the database component. The utilizes the same global transaction as the components that don't reference transactions. We have found that now that we have migrated to IBO the following scenario can occur which did not happen when we were using BDE. In one screen someone makes a change on a data-bound control backed by a TIBOTable which causes it to go in to edit mode. They then, before saving changes, can go to another screen and perform an operation that is explicitly wrapping the database changes in a transaction. Because the transaction is the old global transaction, when the commit happens in the second screen IBO is apparently forcing all datasets not bound to an explicitly defined transaction to Post. I am assuming IBO is trying to be helpful and Post unposted work at that time. BDE did not do this. Beyond being unexpected behavior that doesn't mimic the BDE controls that IBO replaced, this can lead to errors in the application. For instance we have found a screen where there is a Save button which contains validation logic as well as other required logic that must happen before saving the record by calling Post. By having IBO decide to call the Post itself, it bypassed this logic and caused bad data to be saved to the database. If you have two components that don't specify a transaction and the code doesn't call StartTransaction / Commit on the database component then they work fine. This is only a problem when you have components that don't specify a transaction along with other components/queries you intend to be wrapped in a transaction. We can fix this on our own but it is painful. In cases where we just need to instantiate a transaction and attach it to a few queries it is no big deal. But many of our screens utilize utility classes that perform database operations. When those are used we then have to plumb the transaction to them so they can uses it if provided. Those classes themselves may use other classes which also need the transaction provided to them. It has taken us a week of work to migrate a single screen and all it's dependencies to use an explicitly instantiated transaction instead of the global one. Doing a search I see that there are at least 60 other places in our code where this needs to be done before we have fully removed using the TIBODatabase component's transaction methods. Some I'm sure will be easy, but I bet others will be the same complicated quagmire this screen has been with confusing legacy code and deep dependencies that need to be updated to use IBO transactions. Here is my question. Is there a way to have IBO operate like BDE did? If we call StartTransaction / Commit on the database component, we don't want that operation to try and call Post on all other datasets in edit or insert mode. The way it appears that it use to work was that datasets in those states would automatically start and commit the global transaction at the point at which we called Post on them explicitly. Can IBO let the TDataset components operate using this old workflow and not call Post automatically? |