With FB3 time came to solve this problem. I plan to follow mainly Vlad's suggestions regarding database encryption with some small changes, arising from new firebird features, present in FB3.

First of all I want to review requirements, mentioned by Vlad.
Requirements R1-R4 do not cause any questions. But R5 on my mind conflicts with an opportunity of authorization of the application for work with concrete DB. Even with encrypted backups it's easy to steal database using isql script which dumps metadata and data from tables. Therefore on my mind mentioned requirement should look like:

R5. All utilities should be able to work with protected DB provided it does not require application authorization. Backup tools should be able to produce encrypted backups and be able to restore DB from them with protection not worse than used for database. It's possible to let the create unencrypted backup with an assumption that users plan to encrypt it later themself.

When talking about R6 I prefer to separate users that develop firebird-based software and users who just run third-party developed applications. First group will have to modify their applications to make them work in "databases distributed with applications" mode. Second group will have to take some measures if they want to protect valuable enterprise data.

And I see no problems with R7/R8.

Now lets' try to understand how that requirements may be satisfied
inside FB3.

Encryption should be performed by plugin of new type. Requirements for such plugin in enterprise data protection mode and in distributed database protection mode are rather different, i.e. probably it makes sense to have 2 different plugin types.

Any encryption plugin must:
- support encryption of database by the engine
This is main obvious requirement for crypt plugins.
- support encryption of non-database data
This is necessary to support getting encrypted backups using services because gbak service will use same plugin to crypt backup file.
- distinguish ciphered DB from not ciphered
Engine must keep first database page (header) not crypted and store plugin name and may be some plugin-specific data (something like 'name of a key') in it. When opening database appropriate plugin is loaded and plugin-specific data (if any) is passed to it.

To protect enterprise data plugin must:
- does not need to support authorization of the application
In enterprise environment any application may work with data, access level must be limited for different users connecting to database.
- have a way of finding external encryption key
If hard disk or server is stolen, there are must not be any trace of secret key on it. Passing a key from workstations to server is definitely bad way to go - it's too easy to get lost when multiple copies are distributed over network.
- does not need to protect against theft of DB by substitution of the engine
Nobody can substitute engine before database is stolen. If after stealing database one can access secret key his task is solved and replacing engine with something else makes no sense. If key is not accessible, replacing engine would not help.

To protect distributed databases plugin must:
- should not keep a key externally
Keeping key externally makes it too simple to steal it. It's possible to store a key in external file in crypted form provided it's decrypted by plugin itself on startup. But looks like the best way is to keep a key in distributed application and pass it to plugin because it help to also...
- provide support for application authorization
Must notice that blindly passing a secret key from client to server is not an option - such key can be easily stolen by substitution of the engine. To make it possible ot authorize in secure way firebird must provide a channel between application and disk crypt plugin to help them pass a key in some secure way.
- protect against theft of DB by substitution of the engine
Plugin must know certified binaries to work with using for example some checksum of firebird engine and yvalve. Such binaries are typically distributed together with encrypted database.
- be as much as possible invisible for end-users of distributed database
Application developers who distribute databases will certainly need some programming efforts to protect database.