Limax Project

7.14 Dynamic XBean

The Zdb database that supports dynamic XBean allows adding, deleting, and modifying the definition of the XBean field without the need to perform data format conversions on the original Zdb database and reduce maintenance costs. Please refer the "Operation Management" data conversion section for details.

7.14.1 Fundamental

1. Dynamic XBean consists of static fields and dynamic fields. Adjustment of static fields requires data format conversion, and adjustment of dynamic fields is not required.

2. Whether it is a static field or a dynamic field, there is no difference in the way programming access.

3. A dynamic XBean organizes all dynamic fields into a Map. This Map is defined as dynamic data and is extended in code. The encoding rule is to first encode all static fields in the defined order, then encode the number of dynamic data, and then encode the dynamic data entry. Among them, the key of dynamic data entry is serial and value is the byte array obtained after the dynamic field is encoded separately.

4. Serial is the key to implementing dynamic XBean. Serial is allocated on XBean and any modification must steadly increase serial. Serial reflects the modification history of the XBean.

5. The XBean reads the serial that is not known by the current running code, which means that the field corresponding to the serial needs to be deleted, and the data corresponding to this serial is ignored in the decoding process.

6. Adding dynamic fields will inevitably allocate all the latest serials that are not known to the stored XBean. The XBean dynamic data obtained from the decoding process cannot update the value of this field, so what the user sees is the default value initialized when the XBean is constructed ( Unless the script is defined in the field description, please refer the discussion below for the detail).

7. The same dynamic field, if the definition needs to be modified, the latest definition has the latest serial, the previous definition must be retained, and these definitions reflect the history of the current data. Design allows providing a script description, defining the previous serial conversion to the new serial, in the decoding process if the previous serial is decoded, calling the conversion process to convert the field to the latest version, update XBean corresponding field, and provide to the user access. When encoding stored XBean, the latest version is written to the database.

7.14.2 Description of dynamic XBean
- Description style
```
<?xml version="1.0" encoding="utf-8"?>
<zdb dynamic="true">
    <xbean name="MyXbean" nextserial="1">
        <variable name="var0" type="int" />
        <variable name="var1" dynamic="true"> 
            <variable serial="0" type="int"/> 
        </variable>
    </xbean>
    <table name="mytable" key="long" value="MyXbean" autoIncrement="true" />
</zdb>
```
  1. The zdb element dynamic attribute determines that the dynamic XBean is allowed in this zdb. Once zdb is defined in this way, all XBeans in the zdb are dynamic, whether or not dynamic fields are defined.
  
  2. The nextserial attribute of the xbean element determines the serial that should be used when the dynamic field definition is next modified. After the serial is used, the nextserial must be incremented. This attribute is used for code generation checks to avoid the wrong allocation of serial, which results in runtime errors. Default nextserial is 0, and XBean allows default when no dynamic fields are defined.
  
  3. Field var0 is a static field.
  
  4. The field var1 is a dynamic field, the current serial is 0, and the type is int.
- Add dynamic field
  
  For example, adding a string field var2, add a description fragment
```
        <variable name="var2" dynamic="true"> 
            <variable serial="1" type="string"/> 
        </variable>
```
  This uses the serial 1 specified by the xbean attribute nextserial, so the nextserial should be modified to 2.
- Delete dynamic field
  
  For example, deleting var2, delete the above description fragment. The nextserial attribute of the xbean is not modified.
- Modify dynamic field
  
  For example, modify the dynamic field var1, where the new type is double, and the value is the previous value multiplied by 3.14.
```
        <variable name="var1" dynamic="true"> 
            <variable serial="0" type="int"/> 
            <variable serial="2" type="double" script="$0 * 3.14"/> 
        </variable>
```
  Here defines the var1 update type, and serial is 2. It should be remembered to update the xbean's nextserial to 3.
  
  In the dynamic field definition, the information described by the element with the largest serial is the current description information of the field, and other elements provide history description information.
  
  The script attribute can fill in an expression, where $0 represents the history value that depends on serial="0", and the basic process of decoding the dynamic field is:
  
  1. If the data corresponding to the latest version of serial already exists, decode it directly.
  
  2. If it does not exist, the dependency value is recursively parsed according to the dependency described by the script, and the result is taken as the value of the current dynamic field.
  
  3. If any dependency value does not exist during recursive parsing, the default value initialized at the time of field construction is used.
  
  From the current example, if there is a serial="2" value in the read XBean dynamic data, directly decode and initialize var1, otherwise check the value of serial="0", if it exists, then decode it as an int type, which is multiplied by 3.14 and get the result to initializs var1, and if the value of serial="0" does not exist, the default value when using the var1 construct is 0.0.
  
  The history description information is not referenced by any script within the scope defined by the XBean, meaning that historical values may be discarded and a warning will be generated when the code is generated. Here, if the script corresponding to serial="2" is not defined, it will warn that serial="0" is not used.
  
  The design principle of script is:
  
  1. For flexibility, it is allowed to use $ to refer to any field in the XBean, not limited to the historical description of the current dynamic field.
  
  2. The form of $number refers to the history dynamic field, where number must be smaller than its own serial and exists in the definition of current XBean, otherwise an error occurs during generation.
  
  3. The form of $word refers to a static field, and word must be a valid static field name, otherwise an error occurs during generation.
  
  4. The generated code only checks the legitimacy of the $ reference and does not perform syntax checks. Therefore, after generation, should check whether the generated code is wrong. If there is any error, modify the script description and regenerate it.
  
  5. A script can only fill in a valid expression. If need to perform complex operations, can first define a static function that is visible to the generated XBean and perform the conversion. For example script="mypkg.MyApp.transform($0)".
  
  6. In particular, the result of a script expression defined in the smallest serial element of a dynamic field cannot be interpreted as the initial value of the field. This value is just the default value used when decoding XBean. It is not the default value for constructing XBean. This requires special attention. Usually, it is better not to define the script attribute in the smallest serial element. This ensures that the decoding default value is the same as the construction default value.
  
  7. When modifing the dynamic field and designing the script, correctly distinguishing the decoding default value and the construction default value is very important for insert new records, how to correctly fill in the value of the corresponding field.

7.14.3 Data format conversion

Using dynamic XBean can reduce the cost of data format conversions that result from modifying the Zdb description. There are at least two situations that need to be converted. First, it is the conversion from a non-dynamic XBean system to a dynamic XBean system. Second, Zdb describes revisions too often, and developers feel difficult to maintain.

The operation of data format conversion is exactly the same as that of the "Operation Management" data conversion section. The developers should first familiarize themselves with the operation process.
- Static to dynamic
```
<?xml version="1.0" encoding="utf-8"?>
<zdb>
    <xbean name="MyXbean">
        <variable name="var0" type="int" />
    </xbean>
    <table name="mytable" key="long" value="MyXbean" autoIncrement="true" />
</zdb>
```
  Assume that the original description of Zdb has been running for some time. Now it needs to modify the form defined for the previous example and add the dynamic field var1. After modifing the Zdb description, generate the code and run the program. It will report the error "convert needed to cast to dynamic".
```
java -cp ../limax/bin/limax.jar;bin limax.zdb.tool.DBTool -e "convert zdb zdbcov"
```
  In the current development environment, refer to the above command execution to generate the conversion code.
  
  After compiling the code, execute the above command again (note that the zdbcov directory is created) and the data is converted to zdbcov.
  
  Back up the zdb directory and rename zdbcov to zdb. The program will execute normally.
  
  In particular, the dynamics cannot be converted back to static.
- Maintenance conversion
  
  The dynamic field in the Zdb description has been modified too many times, and developers feel difficult to maintain, so can consider generating code to perform maintenance conversion. The nature of maintenance conversion is to read all stored XBean once, update to the latest version, and clear all historical data. Then can clean up the Zdb description, delete the history description of all dynamic fields, only keep the largest element of serial.
```
java -cp ../limax/bin/limax.jar;bin limax.zdb.tool.DBTool -e "convert zdb zdbcov"
```
  Performing a maintenance conversion also requires running such a command. The result is to generate conversion code for all tables that have an XBean value, and to repeat the above command to perform the conversion after compiling the code.
  
  It should be noted here that even if the XBean does not describe any dynamic fields, the conversion code is generated because the conversion tool cannot determine whether the dynamic field never existed or was simply deleted in the current description. Therefore, developers can clean up the corresponding generated code according to the actual situation. There are two basic principles, first, really never define the dynamic field; second, the dynamic field has only a definition, has never been modified, and can be sure that there is no database history information. The simplest strategy is not to clean up at all, and the conversion time is longer.
  
  If the definition of a static field is modified, the running program will naturally report that it needs to be converted. Refer to the "Operation Management" data conversion section.

7.14.4 Dynamic or static discussion

Basic comparison

	Dynamic	Static
Performance	The coding and decoding process is complicated, and performance is poor.	Good
Impact of modifying the description	Small impact, dynamic field revisions have no need to perform data conversion	Big impact, need to be converted every time
Safety	Low, modification of dynamic field must be very careful, and executing the wrong code on the database will have an irreparable impact.	High, since each modification needs to be converted, the source of the conversion can be used as a backup. Once the problem occurs, it is easy to recover.

Discussion of the modifying description

1. Database design must be carefully considered and should not be modified if not must.

2. Dynamic mode is not a reason to arbitrarily modify the design; usually the dynamic system will take the bad influence on the designer.

3. The usual modifications are brought about by changes in demand. It is better to discuss the demand, anticipate some possible changes, and prepare in advance.

Recommended way

1. The project starts to be put into use, and static methods are recommended.

2. During maintenance, it is a very safe method to use the conversion tool to convert the database and back up the original database.

3. As the number of records increases, the conversion time will soon exceed the tolerance limitation; it can consider converting to a dynamic mode, reducing the subsequent conversion frequency.

4. Once the use of dynamic methods requires special care, strict testing should be performed on the backed up data and then applied to the actual system to avoid operational accidents.

Prev Next

7.14 Dynamic XBean

7.14.1 Fundamental

7.14.2 Description of dynamic XBean

7.14.3 Data format conversion

7.14.4 Dynamic or static discussion