Limax Project

5.4 Server development

5.4.1 Code Generated -- xmlgen
- Operations
  
  The Limax.jar package integrates the tools to generate source codes.
```
java –jar limax.jar xmlgen –java server.xml                                   
```
  Generating the code in current path according to the model described by server.xml. Executing the command java –jar limax.jar xmlgen to get the detailed help information.
  
  Generally speaking, the description of the xml to build a project using the following organization:
  
  example.share.xml
```
<?xml version="1.0" encoding="utf-8"?> 
<namespace name="share" pvid="100">
 

    <bean name="MyBean">
        <enum name="e0" value="0"/>
        <enum name="e1" value="1"/>
        <variable name="var0" type="int"/>
    </bean> 
    <protocol name="MyProtocol" type="101" maxsize="4">
        <variable name="var0" type="MyBean"/>
    </protocol>
    <view name="MyGlobalView" lifecycle="global">
        <variable name="var0" type="MyCbean"/>
    </view>  
    <view name="MySessionView" lifecycle="session">
        <variable name="var0" type="example..MyXbean"/>
        <bind name="bind0" table="mytable"/>
        <bind name="bind1" table="mytable">
            <ref name="var0"/>
        </bind>
        <control name="control">
            <variable name="var0" type="int"/>
        </control>    
    </view> 
    <view name="MyTemporaryView" lifecycle="temporary">
        <variable name="var0" type="int"/>
        <subscribe name="_var0" ref="MySessionView.var0"/>
    </view>      
</namespace>                                  
```
  example.zdb.xml
```
<?xml version="1.0" encoding="utf-8"?> 
<zdb>
 

    <cbean name="MyCbean">
        <variable name="var0" type="int"/>
    </cbean> 
    <xbean name="MyXbean">
        <variable name="var0" type="int"/>
        <variable name="slist" type="vector" value="string"/>
    </xbean>
    <table name="mytable" key="long" value="MyXbean" autoIncrement="true"/>     
</zdb>                                  
```
  example.server.xml
```
<?xml version="1.0" encoding="utf-8"?> 
<project name=”example” xmlns:xi="http://www.w3.org/2001/XInclude">
    <xi:include href="example.share.xml"/>
    <xi:include href="example.zdb.xml"/>
    <state name="Server"> 
        <namespace ref="share" />
    </state>
    <service name="ExampleServer" useGlobalId="true" useZdb="true">
        <manager name="ExampleServer" type="provider" initstate="Server" port="10100"/>
    </service>         
</project>                                  
```
  example.client.xml
```
<?xml version="1.0" encoding="utf-8"?> 
<project name="example" xmlns:xi="http://www.w3.org/2001/XInclude">
    <xi:include href="example.share.xml"/>
    <xi:include href="example.zdb.xml"/>
    <state name="Client"> 
        <namespace ref="share" />
    </state>
    <service name="ExampleClient">
        <manager name="ExampleClient" type="client" initstate="Client" port="10000"/>
    </service>         
</project>                                  
```
  Use XInclude of the xml to organize the description files, and divide the parts shared by the server and the client, and the part belonging to the zdb. In the case of the complicated project, using the XInclude from the functional modules for more reasonable subdivision is considerable.
  
  The server's xml and the client's xml could be merged to the one if necessary. The -service parameter is added in the xmlgen command to generate the source code for some appointed service. Such use is NOT recommended.
  
  The scripting environment client does not need the client's xml to generate code. The xmlgen parameter should be appointed to generate the source code template corresponding to the script language when generating the server.
  
  The server's source code generated according to the xml's description is stored in the src directory and gen directory these two subdirectories within the current directory. The src directory is the source code directory. The files in this directory are updated and edited according to the requirement and this directory should be submitted to the version control system. It is not necessary for the gen directory for version controlling because it will be rebuilt when executing xmlgen and any modification will be lost. In addition, the service-XXX.xml file is generated in current directory and XXX is the service name described in the xml. The configuration required by running server is generated in this file and could be adjusted according to the operation requirement when deploying the detailed operating environment. If there are multiple services in one description file, the will be multiple generated service-XXX.xml files.
  
  New a eclipse Java project and appoint the current directory in the Location.
  
  In the Package Explorer, right click the mouse to edit the project property.
  
  In the Java Builder Path – Source, add the src directory and gen directory.
  
  In the ava Builder Path – Projects, add the reference to the Limax.
  
  Or in the Java Builder Path – Libraries, add the limax.jar.
  
  Then, the project organization could be clearly seen.
- Namespace Mapping
  
  Mapping the namespace described in the xml to the Java namespace.
  
  The name specification of protocol, rpc, view:
  
  The project name, the service name, the namespace name of the (protocol, rpc, and view), the name of the (protocol, rpc, and view).
  
  For example, the MyGlobalView defined in the previous content is named as example.ExampleServer.share.MyGlobalView.
  
  In particularly, there is the namespace example.ExampleServer.states in the gen directory, and all the generated source codes corresponding to the state element referred by the service via manager element in this namespace. The outermost namespace of the project could not be named as states to avoid the confusion, or there is error reported during generating.
  
  The name specification of the bean, monitorset:
  
  The project name, the namespace name of the (bean,monitorset), the name of the (bean,monitorset).
  
  For example, the MyBean defined in the previous content is named as example.share.MyBean.
  
  Although the bean and the protocol, rpc and view together are defined in the namespace element of the xml, the reason that using the different name specification is that the bean could be referenced by the protocol, rpc and view in multiple services. It could help to reduce generating the duplicated source code.
  
  To the monitorset, this name mode could be easily reflected to the expected jmx domain.
  
  The name specification of the cbean, xbean, table:
  
  The cbean is placed in the namespace of the cbean.
  
  The xbean is placed in the namespace of the xbean.
  
  The table in placed in the namespace of the table, and the table's name is revised as the initial upper case.
  
  When using the cbean, xbean and table, do not import the corresponding namespace, but directly use xbean.MyXbean and table.Mytable. It is clear to figure out which component of the zdb is used by the code in order to conveniently maintain.

5.4.2 View

After generating the source code, it is only necessary to edit some files in the src directory. The MyProtocol is not be discussed here.

Observing the source codes of the MyGlobalView，MySessionView，MyTemporaryView.

These three section source codes separately inherit from the _MyGlobalView, _MySessionView, _MyTemporaryView in the gen directory, and the public functions in the parent's classes contain the required code to operate the view.
- Obtain and maintain the View object
  
  Obtain the global View
```
MyGlobalView gview = MyGlobalView.getInstance();                                 
```
  There is only one global View and no parameter required to obtain.
- Manually synchronize the global View
```
gview.syncToClient(sessionid);                                 
```
  Synchronize the global View to the client
```
gview.syncToClient(sessionid, "var0");                                
```
  Synchronize the var0 field of the global View to the client
```
gview.syncToClient(Arrays.asList(sessionid1, sessionid2));                                
```
  Broadcast the global View to the appointed client set
```
gview.syncToClient(Arrays.asList(sessionid1, sessionid2), "var0");                               
```
  Broadcast the var0 field of the global View to the appointed client set.
  
  Obtain the session View
```
MySessionView sview = MySessionView.getInstance(sessionid);                               
```
  The parameter sessionid is required because the session View classified with the sessionid as the key will be automatically created when the session is created.
  
  Create the temporary View
```
MyTemporaryView tview = MyTemporaryView.createInstance();                              
```
  Destroy the temporary View
```
tview.destroyInstance();                            
```
  Obtain the temporary View
```
MyTemporaryView tview = MyTemporaryView.getInstance(sessionid, instanceid);                           
```
  Each temporary View maintains a Membership, and the sessionid in the Membership could be used as the parameter sessionid. After creating the temporary view, calling the function tview.getInstanceIndex() to obtain the instanceid required by the second parameter.
  
  Add the temporary View
```
tview.getMembership().add(sessionid);                          
```
  After successful execution, the onAttached method in the source of the temporary View will be called. Please refer the corresponding generated code of the MyTemporaryView. The action which will be executed behind the success should be added here.
  
  After the unsuccessful execution, the onAttachAbort method of the temporary View will be called. Please refer the corresponding generated code of the MyTemporaryView. The method AbortReason points out the failure reason, such as that the corresponding user of the sessionid is offline in the processing of adding sessionid.
  
  Remove the temporary View
```
tview.getMembership().remove(sessionid, reason);                         
```
  After successful execution, the onDetached method of the temporary View will be called. Please refer the corresponding generated code of the MyTemporaryView, where the parameter reason will be forwarded to the onDetached method. The reason should be none-negative because the negative value is used internally to define the system reasons, for example, closing the whole view causes the user detach.
  
  After unsuccessful execution, the onDetachAbort method of the temporary View will be called.
  
  There must be clear that the onDetachAbort method is the feedback to the operation failure of the Membership.remove. Destroying the temporary view causes the leave, and the system call the onDetached method with the negative value. If the operation of destroying the temporary view occurs before calling Membership.remove, there will be this situation, first the onDetached method is called with the negative value of the reason, then the onDetachAbort method is called with the VIEWCLOSED as reason. If the behavior of the leave operation need to be accurately processed, this situation should be considered.
- Operation to the View object
  
  There are three operations to the view object, which respectively correspond to the three sub-element variable, bind, and control of the view element described in the xml.
  
  Operation to the variable
```
<variable name="var0" type="int"/>                        
```
  Generating the setVar0(Integer)，_setVar0(Integer) methods.
  
  Providing two versions of the set method is related to the supporting to the transaction. Under the transaction environment, if using set version, the set operation will be really executed after transaction success, and the set operation will not be executed if transaction failure; however, in _set version, the set operation will be immediately executed and has no relationship with the success or failure of the transaction. Under the none-transaction environment, the behavior of these two versions is consistent. Performing the set with the null as the parameter means deleting this field.
  
  Operation to the bind
```
<table name="mytable" key="long" value="MyXbean" />
<bind name="bind0" table="mytable" />                        
```
  Generating the bindMytable(long) method, and the parameter type long is the long as the type of key in the mytable.
  
  For example, after the method view.bindMytable(100) is executed, if the data (which is the xbean of the type MyXbean) of the row key=100 in the table mytable is changed at the point of the transaction success, the changed data will be automatically set to the view's bind0 field.
  
  Operation to the control
```
<control name="control">
    <variable name="var0" type="int" />
</control>                            
```
  Generating the following methods:
```
protected static final class control implements limax.codec.Marshal, Comparable<control> { 
    public int var0;
    ......
}
protected void onControl(_MySessionView.control param, long sessionid) { }                          
```
  The response code to the control should be added in the onControl.
  
  In particularly, in the generated source codes of each view, there is
```
protected void onMessage(String message, long sessionid) { }                          
```
  When writing the message handling code, it is difficult to effectively construct the typed data for script client, so the default onMessage method will be used to forward the string message. The application should define its own schema specification for the string data and the server decodes the string message according to the specification.

5.4.3 ZDB
- The Storage Procedure
  
  The zdb provides the transaction support to the database through the storage procedure by using the limax.zdb.Procedure package. When the storage procedure returns the value TRUE, the transaction should be committed. If the value returned from the storage procedure is FALSE, the transaction should be rollback. There are three kinds of executing operation: execute, submit and call.
  
  execute：asynchronously execute the storage procedure
```
Procedure.execute(()->{
    the content of the storage procedure
    return true or false
});                          
```
  The above method does not care about the returned result, and is the easiest and commonly used method.
```
Procedure.execute(()->{
    the content of the storage procedure
    return true or false
}, (procedure, result)->{
    if (!result.isSuccess()) {
        System.err.println("Procedure " + procedure + "  fail");
        if(result.getException() != null) 
            result.getException().printStackTrace(System.err);
        else
            System.err.println(“procedure return false;”);
    }
});
```
  The returned value of the storage procedure could be obtained through the parameter of the second lambda expression, and the returned value provides the information of success or failure. If failed, it could detect whether it is caused by the exception.
  
  submit: asynchronously execute the storage procedure and return the Future<Procedure.Result>
```
Future<Procedure.Result>future = Procedure.submit(()->{
    the content of the storage procedure
    return true or false
});                          
```
  The execution result could be obtained via get function through calling future.get() and waiting for the storage procedure to finish.
  
  call: synchronously execute the storage procedure and return the Procedure.Result.
```
Result result = Procedure.call(()->{
    the content of the storage procedure
    return true or false
});                          
```
  It is particularly noted that the submit MUST NOT be called in the storage procedure, or the IllegalStateException exception will be thrown. If the storage procedure is nested called, the call function MUST be called. The nestification of the storage procedure causes the transaction nested, the failure of the inner transaction which causes the rollback would not affect the outer transaction. Calling execute in the internal storage procedure will start a new procedure, which could not achieve the nestification.
- Specification for the exception
  
  The ZDB framework has two kinds of exception:
  
  The User Exception, derives from java.lang.Exception.
  
  The Framework Exception XError, derivies from java.lang.Error, and is used by the framework and the generated source code.
  
  The user could throw or catch the user exception when implementing the storage procedure. It is forbidden to catch the Error and Throwable.
  
  The storage procedure throws the user exception when executing, which causes the current transaction to rollback, and the effect is similar to return false in the procedure. The storage procedure sets the exception to the return result.
  
  The storage procedure throws the XError when executing, which causes the current transaction to rollback to the outermost and directly set the outermost procedure's return result.
  
  In the submit mode, if the storage procedure throws the exception to the outermost when executing, the exception java.util.concurrent.ExecutionException e should be catched by using e.getCuase() to obtain this exception.
  
  Correctly set th procedure's log level, could record the exception catched by the framework.
- Standard operation on the table
  
  There are two types for the value of the table. One is the xbean type, the other is the const type (cbean and the other primitive types except for the binary). There is a little difference between these two types and will be introducted separately in the following.
  
  The operation on the table with xbean type as the value:
```
xbean = table.Tablename.insert(key);                          
```
  This statement is inserting one record according to the key. If the record has existed, the insert operation fails and returns null as value, or returns xbean. Then fill the content for the xbean, and the content will be added into the database after submitting.
```
key = table.Tablename.newKey();                          
```
  The autoincremental table could use this method to get the new key, then call the insert operation.
```
pair = table.Tablename.insert();                          
```
  The autoincremental table supports this operation, and the function equals the table.Tablename.insert(table.Tablename.newKey()). The returned pair is the limax.util.Pair type, which uses the pair.getKey() to get key and the pari.getValue() to get xbean, then fills the content for the xbean. Unless the autoincremental configuration is wrongly reset after it has been used, the returned xbean has no possible be the null.
```
xbean = table.Tablename.update(key);                          
```
  This statement is used to get the xbean record related to the key. After the obtained xbean has been modified and the transaction has been submitted, the modification will be updated to the database.
```
xbean = table.Tablename.select(key);                          
```
  This statement is obtaining the xbean record related to the key. The obtained xbean is READ ONLY, so the exception will be thrown and the transaction fails once the xbean is modified.
```
result = table.Tablename.delete(key);                          
```
  This statement is deleting the records related to the key and the return type is boolean. If success, return true, or return false.
  
  The operation on the table with the const type as the value (a case in cbean):
```
cbean = table.Tablename.insert(key, cbean);                         
```
  Unless the record has existed, the previous cbean will be returned, or the null will be returned.
```
pair = table.Tablename.insert(cbean);                         
```
  In addition to the multiple cbean parameters, it is similar to the autoincremental table with the xbean as the value.
  
  The update operation returns the const itself, but could not be modified. The difference between the select and update is that update obtains the write lock, however the select only obtains the select lock.
  
  The operation select and remove of this table is similar to the table with the xbean as the value.
- None-standard operation on the table
```
table.Tablename.get().getCache().walk((key, value) -> {
    //readonly action
});                        
```
  This statement is traversing the cache of the table. The record will be READ locked one by one during the traversing, and the provided record is READ ONLY. This method could used not only on the disk table, but also on the memory table.
```
table.Tablename.get().walk((key, value) -> {
    //readonly action
});                        
```
  This statement is used to traverse the contents of the underlying database. The uncheckpointed records of the memory database would not be accessed. The zdb periodically checkpoints the submitted data to the underlying database. The paramter limax.zdb.Checkpoint.SCHED_PERIOD of the virtual machine controls the minimum period of the checkpoint check and the defaul value is 100ms. The detailed Chekpoint period is configured by the running xml parameter.
  
  Due to the accessed contents in the underlying database, it is not related to the zdb lock. This walk is only used on the disk table.
  
  For the two kind of the walk, the result is unordered, execute in transaction context is not needed.
- Transaction isolation level
  
  The default isolation level is level2 and the level3 could be used in the very necessary condition. In this condition, the transactions are serially executed and the performace is low.
```
Transaction.setIsolationLevel(Transaction.Isolation.LEVEL3);                       
```
  The current transaction isolation level is set as level3, which means that any transaction started by current thread has the level3 isolation.
```
Transaction.setIsolationLevel(Transaction.Isolation.LEVEL2);                      
```
  The isolation level is modified as level2.
- Lock
  
  The lock of the zdb is the row lock and the default transaction isolation level is level2. The holdlock is implemented to guarantee the repeatedly read till the end of the transaction. The select operation obtains the READ lock, and the insert, update, and delete operations obtain the WRITE lock. (After the select option is executed, if the update option need to be executed next in the same key, the previous READ lock need be released in order to add the WRITE lock, which means that the READ lock is upgraded as the WRITE lock.) In reverse, the lock does not be downgraded because if the lock is downgraded as the READ lock after modifying the record, the other transaction has the possible to read the modified data, which causes the transaction isolatoin level to be downgraded to the level0, so the dirty read happens, and it might cause the logic error. That means:
```
xbean = table.Tablename.select(key);
table.Tablename.update(key);                      
```
  After these serial operations have been finished, the xbean allows to be modified. Even table.Tablename.select(key) operation is called again, the xbean still could be modified. From the experience, the typical transaction that one key is judged as not existed and then is inserted is similar to the operations on the relational database.
```
if (table.Tablename.select(key) == null)
    table.Tablename.insert(key);                      
```
  The above operation is considered as the bad one becuase it is possible that between releasing the select READ lock and inserting the WRITE lock, the other transction uses this key to execute the insert operation and causes the insert operation failure, which directly breakes the original intention of the above statements. The reasonable statement should be:
```
xbean = table.Tablename.update(key)
if(xbean == null)
    table.Tablename.insert(key);                      
```
  The complicated transaction may result in the deadlock. The zdb supports the detection to the deadlock. The detection period of the deadlock, the frequency of the retries, and the maximum time for the retry backoff could be configured. The deadlock as the exception is recorded to the return value of the storage procedure and could be recorded through log configuration. The design under the optimistic mode does not consider the lock issue. Checking the log, finding the deadlock hot, considering the explicit pre-lock solution are the necessary steps if there is case to be solved.
  
  By locking the row of the table involved in the transaction one time in advance, the deadlock issue could be effectively resolved. The package limax.zdb.Transaction.LockContext will be used.
  
  For Example:
  
  WRITE locking the row row1, row2 of the table ta, tb at the same time could use the below:
```
Transaction.getLockContext().wAdd(row1, row2, table.Ta.get(), table.Tb.get()).lock();                      
```
  WRITE locking the row row1, row2 of the table ta, tb, and READ locking the row3 of the table tc at the same time could use this:
```
Transaction.getLockContext().wAdd(row1, row2, table.Ta.get(),table.Tb.get()).rAdd(row3, table.Tc.get()).lock();                     
```
  An add operation requires to lock all the listed rows of all the listed tables in the parameters. It is flexible and could be listed one by one, or given through the container. There is no any limitation for the order.
  
  For Example:
```
wAdd(table.Ta.get(),table.Tb.get(), row1, row2);
wAdd(new Object[]{row1, row2}, new Object[]{table.Ta.get(),table.Tb.get()});
wAdd(new Object[]{table.Ta.get(),table.Tb.get(), row1, row2});
wAdd(Arrays.asList(table.Ta.get(), row1), Arrays.asList(table.Tb.get(), row2));                     
```
  All of the above examples are the same.
  
  Actually, the parameter value of the collection type or array type are totally recursively parsed to distinguish the table set construct locked by the table type object, and the construct row set of the none-table type object.
- Access Xbean
  
  The Xbean only could be accessed in the transaction environment. Ensure that the appropriate lock exists when accessing to avoid the concurrency conflicts which leads to incorrectly serializing so that the error is spread to the next database read and affect the normal operations of the service. In the implementation, the zdbVerify of the zdb running configuration controls the relative lock detection, and the attribute zdbVerify is true as default. The lock is verified when xbean is accessed. If the accessing lacks of the lock, the limax.zdb.XLockLackedError is thrown. Unless the strict coverage test is finished, do not set the zdbVerify=false for a small quantity of performance improvement, especially the usage of the closure easily brings the Xbean outside the scope of the lock. The xbean obtained by the read-only operation such as the select and walk table cache, only supports the read accessing, and the limax.zdb.XLockLackedError will be thrown when write accessing. The server running log needs to be checked regularly. Once limax.zdb.XLockLackedError is found, it needs to revise the code according to the relative stack information to ensure the concurrency safety.
```
<xbean name="MyXbean">
    <variable name="var0" type="int"/>
    <variable name="slist" type="vector" value="string"/>
</xbean> 

MyXbean xbean = new xbean.MyXbean ();                   
```
  The xbean object uses the get and set methods to access the primitive type data.
```
int x = xbean.getVar0();
xbean.setVar0(100);                  
```
  The xbean object uses get method to obtain the container to modify when accessing the collection type.
```
xbean.getSlist().add("abc");
xbean.getSlist().remove(0);                  
```
  The record is regarded as the root element by the Xbean from the organization structure, the layers connect together one by one and form a tree. The xbean as the element in a tree could not connect to another tree.
  
  For Example:
```
Xbean1 x1 = table.Table1.select(key1);
Xbean2 x2 = table.Table2.update(key2);
x2.getArray().add(x1); //It is assumed that the array field of the Xbean2 is defined as the vector of the Xbean1.                  
```
  The statement x2.getArray().add(x1) is forbidden. In this condition, the exception is thrown to report the Xbean management error and the transaction ends. The above example could resolve the issue by copying the data. For example:
```
x2.getArray().add(new Xbean1(x1))                 
```

5.4.4 Monitor

Here simply explain the usage of the Monitor.

The limax.xml defines the TransactionMonitor. The generated source code limax.zdb.TransactionMonitor provides six methods for application development:

public static void increment_runned(String procedureName);
public static void increment_runned(String procedureName, long _delta_);
public static void increment_false(String procedureName);
public static void increment_false(String procedureName, long _delta_);
public static void increment_exception(String procedureName);
public static void increment_exception(String procedureName, long _delta_);

two methods are provided for the collection application in the execution environment to use :

public static String buildObjectNameQueryString(String procedureName);
public interface Collector;

Prev Next

5.4 Server development

5.4.1 Code Generated -- xmlgen

5.4.2 View

5.4.3 ZDB

5.4.4 Monitor