This chapter provides a description for each CUBRID server components. It will cover the following topics:
- Connection management
- Threads module
The Figure 1.1 illustrates the connection and role of these components in CUBRID server.
In CUBRID server, each connection’s information is stored in a CSS_CONN_ENTRY structure. All CSS_CONN_ENTRY store in an array, this array is illustrated as css_Conn_array. CSS_CONN_ENTRY can be divided into two parts: the active part and the free part.
If a CSS_CONN_ENTRY is in the active part, it means this CSS_CONN_ENTRY stores the information about a connection. All active CSS_CONN_ENTRY form a linked-list. The pointer css_Active_conn_anchor points to the head of the list.
If a CSS_CONN_ENTRY is in the free part, it means this CSS_CONN_ENTRY is not used. If a new connection is established, the information about the new connection will be stored into a free CSS_CONN_ENTRY. Free CSS_CONN_ENTRY also form a linked-list with css_Free_conn_anchor pointing to its head.
- A. CUBRID Master
- CUBRID Master runs as a daemonprocess named ‘cub_master’. It has two purposes:
- Registrar of CUBRID Servers.
CUBRID Server will register its information to CUBIRD Master.
- Listening to the TCP port for clients’ connection.
If clients, such as CSQL, CUBRID Broker or admin utilities, want to connect to CUBRID Server, first, they should connection to CUBIRD Master.
- B. CSS_CONN_ENTRY structure
- This data structure contains all the information about connection. It’s defined in src/connection/connection_defs.h. CSS_CONN_ENTRY contains several queues used to store data, request, or error from clients. For example:
- Request_queue is a list of requests from a client.
- Data_queue is a list of data from a client.
- Buffer_queue is a list of buffer which is provided by a user to receive data from a client.
- Data_wait_queue is a list of waiting threads. These threads wait for data from a client. They are blocked until data arrives.
- C. css_Master_conn
- There is a connection between CUBRID Server and CUBRID Master. The information about this connection is stored in css_Master_conn. css_Master_conn is the first CSS_CONN_ENTRY as shown in Figure 1-1.
The establishment of new connections
If a client wants to connect to CUBRID Server, first, it should connect to CUBRID Master.
- In Linux, CUBRID Master establishes a new connection. Then it sends the file descriptor of the new connection to CUBRID Server by passing descriptor between processes. CUBIRD Server receives it via css_Master_conn. Then it stores the file descriptor and related information into a free CSS_CONN_ENTRY and delinks this CSS_CONN_ENTRY from the free part and links to the active part.
- In Windows, CUBRID Master sends CUBRID Server’s port id to a client. Then the client connects to CUBRID Server directly. After that CUBIRD Server establishes the connection all by itself.
There are three kinds of threads in CUBRID Server: one master thread, several worker threads and six daemon threads. Between the master thread and worker threads there is a component called css_Job_queue. It is used to send jobs to worker threads.
css_Job_queue is used to manage jobs. In CUBRID Server, each job is described as a CSS_JOB_ENTRY. The master thread and a connection handler thread (a kind of worker thread) adds a css_job_entry into css_Job_queue. The worker thread which has no job will try to get a css_job_entry from this css_Job_queue. If the worker thread gets a css_job_entry, this thread will execute the request handling function in it.
- A. CSS_JOB_ENTRY
- CSS_JOB_ENTRY defined like this:
typedef struct css_job_entry CSS_JOB_ENTRY;
int jobq_index; /* job queue index */
CSS_CONN_ENTRY *conn_entry; /* conn entry from which we read request */
CSS_THREAD_FN func; /* request handling function */
CSS_THREAD_ARG arg; /* handling function argument */
- jobq_index indicates which JOB_QUEUE in css_Job_queue this CSS_JOB_ENTRY belongs to. The value should be between 0 ~ CSS_NUM_JOB_QUEUE - 1. jobq_index is assigned in function css_make_job_entry.
- conn_entry is the CSS_CONN_ENTRY from which the worker thread reads the request.
- func is the request handling function that the worker thread will execute.
- arg contains the argument for handling function.
- next is a pointer to the next CSS_JOB_ENTRY. It is used to organize all CSS_JOB_ENTRY as a linked-list.
- B. The structure of css_Job_queue CUBRID Cluster SQL Guide
- Below is the definition of css_Job_queue:
static JOB_QUEUE css_Job_queue[CSS_NUM_JOB_QUEUE]
- The value of CSS_NUM_JOB_QUEUE is 10
- Each JOB_QUEUE will hold a CSS_JOB_ENTRY list.
Figure 1-1 illustrates incomplete css_Job_queue structure. css_Job_queue should look like shown in Figure 1-2 as shown below.
- A. Master thread
- Master thread is used to create a new connection, accept and process a request from CUBRID master process or a client.
- When a new connection is established, master thread will add a CSS_JOB_ENTRY to css_Job_queue with the handling function css_connection_handler_thread. A worker thread which has no job will get the CSS_JOB_ENTRY, and then this worker thread will execute the handling function and work as a connection handler thread.
- The thread function of master thread is css_master_thread
- B. Worker thread
- In CUBRID server, there are more than one worker threads. The default number is 100. This value can be modified by setting the value max_clients in the cubrid.conf configuration file. The number of worker threads = max_clients * 2. The minimum value of max_clients is 10, the maximum value is 1024.
- Worker thread contains two types: connection handler thread and request handler thread. If a worker thread gets a job and the handling function of this job is css_connection_handler_thread, the worker thread will work as a connection handler thread; if the handling function is css_Request_handler, worker thread will work as a request handler thread.
- In CUBRID Server css_Request_handler is a global function pointer to css_internal_request_handler.
- As a request handler thread, the thread will call css_Server_request_handler to process requests from clients. All request types and reply functions are stored in the global variable net_Request. This variable is initialized in function net_server_init.
- If the job is finished, the worker thread will back to the no job state, the worker thread will try to get another CSS_JOB_ENTRY.
- As a connection handler thread, the thread will process all data/error/request/etc. from the connection which belongs to this thread. If the data arrives, the thread will put the data into data_queue. If another thread is waiting for the data, connection handler thread will wake up the waiting thread. If request arrives, the thread will put the request into request_queue and add a CSS_JOB_ENTRY to css_Job_queue with the handling function css_Request_handler.
- When the connection is closed or error occurs, connection handler thread will return back to no job state. The change of a worker thread state is shown below in Figure 1-3.
- C. Daemon thread
- There are six daemon threads:
It seems to be used to handle out-of-band data, but, currently, it is just a shell. The signal handler function dummy_sigurg_handler is an empty function.
This thread is used to detect the deadlock of transaction. Here is the way to detecting the deadlock:
- If the number of threads in lockwait state is greater than 1, create the directed graph Wait-For-Graph (in lock_detect_local_deadlock function). Check whether there are cycles in the graph. The number of cycles means how many deadlock.
- Here is the way to remove deadlock:
- For every cycle there is either timeout or abort transaction options available. The timeout option is always preferred over the unilaterally abort option. When the unilaterally abort option is exercised, the youngest transaction in the cycle is selected. The youngest transaction is hopefully the one that has done less work.
- The transaction which was selected to abort or timeout called victim, selected in function lock_select_deadlock_victim.
- Victim Selection Strategy:
- 1) Must be a lock holder.
- 2) Must be an active transaction.
- 3) Prefer a transaction with a closer timeout.
- 4) Prefer the youngest transaction.
- Macro LK_ISYOUNGER is to determin whether the transaction is younger. Defined as follow:
#define LK_ISYOUNGER(young_tranid, old_tranid) (young_tranid > old_tranid)
- D. Thread Management
- There is a global variable named thread_Manager. It is used for thread management. It is defined as the following:
static THREAD_MANAGER thread_Manager;
typedef struct thread_manager THREAD_MANAGER;
THREAD_ENTRY *thread_array; /* thread entry array */
- thread_array pointw to an array which contains all threads’ thread entry.
- num_total is a total number of threads.
- num_workers and num_daemons are the worker thread number and daemon thread number.
- initialized is used to mark whether thread_manager is initialized or not.
- thread_Manager is initalized in function thread_initialize_manager.