Skip to main content

Glitter Engine

In order to adapt to the diverse data structures and complex data retrieval requirements of the web3 world, Glitter's engine layer is designed in a plug-and-play mode, making it convenient to expand the underlying capabilities in the future. Access to and execution of the underlying engine can be performed through SQL, and access control can be achieved through RBAC(Role-Based policies Access Control). Currently, Glitter's Index Storage Layer has mainly implemented two types of indexing engines:

  • standard functions similar to traditional relational databases, providing basic indexing capabilities such as primary key indexes, secondary indexes, and composite indexes.
  • full_text functions similar to search engines such as Elasticsearch, providing full-text search capability.

Glitter Engine Module​

  • SQL Parser: Verify and parse SQL statements, and extract information such as table names.
  • Access Control: Access control that verifies if users have operational permissions.
  • Metadata Store: Storage for centrally managed metadata, such as table definitions and permission roles.
  • Executor: Call the Engine API to execute AST, encapsulate the return result, and coordinate the consistency between Proxy and Engine.
  • Engine Interface: Define the Engine interface, register, and discover engine implementations.

Search

Glitter Execution Flow​

After a user initiates a transaction, it will be broadcasted to Validator nodes on Glitter-Chain. Typically, a transaction includes at least the following information:

  • SQL: Description of the statement to be executed.
  • Arguments: Execution argument(s).
  • Address: The user's address.

These messages are processed in the Keeper layer, which first attempts to parse the SQL to extract an AST(abstract syntax tree) using the SQL Parser module.

If the SQL statement is parsed successfully, the table information is extracted from the AST, then combined with user information to perform RBAC(role-based access control) verification in the Access Control module.

If the access is verified, the SQL statement will be executed by the Executor module. The executor obtains the corresponding execution engine from the table's metadata and dispatches the SQL statement to the corresponding engine for execution. Before the execution, semantic validation is performed to eliminate unsupported statements or any runtime errors found through pre-execution.

For DDL-type statements, the Executor module is also responsible for coordinating the consistency between the engine and Glitter's metadata.

Finally, the Executor module processes the execution results and returns them to the Keeper.

Search

Standard Engine​

Standard Engine is a MySQL 5.7 protocol-compatible engine that stores data in KV(a form of key-value). The design makes it easy for developers to migrate, reducing the cost of learning and understanding

SQL Engine has implemented most of the features of the MySQL 5.7 protocol, and also added blockchain-related features. Therefore, SQL Engine not only has the excellent features of the MySQL 5.7 protocol but also provides additional functionality for blockchain applications.

For SQL support please refer to SQL Compatibility

Full_Text Engine​

Glitter full_text Engine is a search engine similar to Elasticsearch. Implementing a complete search engine is a challenge, and currently, Glitter full_text engine is relatively simple. However, it already has the core functionality of a Full-Text search database and supports SQL for data management, which reduces the threshold for application developers to use.

The main core capabilities are as follows:

  • For table creation: tables can be created through SQL and multiple word segmenters are supported. For more examples, please refer to Create Table.

  • For querying: SQL queries are supported, and QueryString syntax is extended for users to conduct customized queries. Additionally, support for highlighting, sorting, and scoring explanation fields is provided. For more information, please refer to Search Query.