How to consume

Retrieve news data

Query the news data we just wrote, here we show submitting a SQL statement through the gateway to query news, as well as a news retrieval tool implemented based on vector databases.

The gateway query interface protocol is as follows:

Parameters:

• dataset_name(string) The dataset to query, required, here is "vec"
• sql(string) The SQL statement to execute, the SQL specifies search conditions here on content_vector
• arguments(list) Placeholders in the SQL statement, here is the search term.

The user sends the request to the gateway. The gateway routes the request based on the dataset registration information and forwards the request to the correct engine-server. The engine-server then executes the actual vector retrieval.

CURL

curl --location --request POST 'https://orlando-gateway.glitterprotocol.tech/v1/sql/query' \
--data-raw '{
  "dataset_name": "vec",
   "sql": "SELECT id,url,title,VECTOR_L2_DISTANCE(content_vector,TEXT_TO_VEC(?,'\''text-embedding-3-small'\'')) AS distance FROM vec.news_demo ORDER BY distance LIMIT 2",
   "arguments": [
      {
         "type": "STRING",
         "value": "Reasons to buy iPhone 16"
      }
   ]
}'

The expected return is as follows, with the results sorted by the vector distance and returning the fields specified in the select clause: id, title and url:

Json

{
  "result": [
    {
      "row": {
        "distance": {
          "value": "0.89962983"
        },
        "id": {
          "value": "1853255430555989830"
        },
        "title": {
          "value": "Three reasons to choose iPhone 16 Pro over the iPhone 16"
        },
        "url": {
          "value": "https://9to5mac.com/?p=967169"
        }
      }
    },
    {
      "row": {
        "distance": {
          "value": "1.0079896"
        },
        "id": {
          "value": "1853202186926754931"
        },
        "title": {
          "value": "You Can Now 'Get Ready' for iPhone 16 Launch With Pre-Order Setup"
        },
        "url": {
          "value": "https://www.macrumors.com/2024/09/09/iphone-16-get-ready-pre-order/"
        }
      }
    }
  ],
  "code": 0,
  "full_took_times": 0.6012932,
  "trace_id": "49a24f24-bcf6-484f-956e-2d00bf28808d"
}

News Retrieval Tool

This is a news retrieval tool developed based on the code in the demo.If you wish to explore the full product, please click here.
Input the news you want to find, and it will return the titles of matching news articles:

Data Flow

Below is the process of executing a retrieval request:

1. The user sends a request to the app, such as the news application in the demo.
2. The app constructs an SQL query statement according to the user's request and submits it to the gateway.
3. Upon receiving the request, the gateway routes it based on the dataset name and registration information in the request to select the correct engine-server.
4. After receiving the request, the engine-server first selects the specified embedding-provider based on the model name specified in the SQL statement, and vectorizes the retrieved words through this provider.
5. After obtaining the vectorization result, the engine-server constructs a vector search request and requests the Milvus engine to carry out the actual vector retrieval process.
6. The Milvus engine returns the specific retrieval data.