使用Python SDK写数据

本节将使用python代码直接往OpenSearch Serverless集群中写入数据。

下载python代码：

wget -O opensearch-client-generator.py 'https://static.us-east-1.prod.workshops.aws/public/59116165-bf19-4416-8767-2c6337e4197d/static/ingestion-pipelines/opensearch-client-generator.py'

此代码会自动生成运货卡车的示例数据，包括目的地，天气，速度，交通等字段：

"_source" : {
    "deviceId" : 2823605996,
    "fleetRegNo" : "IRV82MBYQ1",
    "oilLevel" : 0.92,
    "milesTravelled" : 1.105,
    "totalFuelUsed" : 0.01,
    "carrier" : "AOS Van Lines",
    "temperature" : 14,
    "tripId" : 6741375582,
    "originODC" : "ODC Las Vegas",
    "originCountry" : "United States",
    "originCity" : "Las Vegas",
    "originState" : "Nevada",
    "originGeo" : "36.16,-115.13",
    "destinationODC" : "ODC San Jose",
    "destinationCountry" : "United States",
    "destinationCity" : "San Jose",
    "destinationState" : "California",
    "destinationGeo" : "37.33,-121.89",
    "speedInMiles" : 18,
    "distanceMiles" : 382.81,
    "milesToDestination" : 381.705,
    "@timestamp" : "2022-11-17T17:11:25.855Z",
    "traffic" : "heavy",
    "weather_category" : "Cloudy",
    "weather" : "Cloudy"
}

安装依赖：

pip3 install requests_aws4auth
pip3 install boto3
pip3 install opensearch-py
pip3 install geopy

运行代码：

# python opensearch-client-generator.py -h hostname.aoss.amazonaws.com -r us-east-1
# 将hostname.aoss.amazonaws.com替换成集群实际的URL，比如pk1h0cp4ualg91vmq8h7.us-east-1.aoss.amazonaws.com 
python opensearch-client-generator.py -h pk1h0cp4ualg91vmq8h7.us-east-1.aoss.amazonaws.com  -r us-east-1

image-20230818100113080

进入到OpenSearch Dashboard，查询数据是否写入成功：

GET opensearch-client-ingest-*/_search
{
    "query": {
        "match_all": {}
    }
}

image-20230818100150039

创建dashboard

接下来我们将基于上面的数据，创建一个dashboard。下载模板文件：

https://static.us-east-1.prod.workshops.aws/public/59116165-bf19-4416-8767-2c6337e4197d/static/log-analytics-export.ndjson

点击Stack Management:

image-20230818110239877

选择Saved Objects:

image-20230818110228984

点击Import，选择刚才下载的ndjson文件：

image-20230818110441046

上传成功后打开dashboard（名为Octank），效果如下：

image-20230818111937575