DynamoDB get_item 以毫秒为单位读取 400kb 数据

tl;dr：将您的函数内存增加到至少 1024MB，请参阅更新 2

我很好奇，所以我做了一些测量。我创建了一个脚本，用于在新表中创建一个大小几乎为 400KB 的大 boi 项。

然后我测试了 Python 的两次读取 - 一次使用资源 API，另一次使用较低级别的客户端 - 最终在两种情况下读取一致。

这是我测量的：

Reading Big Boi from a Table resource took 0.366508s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.301585s and consumed 50.0 RCUs

如果我们从 RCU 推断，它读取的项目大小约为 50 * 2 * 4KB = 400 KB（最终一致性读取消耗 0.5 个 RCU）。

我在德国本地针对 eu-central-1（德国法兰克福）运行了几次，我看到的最高延迟约为 900 毫秒。 （这没有 DAX。）

因此，我认为您应该向我们展示您是如何进行测量的。

import uuid
from datetiR_354_11845@e import datetiR_354_11845@e,timedelta

import boto3
import boto3.dynamodb.conditions as conditions

TABLE_NAME = "big-boi-test"
BIG_BOI_PK = "f0ba8d6c"

TABLE_resourcE = boto3.resource("dynamodb").Table(TABLE_Name)
DDB_CLIENT = boto3.client("dynamodb")

def create_table():
    DDB_CLIENT.create_table(
        AttributeDefinitions=[{"Attributename": "PK","AttributeType": "S"}],Tablename=TABLE_NAME,KeyscheR_354_11845@a=[{"Attributename": "PK","KeyType": "HASH"}],BillingMode="PAY_PER_requEST"
    )

def create_big_boi_item() -> str:
    # based on calculations here: https://zaccharles.github.io/dynamodb-calculator/
    template = {
        "PK": {
            "S": BIG_BOI_PK
        },"bigBoi": {
            "S": ""
        }
    } # This is 16 bytes

    big_boi = "X" * (1024 * 400 - 16)
    template["bigBoi"]["S"] = big_boi
    return template

def store_big_boi():
    big_bio = create_big_boi_item()

    DDB_CLIENT.put_item(
        Item=big_bio,Tablename=TABLE_NAME
    )

def geT_Big_boi_with_table_resource():

    start = datetiR_354_11845@e.now()
    response = TABLE_resourcE.get_item(
        Key={"PK": BIG_BOI_PK},ReturnConsumedCapacity="@R_279_10586@L"
    )
    end = datetiR_354_11845@e.now()
    seconds = (end - start).@R_279_10586@l_seconds()
    capacity_units = response["ConsumedCapacity"]["CapacityUnits"]

    print(f"Reading Big Boi from a Table resource took {seconds}s and consumed {Capacity_units} RCUs")

def geT_Big_boi_with_client():

    start = datetiR_354_11845@e.now()
    response = DDB_CLIENT.get_item(
        Key={"PK": {"S": BIG_BOI_PK}},ReturnConsumedCapacity="@R_279_10586@L",Tablename=TABLE_NAME
    )
    end = datetiR_354_11845@e.now()
    seconds = (end - start).@R_279_10586@l_seconds()
    capacity_units = response["ConsumedCapacity"]["CapacityUnits"]

    print(f"Reading Big Boi from a Client took {seconds}s and consumed {Capacity_units} RCUs")

if __name__ == "__main__":
    # create_table()
    # store_big_boi()
    geT_Big_boi_with_table_resource()
    geT_Big_boi_with_client()

更新

我对一个看起来更像您使用的那个的项目再次进行了相同的测量，无论我以何种方式请求它们，我的平均测量值仍然低于 1000 毫秒：

Reading Big Boi from a Table resource took 1.492829s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.871583s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.857513s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.769432s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.690172s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.670099s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.633489s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.605999s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.598635s and consumed 50.0 RCUs
Reading Big Boi from a Table resource took 0.606553s and consumed 50.0 RCUs
Reading Big Boi from a Client took 1.66636s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.921605s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.831735s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.707082s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.668602s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.648401s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.5695s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.592073s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.611436s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.553827s and consumed 50.0 RCUs
Average latency over 10 requests with the table resource: 0.7796304s
Average latency over 10 requests with the client: 0.7770621s

这是项目的样子：

这是供您验证的完整测试脚本：

import statistics
import uuid
from datetiR_354_11845@e import datetiR_354_11845@e,timedelta

import boto3
import boto3.dynamodb.conditions as conditions

TABLE_NAME = "big-boi-test"
BIG_BOI_PK = "nestedBoi"

TABLE_resourcE = boto3.resource("dynamodb").Table(TABLE_Name)
DDB_CLIENT = boto3.client("dynamodb")

def create_table():
    DDB_CLIENT.create_table(
        AttributeDefinitions=[{"Attributename": "PK",BillingMode="PAY_PER_requEST"
    )

def create_big_boi_item() -> str:
    # based on calculations here: https://zaccharles.github.io/dynamodb-calculator/
    template = {
        "PK": {
            "S": "nestedBoi"
        },"bigBoiContainer": {
            "M": {
            "bigBoiList": {
                "L": [
                
                ]
            }
            }
        }
    } # 43 bytes

    item = {
        "M": {
        "t": {
            "S": "1614712316"
        },"a": {
            "S": "product_view"
        },"i": {
            "S": "1275"
        }
        }
    }  # 36 bytes

    number_of_items = int((1024 * 400 - 43) / 36)

    for _ in range(number_of_items):
        template["bigBoiContainer"]["M"]["bigBoiList"]["L"].append(item)

    return template

def store_big_boi():
    big_bio = create_big_boi_item()

    DDB_CLIENT.put_item(
        Item=big_bio,ReturnConsumedCapacity="@R_279_10586@L"
    )
    end = datetiR_354_11845@e.now()
    seconds = (end - start).@R_279_10586@l_seconds()
    capacity_units = response["ConsumedCapacity"]["CapacityUnits"]

    print(f"Reading Big Boi from a Table resource took {seconds}s and consumed {Capacity_units} RCUs")

    return seconds

def geT_Big_boi_with_client():

    start = datetiR_354_11845@e.now()
    response = DDB_CLIENT.get_item(
        Key={"PK": {"S": BIG_BOI_PK}},Tablename=TABLE_NAME
    )
    end = datetiR_354_11845@e.now()
    seconds = (end - start).@R_279_10586@l_seconds()
    capacity_units = response["ConsumedCapacity"]["CapacityUnits"]

    print(f"Reading Big Boi from a Client took {seconds}s and consumed {Capacity_units} RCUs")

    return seconds

if __name__ == "__main__":
    # create_table()
    # store_big_boi()

    n_experiments = 10
    experiments_with_table_resource = [geT_Big_boi_with_table_resource() for i in range(n_experiments)]
    experiments_with_client = [geT_Big_boi_with_client() for i in range(n_experiments)]
    print(f"Average latency over {n_experiments} requests with the table resource: {statistics.mean(experiments_with_table_resourcE)}s")
    print(f"Average latency over {n_experiments} requests with the client: {statistics.mean(experiments_with_client)}s")

如果我增加 n_experiments，它往往会变得更快，可能是因为 DDB 内部缓存。

仍然：无法重现。

更新 2

在得知您正在运行 Lambda 函数后，我在具有不同内存配置的 Lambda 内部再次运行了测试。

如评论中所述，CPU 和网络性能随您分配给函数的内存量而变化。 你可以通过投钱来解决你的问题:-)

听起来您遇到了一些问题。第一个问题是您遇到了 400kb 项目大小限制。虽然您没有说这是一个问题，但可能值得重新审视您的数据模型，以便您可以存储更多事件数据。

性能问题不太可能与您的数据模型有关。 get_item 操作的平均延迟应为个位数毫秒，尤其是当您指定最终一致的读取时。这里还有其他事情要做。

您如何测试和衡量此操作的性能？

AWS 文档从 about troubleshooTing high latency DynamoDB operations 中提供了一些可能有用的建议。