Question
CBA (Commonwealth Bank of Australia)
IN
Last activity: 29 Aug 2024 0:23 EDT
How many records can Cassandra Table can store?
How many records can Cassandra Table can store?
we are planning to design multiple (csv) file ingestion. As part of that we wanted to know how many records can a cassandra (staging) store.
Any suggestion can be helpful.
Thanks!
Accepted Solution
Updated: 29 Aug 2024 0:23 EDT
Pegasystems Inc.
GB
@Priyesh The number of records a Cassandra table can store depends on several factors, including the size of each record and the configuration of your Cassandra cluster. It is recommended to avoid writing large records to Cassandra, with the default maximum record size being 16MB. For DDS Data Sets, when the size of the data record exceeds 1MB, it triggers the PEGA0079 alert. Additionally, it is important to monitor the available disk space regularly and avoid distributing data unequally across nodes by limiting the size of data under a single partition key to 100MB. (SC)
Cassandra can store billions of records, but the actual capacity depends on your cluster's hardware and configuration. Each node can handle up to 1 TB of data, and you should allocate at least double the disk space of your data size. For efficient CSV file ingestion, use partitions for parallel processing and consider incremental updates.(sB)
⚠ This is a GenAI-powered tool. All generated answers require validation against the provided references.
Best practices for Cassandra disk space management
External Cassandra in your deployment > Recommended sizing
@Priyesh The number of records a Cassandra table can store depends on several factors, including the size of each record and the configuration of your Cassandra cluster. It is recommended to avoid writing large records to Cassandra, with the default maximum record size being 16MB. For DDS Data Sets, when the size of the data record exceeds 1MB, it triggers the PEGA0079 alert. Additionally, it is important to monitor the available disk space regularly and avoid distributing data unequally across nodes by limiting the size of data under a single partition key to 100MB. (SC)
Cassandra can store billions of records, but the actual capacity depends on your cluster's hardware and configuration. Each node can handle up to 1 TB of data, and you should allocate at least double the disk space of your data size. For efficient CSV file ingestion, use partitions for parallel processing and consider incremental updates.(sB)
⚠ This is a GenAI-powered tool. All generated answers require validation against the provided references.
Best practices for Cassandra disk space management
External Cassandra in your deployment > Recommended sizing