Troubleshooting ROS Pushback
Determine how many containers you have per projection per node
**(*)** => SELECT node_name, schema_name, projection_name, sum(delete_vector_count/cnt)::int + count(*) container_count FROM storage_containers JOIN (select projection_id , count(*) cnt from projection_columns group by 1) as proj_cols on storage_containers.projection_id = proj_cols.projection_id WHERE storage_type = 'ROS' GROUP BY 1,2,3 ORDER BY 4 DESC;
Troubleshoting cases
You can find many troubleshoting cases in the referenced article, then follow the instruction for resolving the issue.
List of issues/troubleshoting cases:
Case 1: Long Running Mergeout Jobs
Case 2: Long Running Replay Deletes
Case 3: Too Many Partitions per Table
Case 4: Pending Partition Reorganize
Case 5: Frequent Loads into Inactive Partitions
Case 6: WOS Spillover
Reference to: https://www.vertica.com/kb/TroubleshootingROSPushback/Content/BestPractices/TroubleshootingROSPushback.htm
Self-cases
ERROR: Too many ROS containers exist for the following projections
[Vertica][VJDBC](5065) ERROR: Too many ROS containers exist for the following projections: sample_schema.sample_table_b0 (limit = 66234, ROS files = 103248, DV files = 0, new files = 48) sample_schema.sample_table_b1 (limit = 66234, ROS files = 68880, DV files = 0, new files = 48)
⇒ Check out the “Determine how many containers you have per projection per node” on the top of this page
⇒ Resolved by increasing ContainersPerProjectionLimit
, MaxPartitionCount
ALTER RESOURCE POOL tm PLANNEDCONCURRENCY 6 MAXCONCURRENCY 12 MAXMEMORYSIZE '10%'; SELECT SET_CONFIG_PARAMETER ('ContainersPerProjectionLimit','2048'); SELECT SET_CONFIG_PARAMETER ('MaxPartitionCount','2048');