Hi Aaron, After reading through the code, I finally figured out the issue. So back to your original question where you failed to run $>run denylistKey stackoverflow weather_sensor_data "'Minneapolis, MN',202210" #IllegalArgumentException: Operation denylistKey with 4 parameters doesn't exist in bean org.apache.cassandra.db:type=StorageProxy
It's not a Cassandra issue since it failed at the JMX parser stage, even before it goes to the Cassandra internal StorageProxy::denylistKey method. Yes, you got the right gist. It's because of the extra space between "Minneapolis" and "MN". To avoid the error, you need to add "\\" before the white space as below : $>run denylistKey "stackoverflow" "weather_sensor_data" "Minneapolis,\\ MN:202210" #calling operation denylistKey of mbean org.apache.cassandra.db:type=StorageProxy with params [stackoverflow, weather_sensor_data, Minneapolis, MN:202210] #operation returns: true And yes, the composite key is delimited by ":" . You can find more examples from the test code as it used composite keys https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/service/PartitionDenylistTest.java And then when you select the key, it will get denylisted: cqlsh> select ks_name, table_name, key from system_distributed.partition_denylist; ks_name | table_name | key ---------------+---------------------+------------------------------------------------------ stackoverflow | weather_sensor_data | 0x000f4d696e6e6561706f6c69732c204d4e000004000315e200 (2 rows) cqlsh> select * from stackoverflow.weather_sensor_data where city = 'Minneapolis, MN' AND month = 202210; InvalidRequest: Error from server: code=2200 [Invalid query] message="Unable to read denylisted partition [0xDecoratedKey(8132598509541575594, 000f4d696e6e6561706f6c69732c204d4e000004000315e200)] in stackoverflow/weather_sensor_data" cqlsh> For your second question why even a single partition key doesn't work, my apologies my previous answer on direct insert to the s ystem_distributed.partition_denylist like insert into system_distributed.partition_denylist (ks_name, table_name, key) values ('stackoverflow', 'weather_sensor_data', textAsBlob('''Minneapolis, MN'', 202210')); is wrong. It's because the way Cassandra stores the key in the system table is it converts the string into a hexadecimal literal. You can find the source code here https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/PartitionDenylist.java And check out the function addKeyToDenylist final String insert = String.format("INSERT INTO system_distributed. partition_denylist (ks_name, table_name, key) VALUES ('%s', '%s', 0x%s)", keyspace, table, ByteBufferUtil.bytesToHex(key)); You can see that it's different from the way of textAsBlob. A better (yet general) explanation is here https://stackoverflow.com/questions/31994880/why-cassandra-cql-query-with-hexadecimal-literal-works-but-textasbinary-does-not So... the short answer is, I get back my previous answer. My apologies! It's not recommended to directly insert rows into the system_distributed.partition_denylist table.You'd better go with the JMX interface since it will go to the StorageProxy which handles more logic like the hex string conversion and other error handling stuff. Hope the answer is helpful! Pls feel free to let me know if you have any questions! Thanks, Cheng On Thu, Oct 20, 2022 at 7:07 AM Aaron Ploetz <aaronplo...@gmail.com> wrote: > No worries, Cheng! > > So I actually pivoted a little and adjusted my example table to use a > single integer-based partition key. > > aaron@cqlsh:stackoverflow> SELECT ks_name, table_name, blobAsint(key) > FROM system_distributed.partition_denylist WHERE ks_name='stackoverflow' > AND table_name='weather_sensor_data_by_month'; > > ks_name | table_name | system.blobasint(key) > ---------------+------------------------------+----------------------- > stackoverflow | weather_sensor_data_by_month | 202210 > > But even with the yaml changes, it still allows me to SELECT that > partition. > > aaron@cqlsh:stackoverflow> SELECT * FROM weather_sensor_data_by_month > WHERE month=202210 LIMIT 1; > > month | city | recorded_time | temp > --------+----------------+---------------------------------+------ > 202210 | Minneapolis,MN | 2022-10-17 11:30:00.000000+0000 | 1 > > Just wondering what I should do to get it to deny access? > > Thanks, > > Aaron > > > > On Wed, Oct 19, 2022 at 5:34 PM Cheng Wang <che...@netflix.com> wrote: > >> Hi Aaron, >> >> Sorry for the late reply, was dealing with a production issue (maybe >> another topic for Cassandra Summit :-)). Are you running on your local >> machine? Then yes, you do need to enable the config for all the following >> >> enable_partition_denylist: true >> >> enable_denylist_writes: true >> enable_denylist_reads: true >> enable_denylist_range_reads: true >> >> I am not sure enable_partition_denylist will enable for the rest of three >> @Jordan >> West <jord...@netflix.com> maybe you can confirm for that? But it's >> better to enable for all just to be safe. >> I will play by my side and get back to you soon about the composite keys. >> >> Thanks >> Cheng >> >> On Wed, Oct 19, 2022 at 1:42 PM Aaron Ploetz <aaronplo...@gmail.com> >> wrote: >> >>> Just checking, but for this to work, do I have to mess with these >>> settings in the YAML at all? >>> >>> partition_denylist_enabled: true >>> denylist_reads_enabled: true >>> >>> They're commented out by default. >>> >>> Thanks, >>> >>> Aaron >>> >>> >>> On Mon, Oct 17, 2022 at 4:53 PM Aaron Ploetz <aaronplo...@gmail.com> >>> wrote: >>> >>>> Thanks for the help with the INSERT, Cheng! I'm further along than >>>> before. But it still must not be matching up quite right, because I can >>>> still select that partition. >>>> >>>> I have several different combinations of the two keys (and I removed >>>> the space) of "Minneapolis,MN" and 202210. Here's what I've got out there >>>> so far: >>>> >>>> aaron@cqlsh:stackoverflow> select ks_name, table_name, blobAsText(key) >>>> from system_distributed.partition_denylist; >>>> >>>> ks_name | table_name | system.blobastext(key) >>>> ---------------+---------------------+-------------------------- >>>> stackoverflow | weather_sensor_data | 'Minneapolis,MN', 202210 >>>> stackoverflow | weather_sensor_data | 'Minneapolis,MN',202210 >>>> stackoverflow | weather_sensor_data | 'Minneapolis,MN':202210 >>>> stackoverflow | weather_sensor_data | Minneapolis,MN, 202210 >>>> stackoverflow | weather_sensor_data | Minneapolis,MN202210 >>>> stackoverflow | weather_sensor_data | Minneapolis,MN:202210 >>>> >>>> (6 rows) >>>> >>>> aaron@cqlsh:stackoverflow> SELECT * FROM weather_sensor_data WHERE >>>> city='Minneapolis,MN' AND month=202210; >>>> >>>> city | month | recorded_time | temp >>>> ----------------+--------+---------------------------------+------ >>>> Minneapolis,MN | 202210 | 2022-10-17 11:30:00.000000+0000 | 1 >>>> Minneapolis,MN | 202210 | 2022-10-17 11:25:00.000000+0000 | 1 >>>> Minneapolis,MN | 202210 | 2022-10-17 11:20:00.000000+0000 | 1 >>>> Minneapolis,MN | 202210 | 2022-10-17 11:15:00.000000+0000 | 1 >>>> Minneapolis,MN | 202210 | 2022-10-17 11:10:00.000000+0000 | 2 >>>> Minneapolis,MN | 202210 | 2022-10-17 11:05:00.000000+0000 | 2 >>>> Minneapolis,MN | 202210 | 2022-10-17 11:00:00.000000+0000 | 2 >>>> >>>> (7 rows) >>>> >>>> As you can see, I can still select the partition. I was really hoping >>>> one of those combinations would do it. >>>> >>>> Looking at the StorageProxyTest.java in the project, I saw that it was >>>> delimited by a colon ":", which is why I tried that, too. >>>> >>>> Still looking for the right way to enter both of those keys. >>>> >>>> Thanks, >>>> >>>> Aaron >>>> >>>> >>>> On Mon, Oct 17, 2022 at 4:40 PM Cheng Wang <che...@netflix.com> wrote: >>>> >>>>> Another approach is, instead of using $$, you can put additional pair >>>>> of single quote around the 'Minneapolis, MN' >>>>> >>>>> cqlsh> insert into system_distributed.partition_denylist (ks_name, >>>>> table_name, key) values ('stackoverflow', 'weather_sensor_data', >>>>> textAsBlob('''Minneapolis, MN'', 202210')); >>>>> >>>>> cqlsh> select ks_name, table_name, blobAsText(key) from >>>>> system_distributed.partition_denylist; >>>>> >>>>> >>>>> ks_name | table_name | system.blobastext(key) >>>>> >>>>> ---------------+---------------------+--------------------------- >>>>> >>>>> stackoverflow | weather_sensor_data | 'Minneapolis, MN', 202210 >>>>> >>>>> On Mon, Oct 17, 2022 at 2:30 PM Cheng Wang <che...@netflix.com> wrote: >>>>> >>>>>> Hi Aaron, >>>>>> >>>>>> Yes, you can directly insert into the >>>>>> system_distributed.partition_denylist >>>>>> instead of using JMX. Jordan wrote a blog post for denylist >>>>>> >>>>>> https://cassandra.apache.org/_/blog/Apache-Cassandra-4.1-Denylisting-Partitions.html >>>>>> >>>>>> And the syntax error, one way around is to put $$ around like below >>>>>> >>>>>> cqlsh> insert into system_distributed.partition_denylist (ks_name, >>>>>> table_name, key) values ('stackoverflow', 'weather_sensor_data', >>>>>> textAsBlob($$'Minneapolis, MN', 202210$$)); >>>>>> >>>>>> There is post about this >>>>>> >>>>>> https://docs.datastax.com/en/cql-oss/3.3/cql/cql_reference/escape_char_r.html#:~:text=Column%20names%20that%20contain%20characters,using%20a%20single%20quotation%20mark >>>>>> . >>>>>> >>>>>> and then you can verify the insert by doing >>>>>> >>>>>> cqlsh> select ks_name, table_name, blobAsText(key) from >>>>>> system_distributed.partition_denylist; >>>>>> >>>>>> >>>>>> ks_name | table_name | system.blobastext(key) >>>>>> >>>>>> ---------------+---------------------+--------------------------- >>>>>> >>>>>> stackoverflow | weather_sensor_data | 'Minneapolis, MN', 202210 >>>>>> Pls let me know if it works for you. >>>>>> >>>>>> >>>>>> On Mon, Oct 17, 2022 at 1:35 PM Aaron Ploetz <aaronplo...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I have this table definition: >>>>>>> >>>>>>> CREATE TABLE stackoverflow.weather_sensor_data ( >>>>>>> city text, >>>>>>> month int, >>>>>>> recorded_time timestamp, >>>>>>> temp float, >>>>>>> PRIMARY KEY ((city, month), recorded_time) >>>>>>> ) WITH CLUSTERING ORDER BY (recorded_time DESC) >>>>>>> >>>>>>> Sample data looks like this: >>>>>>> >>>>>>> > SELECT * FROM weather_sensor_data WHERE city='Minneapolis, MN' AND >>>>>>> month=202111; >>>>>>> >>>>>>> city | month | recorded_time | temp >>>>>>> -----------------+--------+---------------------------------+------ >>>>>>> Minneapolis, MN | 202111 | 2021-11-01 08:35:00.000000+0000 | 3 >>>>>>> Minneapolis, MN | 202111 | 2021-11-01 08:30:00.000000+0000 | 3 >>>>>>> Minneapolis, MN | 202111 | 2021-11-01 08:25:00.000000+0000 | 2 >>>>>>> Minneapolis, MN | 202111 | 2021-11-01 08:20:00.000000+0000 | 2 >>>>>>> Minneapolis, MN | 202111 | 2021-11-01 08:15:00.000000+0000 | 2 >>>>>>> >>>>>>> (5 rows) >>>>>>> >>>>>>> Using JMX Term, I've tried to denylist that partition, but I must >>>>>>> have the syntax for composite keys incorrect: >>>>>>> >>>>>>> $>bean org.apache.cassandra.db:type=StorageProxy >>>>>>> $>run denylistKey stackoverflow weather_sensor_data "'Minneapolis, >>>>>>> MN',202210" >>>>>>> #IllegalArgumentException: Operation denylistKey with 4 parameters >>>>>>> doesn't exist in bean org.apache.cassandra.db:type=StorageProxy >>>>>>> >>>>>>> Obviously, it's reading the space between "Minneapolis," and "MN" as >>>>>>> a delimiter. What's the right way to handle commas, spaces, and >>>>>>> composite >>>>>>> keys for this? >>>>>>> >>>>>>> Also, is there another way to accomplish this without using JMX? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Aaron >>>>>>> >>>>>>> >>>>>>> >>>>>>>
