Here is my response. <https://issues.apache.org/jira/browse/CASSANDRA-8332> T Jake Luciani Null pointer after droping keyspace Add/drop keyspace not tested under load, with server logs checked for errors
To detect this we would need a test to write to a table, wait for compaction to start then drop the table before the compaction finished and check the log for an exception (there was no impact to the system other than the error) On Thu, Apr 9, 2015 at 4:05 PM, Tyler Hobbs <ty...@datastax.com> wrote: > Here are my responses for my tickets: > > CASSANDRA-7910 - wildcard prepared statements are incorrect after a column > is added to the table > > As part of the ticket, I added general dtests for prepared statement > invalidation after ALTER TABLE statements: > https://github.com/riptano/cassandra-dtest/blob/18cd4adaba65a9f424b5b9f52f4bf510b6c7e47f/cql_tests.py#L5170-L5204. > In general, though, the dtests don't have great coverage on prepared > statements. We could add an abstraction layer over query execution to use > prepared vs unprepared statements with an environment variable. > > CASSANDRA-8264 - Problems with multicolumn relations and COMPACT STORAGE > > I'm not sure of the best way to catch missing coverage for table settings > like COMPACT STORAGE is. Theoretically a careful inspection of code/branch > coverage would probably reveal this. Of course, we could also ensure all > (applicable) tests cover both COMPACT and non-COMPACT, but other table > settings like the compaction strategy and caching could also affect some > tests. I suppose COMPACT STORAGE is the most likely to make a difference, > so perhaps we should focus on covering that first and postpone coverage for > the other attributes until we have a more general solution? > > CASSANDRA-8286 - Regression in ORDER BY > > The dtests did catch this. We did not ship the bug in 2.0, but > unfortunately, it did make it into the 2.1.2 release despite the failing > test. (The test failed shortly before the release, which had its vote over > the weekend and thus missed test-engineering input.) After the 2.1.2 > release, we decided to make changes to the release process to prevent that: > ensure a full test run completes before voting, and require sign-off from a > test engineer before releasing. > > CASSANDRA-8288 - cqlsh describe needs to show 'sstable_compression' > > We do have round-trip tests for DESCRIBE in the python driver ( > https://github.com/datastax/python-driver/blob/c381fa0f04dbeb9ac389fa017bac8d9cc2ded0be/tests/integration/standard/test_metadata.py#L41), > but they don't cover every config option permutation. Although we now have > coverage for this particular case, adding a suite of tests for this should > be pretty straightforward, so I'll open a ticket for that. > > CASSANDRA-8302 - Filtering for CONTAINS (KEY) on frozen collection > clustering columns within a partition does not work > > This bug didn't ship, but wasn't caught by a failing test. I'm not sure > how we could spot the lack of coverage on this -- even branch coverage > analysis wouldn't show this. Perhaps taking the time to manually create a > test-coverage matrix (and post it in jira) when developing the feature is > the best approach? The matrix is too large to simply cover mentally, it > seems. > > CASSANDRA-8408 - limit appears to replace page size under certain conditions > > This was discovered while more thorough tests for paging were being written > (by the reporter). Obviously, these should have existed when the feature > was written, but part of the problem was that it required driver changes to > test and we weren't using the native protocol python driver for dtests > yet. We still have a bit of a problem with testing new protocol features, > but we typically add python driver support for the features while > developing them so that we can at least run relevant dtests locally. Once > https://github.com/riptano/cassandra-dtest/issues/188 is complete, it > should be easier to write dtests against native protocol changes/features. > > CASSANDRA-8410 - Select with many IN values on clustering columns can > result in a StackOverflowError > > Agreed, we need boundary tests for large values of IN clauses, large > inserts/update/deletes/batches (number of query params, number of partition > keys, and number of clustering keys) > > CASSANDRA-8451 - NPE when writetime() or ttl() are nested inside function > call > > I did add a regression dtest for this. We do have tests for composition of > functions, but writetime() and ttl() are special functions and didn't have > the specific test coverage they deserved. (I did add dtest coverage as > part of the ticket.) > > CASSANDRA-8490 - DISTINCT queries with LIMITs or paging are incorrect when > partitions are delete > > We did have test coverage for this query in general, but the tests didn't > cover deletes. I think that in general, our paging tests should all > exercise deletes (of partitions, rows, and individual cells). I propose > opening a jira ticket for this. > > CASSANDRA-8512 - cqlsh unusable after encountering schema mismatch > > I'm not sure what to do here. Test cqlsh with various clusters in degraded > states? In general, cqlsh is way undertested -- we need some better > infrastructure for making it testable. > > CASSANDRA-8550 - Internal pagination in CQL3 index queries creating > substantial overhead > > I'm not generally involved in performance work, so I'm not familiar with > what tools we have here. Have we made progress towards a standard set of > benchmarks for various queries and configurations? > > CASSANDRA-8563 - cqlsh broken for some thrift created tables > > This is one of the ugly (and rarely used) corners of Thrift where CQL > compatibility isn't great. We should probably create a test suite of the > most bizarre Thrift schemas and confirm that CQL compatibility works. > > > On Thu, Apr 9, 2015 at 1:45 PM, Ariel Weisberg <ariel.weisb...@datastax.com> > wrote: > >> Repeated with sort >> *Key* *Assignee* *Summary* *Revisit reason* CASSANDRA-8285 >> <https://issues.apache.org/jira/browse/CASSANDRA-8285> Aleksey Yeschenko >> Move >> all hints related tasks to hints private executor Pierre's reproducer >> represents something we weren't doing, but that users are. Is that now >> being tested? CASSANDRA-8462 >> <https://issues.apache.org/jira/browse/CASSANDRA-8462> Aleksey >> Yeschenko Upgrading >> a 2.0 to 2.1 breaks CFMetaData on 2.0 nodes Have additional dtest coverage, >> need to do this in kitchen sink tests CASSANDRA-8640 >> <https://issues.apache.org/jira/browse/CASSANDRA-8640> Anthony Cozzie >> Paxos >> requires all nodes for CAS If PAXOS is not supposed to require all nodes >> for CAS we should be able to fail nodes or a certain number of nodes and >> still continue to CAS (test availability of CAS under failure conditions). >> No regression test. CASSANDRA-8677 >> <https://issues.apache.org/jira/browse/CASSANDRA-8677> Ariel Weisberg >> rpc_interface >> and listen_interface generate NPE on startup when specified interface >> doesn't exist Missing unit tests checking error messages for >> DatabaseDescriptor CASSANDRA-8577 >> <https://issues.apache.org/jira/browse/CASSANDRA-8577> Artem Aliev Values >> of set types not loading correctly into Pig Full set of interactions with >> PIG not validated CASSANDRA-7704 >> <https://issues.apache.org/jira/browse/CASSANDRA-7704> Benedict >> FileNotFoundException >> during STREAM-OUT triggers 100% CPU usage Streaming testing didn't >> reproduce this before release CASSANDRA-8383 >> <https://issues.apache.org/jira/browse/CASSANDRA-8383> Benedict Memtable >> flush may expire records from the commit log that are in a later memtable >> No >> regression test, no follow up ticket. Could/should this have been >> reproducable as an actual bug? CASSANDRA-8429 >> <https://issues.apache.org/jira/browse/CASSANDRA-8429> Benedict Some keys >> unreadable during compaction Running stress in CI would have caught this, >> and we're going to do that CASSANDRA-8459 >> <https://issues.apache.org/jira/browse/CASSANDRA-8459> Benedict >> "autocompaction" >> on reads can prevent memtable space reclaimation What would have reproduced >> this before release? CASSANDRA-8499 >> <https://issues.apache.org/jira/browse/CASSANDRA-8499> Benedict Ensure >> SSTableWriter cleans up properly after failure Testing error paths? Any way >> to test things in a loop to detect leaks? CASSANDRA-8513 >> <https://issues.apache.org/jira/browse/CASSANDRA-8513> Benedict >> SSTableScanner >> may not acquire reference, but will still release it when closed This had a >> user visible component, what test could have caught it befor erelease? >> CASSANDRA-8619 <https://issues.apache.org/jira/browse/CASSANDRA-8619> >> Benedict using CQLSSTableWriter gives ConcurrentModificationException What >> kind of test would have caught this before release? CASSANDRA-8632 >> <https://issues.apache.org/jira/browse/CASSANDRA-8632> Benedict >> cassandra-stress >> only generating a single unique row We rely on stress for performance >> testing, that might mean it needs real testing that demonstrates it >> generates load that looks like the load it is supposed to be generating. >> CASSANDRA-8668 <https://issues.apache.org/jira/browse/CASSANDRA-8668> >> Benedict We don't enforce offheap memory constraints; regression introduced >> by 7882 Memory constraints was a supported feature/UI, but not completely >> tested before release. Could this have been found most effectively by a >> unit test or a blackbox test? CASSANDRA-8719 >> <https://issues.apache.org/jira/browse/CASSANDRA-8719> Benedict Using >> thrift HSHA with offheap_objects appears to corrupt data Untested >> configuration before release, this would be straightforward if we ran with >> it? CASSANDRA-8726 <https://issues.apache.org/jira/browse/CASSANDRA-8726> >> Benedict throw OOM in Memory if we fail to allocate OOM test Cassandra? Try >> and validate that it fails cleanly and can be restarted on OOM? Same for >> disk full. CASSANDRA-8018 >> <https://issues.apache.org/jira/browse/CASSANDRA-8018> Benjamin Lerer >> Cassandra >> seems to insert twice in custom PerColumnSecondaryIndex Custom secondary >> indexes not tested before release? CASSANDRA-8231 >> <https://issues.apache.org/jira/browse/CASSANDRA-8231> Benjamin Lerer >> Wrong >> size of cached prepared statements Expected cache capacity not validated >> with actual cache capcaity, no regression test CASSANDRA-8365 >> <https://issues.apache.org/jira/browse/CASSANDRA-8365> Benjamin Lerer >> CamelCase >> name is used as index name instead of lowercase How can we establish UI >> consistency? CASSANDRA-8421 >> <https://issues.apache.org/jira/browse/CASSANDRA-8421> Benjamin Lerer >> Cassandra >> 2.1.1 & Cassandra 2.1.2 UDT not returning value for LIST type as UDT Is >> there a test that could have found this condition before release? >> CASSANDRA-8514 <https://issues.apache.org/jira/browse/CASSANDRA-8514> >> Benjamin >> Lerer ArrayIndexOutOfBoundsException in nodetool cfhistograms Not released, >> but not caught by automated tests either CASSANDRA-8243 >> <https://issues.apache.org/jira/browse/CASSANDRA-8243> Björn Hegerfors >> DTCS >> can leave time-overlaps, limiting ability to expire entire SSTables >> Performance >> improving fast path not tested in a representative way CASSANDRA-8448 >> <https://issues.apache.org/jira/browse/CASSANDRA-8448> Brandon >> Williams "Comparison >> method violates its general contract" in AbstractEndpointSnitch This just >> happens periodically? Was the snitch no tested under load and the log >> output checked for errors? CASSANDRA-8028 >> <https://issues.apache.org/jira/browse/CASSANDRA-8028> Carl Yeksigian >> Unable >> to compute when histogram overflowed Histogram output not tested with >> representative data sets, no regression test CASSANDRA-8122 >> <https://issues.apache.org/jira/browse/CASSANDRA-8122> Carl Yeksigian >> Undeclare >> throwable exception while executing 'nodetool netstats localhost' nodetool >> not tested against cluster throughout lifecycle, no regression test >> CASSANDRA-8695 <https://issues.apache.org/jira/browse/CASSANDRA-8695> >> Chris >> Lockfort thrift column definition list sometimes immutable What user >> visible activities reproduced this, could we have done that before release? >> CASSANDRA-8588 <https://issues.apache.org/jira/browse/CASSANDRA-8588> Dave >> Brosius Fix DropTypeStatements isusedBy for maps (typo ignored values) Not >> released, but was it detected before release by an automated test? >> CASSANDRA-8652 <https://issues.apache.org/jira/browse/CASSANDRA-8652> >> Edward >> Ribeiro DROP TABLE should also drop BATCH prepared statements associated to >> it Not sure if this is an optimization or fixes a user visible issue, but >> could this have been detected by exercising the functionality better before >> release. CASSANDRA-8694 >> <https://issues.apache.org/jira/browse/CASSANDRA-8694> Jeff Jirsa Repair >> of >> empty keyspace hangs rather than ignoring the request Missing boundary >> condition test, requesting operation on empty, non-existent, or not >> applicable entity. CASSANDRA-8687 >> <https://issues.apache.org/jira/browse/CASSANDRA-8687> Jeremiah Jordan >> Keyspace >> should also check Config.isClientMode Is there a way to test for missing >> Config.isClientMode checks? CASSANDRA-8579 >> <https://issues.apache.org/jira/browse/CASSANDRA-8579> Jimmy Mårdell >> sstablemetadata >> can't load org.apache.cassandra.tools.SSTableMetadataViewer Running C* from >> source tree not representative of behavior of deployed builds >> CASSANDRA-8401 <https://issues.apache.org/jira/browse/CASSANDRA-8401> >> Jonathan >> Ellis dropping a CF doesn't remove the latency-sampling task Another >> argument for a schema change stress test, maybe tracking for constant >> memory utilization CASSANDRA-8292 >> <https://issues.apache.org/jira/browse/CASSANDRA-8292> Joshua McKenzie >> From >> Pig: org.apache.cassandra.exceptions.ConfigurationException: Expecting URI >> in variable: [cassandra.config]. Please prefix the file with file:/// for >> local files or file://<server>/ for remote files. PIG not tested >> CASSANDRA-8211 <https://issues.apache.org/jira/browse/CASSANDRA-8211> >> Marcus >> Eriksson Overlapping sstables in L1+ Noted hard to reproduce, but still is >> there a way we could have, no regression test CASSANDRA-8316 >> <https://issues.apache.org/jira/browse/CASSANDRA-8316> Marcus Eriksson >> "Did >> not get positive replies from all endpoints" error on incremental repair >> What >> were users doing differently, is there a reproducer for this running now? >> CASSANDRA-8320 <https://issues.apache.org/jira/browse/CASSANDRA-8320> >> Marcus >> Eriksson 2.1.2: NullPointerException in SSTableWriter What were users doing >> that caused this, are we doing that? CASSANDRA-8386 >> <https://issues.apache.org/jira/browse/CASSANDRA-8386> Marcus Eriksson >> Make >> sure we release references to sstables after incremental repair Is there a >> higher level test that could have observed this failure? CASSANDRA-8432 >> <https://issues.apache.org/jira/browse/CASSANDRA-8432> Marcus Eriksson >> Standalone >> Scrubber broken for LCS Standalone scrubber not tested, no regression test >> CASSANDRA-8458 <https://issues.apache.org/jira/browse/CASSANDRA-8458> >> Marcus >> Eriksson Don't give out positions in an sstable beyond its first/last >> tokens Streaming >> not done in realistic scenario with validation of logging CASSANDRA-8463 >> <https://issues.apache.org/jira/browse/CASSANDRA-8463> Marcus Eriksson >> Constant >> compaction under LCS What would have reproduced this before release? >> CASSANDRA-8510 <https://issues.apache.org/jira/browse/CASSANDRA-8510> >> Marcus >> Eriksson CompactionManager.submitMaximal may leak resources Not a user >> visible problem, so difficult to catch in test, but is there a way >> CASSANDRA-8525 <https://issues.apache.org/jira/browse/CASSANDRA-8525> >> Marcus >> Eriksson Bloom Filter truePositive counter not updated on key cache hit >> User >> visible metric not accurate, but only in one config. Possible to guess >> correct FP ratio and validate while exploring config space? CASSANDRA-8532 >> <https://issues.apache.org/jira/browse/CASSANDRA-8532> Marcus Eriksson Fix >> calculation of expected write size during compaction Did this manifest as a >> user visible issue, could we have tested for that? CASSANDRA-8537 >> <https://issues.apache.org/jira/browse/CASSANDRA-8537> Marcus Eriksson >> ConcurrentModificationException >> while executing 'nodetool cleanup' Nodetool cleanup not tested before >> release CASSANDRA-8562 >> <https://issues.apache.org/jira/browse/CASSANDRA-8562> Marcus Eriksson Fix >> checking available disk space before compaction starts Is there a user >> visible negative impact, could it have been tested for? CASSANDRA-8580 >> <https://issues.apache.org/jira/browse/CASSANDRA-8580> Marcus Eriksson >> AssertionErrors >> after activating unchecked_tombstone_compaction with leveled compaction How >> could this have been reproduced before release? No regression test >> CASSANDRA-8623 <https://issues.apache.org/jira/browse/CASSANDRA-8623> >> Marcus >> Eriksson sstablesplit fails *randomly* with Data component is missing >> Feature >> not tested before release? No regression test CASSANDRA-8635 >> <https://issues.apache.org/jira/browse/CASSANDRA-8635> Marcus Eriksson >> STCS >> cold sstable omission does not handle overwrites without reads If this >> workload is a challenge for certain kinds of optimizations we should test >> it if we think it could happen again. CASSANDRA-7538 >> <https://issues.apache.org/jira/browse/CASSANDRA-7538> Sam Tunnicliffe >> Truncate >> of a CF should also delete Paxos CF Truncate not tested with PAXOS, what >> else? CASSANDRA-8280 >> <https://issues.apache.org/jira/browse/CASSANDRA-8280> Sam >> Tunnicliffe Cassandra crashing on inserting data over 64K into indexed >> strings Added tests are good example, could focusing on testing all access >> paths and boundary conditions per access path have prevented this >> CASSANDRA-8370 <https://issues.apache.org/jira/browse/CASSANDRA-8370> Sam >> Tunnicliffe cqlsh doesn't handle LIST statements correctly cqlsh untested >> functionality, no regression test? CASSANDRA-7801 >> <https://issues.apache.org/jira/browse/CASSANDRA-7801> Sylvain Lebresne A >> successful INSERT with CAS does not always store data in the DB after a >> DELETE Multiple access paths for data not tested together CASSANDRA-8558 >> <https://issues.apache.org/jira/browse/CASSANDRA-8558> Sylvain Lebresne >> deleted >> row still can be selected out Validate that deleted data stays deleted >> under * conditions (big matrix of interactions here with different >> configurations, streaming, repair, cleanup, scrub). Deleted data coming >> back shows up a lot. CASSANDRA-8332 >> <https://issues.apache.org/jira/browse/CASSANDRA-8332> T Jake Luciani Null >> pointer after droping keyspace Add/drop keyspace not tested under load, >> with server logs checked for errors CASSANDRA-7910 >> <https://issues.apache.org/jira/browse/CASSANDRA-7910> Tyler Hobbs >> wildcard >> prepared statements are incorrect after a column is added to the table >> Alter >> table not tested concurrently with ? CASSANDRA-8264 >> <https://issues.apache.org/jira/browse/CASSANDRA-8264> Tyler Hobbs >> Problems >> with multicolumn relations and COMPACT STORAGE How can we catch >> interactions like compact storage not being covered by the test >> CASSANDRA-8286 <https://issues.apache.org/jira/browse/CASSANDRA-8286> >> Tyler >> Hobbs Regression in ORDER BY There were tests that failed in some versions, >> but not all? Did this not ship? CASSANDRA-8288 >> <https://issues.apache.org/jira/browse/CASSANDRA-8288> Tyler Hobbs cqlsh >> describe needs to show 'sstable_compression': '' Roundtrip test for >> describe schema? CASSANDRA-8302 >> <https://issues.apache.org/jira/browse/CASSANDRA-8302> Tyler Hobbs >> Filtering >> for CONTAINS (KEY) on frozen collection clustering columns within a >> partition does not work More untested combinations, could we have spotted >> that there was an interaction and tested it? Or did this not ship? >> CASSANDRA-8408 <https://issues.apache.org/jira/browse/CASSANDRA-8408> >> Tyler >> Hobbs limit appears to replace page size under certain conditions No test >> that validates that paging returns the expected number of results? Another >> of the genre of queries we support but don't test all the combinations >> CASSANDRA-8410 <https://issues.apache.org/jira/browse/CASSANDRA-8410> >> Tyler >> Hobbs Select with many IN values on clustering columns can result in a >> StackOverflowError Another missing boundary conditions test, test maximum >> size in clause against * CASSANDRA-8451 >> <https://issues.apache.org/jira/browse/CASSANDRA-8451> Tyler Hobbs NPE >> when >> writetime() or ttl() are nested inside function call Is this testable? Can >> we check that functions compose correctly or validate that they are >> inherently composable. No regression test. CASSANDRA-8490 >> <https://issues.apache.org/jira/browse/CASSANDRA-8490> Tyler Hobbs >> DISTINCT >> queries with LIMITs or paging are incorrect when partitions are >> deleted Untested >> query forms, no regression test CASSANDRA-8512 >> <https://issues.apache.org/jira/browse/CASSANDRA-8512> Tyler Hobbs cqlsh >> unusable after encountering schema mismatch cqlsh not tested with other >> functionality active CASSANDRA-8550 >> <https://issues.apache.org/jira/browse/CASSANDRA-8550> Tyler Hobbs >> Internal >> pagination in CQL3 index queries creating substantial overhead Pagination >> not performance tested with representative data models CASSANDRA-8563 >> <https://issues.apache.org/jira/browse/CASSANDRA-8563> Tyler Hobbs cqlsh >> broken for some thrift created tables. Validate mixed CQL thrift >> interactions? Possibly abstract everything to be done either by CQL or >> Thrift and then permute? Seems low value, but necessary if both are claimed >> to be supported. CASSANDRA-8733 >> <https://issues.apache.org/jira/browse/CASSANDRA-8733> Tyler Hobbs List >> prepend reverses item order There was a test so sometimes this just >> happens. >> CASSANDRA-8641 <https://issues.apache.org/jira/browse/CASSANDRA-8641> >> *Unassigned* Repair causes a large number of tiny SSTables User says >> something doesn't work for them? Could we have anticipated that vnodes >> would not work as formulated for this case. CASSANDRA-8675 >> <https://issues.apache.org/jira/browse/CASSANDRA-8675> *Unassigned* COPY >> TO/FROM broken for newline characters COPY TO/FROM not tested with >> representative data CASSANDRA-8691 >> <https://issues.apache.org/jira/browse/CASSANDRA-8691> *Unassigned* >> SSTableReader.getPosition() >> does not correctly filter out queries that exceed its bounds Is there a >> scenario where this is user visible, should we test for that? >> CASSANDRA-8688 <https://issues.apache.org/jira/browse/CASSANDRA-8688> Yuki >> Morishita Standalone sstableupgrade tool throws exception Tool not tested >> before release, no regression test >> > > > > -- > Tyler Hobbs > DataStax <http://datastax.com/> -- http://twitter.com/tjake
