Actually since the Directories.SSTableLister stores the Components in a
HashMap indexed by the Descriptor. Since the upgrade/downgrade code
retrieves the list in hash order there is no guarantee that they will be in
order. I suspect that this is a bug.
On Tue, Jan 10, 2023 at 12:34 PM Brandon Williams <dri...@gmail.com> wrote:
> > I think this means that the Directories.SSTableLister on occasion
> returns files in the incorrect order during a call to
> lister.list().entrySet()
>
> This seems easy enough to verify by looping it and examining the results.
>
> Kind Regards,
> Brandon
>
> On Tue, Jan 10, 2023 at 4:44 AM Claude Warren, Jr via dev
> <dev@cassandra.apache.org> wrote:
> >
> > Greetings,
> >
> > I am working on the downgradesstables code and seem to have a problem
> with ordering of the downgrade or perhaps the Directories.SSTableLister
> >
> > I lifted the code from upgradesstables to select the files to
> downgrade. The only difference in the code that selects the files to
> downgrade is the actual selection of the file. There is no change to the
> ordering of the files that are evaluated for inclusion. Yet I think the
> downgrade ordering is incorrect.
> >
> > My process is to start 3.1 version to create the tables and then use the
> 4.0 code base to run the standaloneupgrader and then the
> standalonedowngrader
> >
> > When running the standaloneupgrader on system local I see the following
> > {{noformat}}
> > Found 3 sstables that need upgrading.
> > Upgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-1-big-Data.db')
> > Upgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-1-big-Data.db')
> complete.
> > Upgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-2-big-Data.db')
> > Upgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-2-big-Data.db')
> complete.
> > Upgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-3-big-Data.db')
> > Upgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-3-big-Data.db')
> complete.
> > {{noformat}}
> >
> > when running the standalonedowngrader is see
> > {{noformat}}
> > Found 3 sstables that need downgrading.
> > Downgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-6-big-Data.db')
> > Downgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-6-big-Data.db')
> complete.
> > Downgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-4-big-Data.db')
> > Downgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-4-big-Data.db')
> complete.
> > Downgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-5-big-Data.db')
> > Downgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-5-big-Data.db')
> complete.
> > {{noformat}}}
> >
> > Note the order of the generations in the downgrader (I have seen similar
> out of order issues with the upgrader, but infrequently)
> >
> > The difference between the upgrader and downgrader code in the
> questionable section (
> https://github.com/Claudenw/cassandra/blob/CASSANDRA-8928/src/java/org/apache/cassandra/tools/StandaloneDowngrader.java#:~:text=new%20ArrayList%3C%3E()%3B-,//%20Downgrade%20sstables,%7D,-int%20numSSTables%20%3D)
> is on line 101 where the files are selected and put into a list. I think
> this means that the Directories.SSTableLister on occasion returns files in
> the incorrect order during a call to lister.list().entrySet()
> >
> > I believe that the files are processed in the order specified and that
> the generations get switched around. This is evidenced by the file size of
> the Data file associated with the generations as it moves through the
> process. In this case we expect the nb-6 to become ma-7 as per the output
> from the run. In actuality we want nb-6 to be nb-9.
> >
> > {{noformat}}
> > ma nb ma
> > 1 212 4 212 8 212
> > 2 64 5 70 9 70
> > 3 4876 6 4883 7 4883
> > {{noformat}}
> >
> > So now my question, has anyone seen the behaviour before?
> >
> > Oh, to make things more interesting I am using Docker images of 3.1 and
> a modified 4.0 that turns off the execution so I can just run the upgrade
> and downgrade.
> >
> > Any help would be appreciated,
> > Claude
>
