Re: Running 4.0 (trunk) on Windows

[Joshua McKenzie]

>
>  there is no plan from Cassandra Devs to normally support Windows in
> Cassandra 4.0

I was the dev kind of single-handedly making it work on Windows; it was
good up through the 3.X line but I didn't maintain it past that point to
4.0+ and don't have plans to. Reason being: the WSL approach 100% works for
dev environments, and while I/O is pretty slow right now (i.e. use for dev
only) there's a new version of Windows hitting soon with an actual linux
kernel embedded in it[1] where the I/O perf issue should be resolved to the
point of it being viable to run linux native apps on WSL in production. So
no real need for "windows support" on the project, per se, as Windows will
have support *for it*. :)

[1]
https://www.techtimes.com/articles/242972/20190514/future-windows-10-versions-to-ship-with-real-linux-kernel-for-wsl.htm

On Thu, May 16, 2019 at 2:26 AM Attila Wind  wrote:

> Hi Dinesh,
>
> No. And I am not really planning to. Too early to start looking for
> other workarounds I think (I could also put the whole stuff into a Linux
> based Docker container and run from there if it gets to that point - its
> supported by our everything) before it is confirmed fact that there is
> no plan from Cassandra Devs to normally support Windows in Cassandra 4.0.
> Currently it is not the case as far as I see so for now more interested
> about intentions - that was the goal of my mail. Plus of course also
> give myself a chance to let it turn out if I am missing a config options
> somewhere somehow... :-) Code base is pretty big I absolutely have no
> overview of course. Chance is high I am missing something... :-)
>
> thanks
>
> Attila Wind
>
> http://www.linkedin.com/in/attilaw
> Mobile: +36 31 7811355
>
>
> On 2019. 05. 16. 7:49, Dinesh Joshi wrote:
> > Hi Attila,
> >
> > Have you tried building & running Cassandra using Windows Subsystem for
> Linux[1]? The integration is seamless and you get a full fledged Linux
> environment on Windows. The overhead is minimal and you should be able to
> edit in your IDE on Windows.
> >
> > Thanks,
> >
> > Dinesh
> >
> > [1] https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux
> >
> >> On May 15, 2019, at 10:28 PM, Attila Wind 
> wrote:
> >>
> >> Hi Devs,
> >>
> >> Base question:
> >> How to run Cassandra 4.0 (trunk) version on a Windows based environment
> without getting always the errors like this?
> >> org.apache.cassandra.io.FSReadError: java.io.IOException: Invalid
> folder descriptor trying to create log replica
> D:\workspace\java-projects\java-automated-test-suite\target\embeddedCassandra\data\test_keyspace\test_table-f6498e1075ca11e9a59ed3a612c57aaf
> >>
> >> Background / Motivation:
> >> In our project we want to use Map/List data types heavily - also
> combined with IN operator. Due to
> https://issues.apache.org/jira/browse/CASSANDRA-12654 this does not work
> in 3.x but solved in 4.0. Also extended features around the above complex
> data types in 4.0 excites me up. So I decided to do a test deployment on
> our test servers (Linux based) - features worked like charm there.
> >> As a next step I extended our jBehave testing framework to also be able
> to fire up an embedded Cassandra 4.0 for running tests. Since I am
> developing on Windows I started to get the above mentioned FSReadError
> exceptions however folder path is valid.
> >> Same happens if I try to fire up Cassandra 4.0 instances on Windows
> systems.
> >>
> >> More advanced question:
> >> Based on stack traces I took a look into the source. In
> org.apache.cassandra.db.lifecycle.LogReplica and e.g. in open(File file)
> method (along with others) it is using the
> NativeLibrary.tryOpenDirectory(file.getParentFile().getPath()) call - if it
> fails I get the above FSReadError for sure.
> >> And here we arrive to the main point... On Windows the
> org.apache.cassandra.utils.NativeLibraryWindows class is used in the
> background. But in this class basically almost all method implementation is
> just "throw new UnsatisfiedLinkError();" - bingo! This shows me that I have
> no really hope... If I am on Windows I will fail like hell with above
> errors always.
> >>
> >> 1. Are you planning to do something around it?
> >>
> >> What I did as an ass-saver I have added a "mock" implementation for 3
> problematic methods by implementing in Java in NativeLibraryWindows class
> fake_callOpen(), fake_callFsync(), fake_callClose() kinda simulating
> something. fake_callOpen() stores the fake fdId => path mapping in an
> internal Map object. fake_callClose() flushes that out. fake_callFsync()
> gives a WARN if the fdId is uinknown (so fake_callOpen() was not invoked
> earlier). Issue has gone for me after this, Cassandra 4.0 runs fine with
> all of our test cases - I am happy! :-) So and but...
> >>
> >> 2. My quick and dirty implementation shows that I regularly get lots of
> warnings from my fake_callFsync(int fd): "file descriptor {} is unknown so
> invalid!" which means callOpen(int fd) was NOT invoked in 

Re: Running 4.0 (trunk) on Windows

[Attila Wind]

Hi Joshua!

Thanks a lot for your quick answer - appreciated!
Makes sense 100% - also now I got my answers fully.

thanks

Attila

On 16.05.2019 2:58 PM, Joshua McKenzie wrote:


 there is no plan from Cassandra Devs to normally support Windows
in Cassandra 4.0 

I was the dev kind of single-handedly making it work on Windows; it 
was good up through the 3.X line but I didn't maintain it past that 
point to 4.0+ and don't have plans to. Reason being: the WSL approach 
100% works for dev environments, and while I/O is pretty slow right 
now (i.e. use for dev only) there's a new version of Windows hitting 
soon with an actual linux kernel embedded in it[1] where the I/O perf 
issue should be resolved to the point of it being viable to run linux 
native apps on WSL in production. So no real need for "windows 
support" on the project, per se, as Windows will have support *for it*. :)


[1]https://www.techtimes.com/articles/242972/20190514/future-windows-10-versions-to-ship-with-real-linux-kernel-for-wsl.htm

On Thu, May 16, 2019 at 2:26 AM Attila Wind  
wrote:


Hi Dinesh,

No. And I am not really planning to. Too early to start looking for
other workarounds I think (I could also put the whole stuff into a
Linux
based Docker container and run from there if it gets to that point
- its
supported by our everything) before it is confirmed fact that
there is
no plan from Cassandra Devs to normally support Windows in
Cassandra 4.0.
Currently it is not the case as far as I see so for now more
interested
about intentions - that was the goal of my mail. Plus of course also
give myself a chance to let it turn out if I am missing a config
options
somewhere somehow... :-) Code base is pretty big I absolutely have no
overview of course. Chance is high I am missing something... :-)

thanks

Attila Wind

http://www.linkedin.com/in/attilaw
Mobile: +36 31 7811355


On 2019. 05. 16. 7:49, Dinesh Joshi wrote:
> Hi Attila,
>
> Have you tried building & running Cassandra using Windows
Subsystem for Linux[1]? The integration is seamless and you get a
full fledged Linux environment on Windows. The overhead is minimal
and you should be able to edit in your IDE on Windows.
>
> Thanks,
>
> Dinesh
>
> [1] https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux
>
>> On May 15, 2019, at 10:28 PM, Attila Wind
 wrote:
>>
>> Hi Devs,
>>
>> Base question:
>> How to run Cassandra 4.0 (trunk) version on a Windows based
environment without getting always the errors like this?
>> org.apache.cassandra.io
.FSReadError: java.io.IOException:
Invalid folder descriptor trying to create log replica

D:\workspace\java-projects\java-automated-test-suite\target\embeddedCassandra\data\test_keyspace\test_table-f6498e1075ca11e9a59ed3a612c57aaf
>>
>> Background / Motivation:
>> In our project we want to use Map/List data types heavily -
also combined with IN operator. Due to
https://issues.apache.org/jira/browse/CASSANDRA-12654 this does
not work in 3.x but solved in 4.0. Also extended features around
the above complex data types in 4.0 excites me up. So I decided to
do a test deployment on our test servers (Linux based) - features
worked like charm there.
>> As a next step I extended our jBehave testing framework to also
be able to fire up an embedded Cassandra 4.0 for running tests.
Since I am developing on Windows I started to get the above
mentioned FSReadError exceptions however folder path is valid.
>> Same happens if I try to fire up Cassandra 4.0 instances on
Windows systems.
>>
>> More advanced question:
>> Based on stack traces I took a look into the source. In
org.apache.cassandra.db.lifecycle.LogReplica and e.g. in open(File
file) method (along with others) it is using the
NativeLibrary.tryOpenDirectory(file.getParentFile().getPath())
call - if it fails I get the above FSReadError for sure.
>> And here we arrive to the main point... On Windows the
org.apache.cassandra.utils.NativeLibraryWindows class is used in
the background. But in this class basically almost all method
implementation is just "throw new UnsatisfiedLinkError();" -
bingo! This shows me that I have no really hope... If I am on
Windows I will fail like hell with above errors always.
>>
>> 1. Are you planning to do something around it?
>>
>> What I did as an ass-saver I have added a "mock" implementation
for 3 problematic methods by implementing in Java in
NativeLibraryWindows class fake_callOpen(), fake_callFsync(),
fake_callClose() kinda simulating something. fake_callOpen()
stores the fake fdId => path mapping in an internal Map object.
fake_callClose() flushes that out. fake_callFsync() gives a WARN
if th