Re: [PDE West] Re: [PDE East] Geode Conference - Double the Expected Attendance

[Jagdish Mirani]

Adding Pivotal events team: Katheryn Efird, Katie Nelson (plus all the
others who helped with this behind the scenes)

Mike's e-mail and the comments that followed just about said it all. A HUGE
thanks for your contribution to this.

Regards
Jag

On Tue, Dec 5, 2017 at 9:14 AM, Ivan Novick  wrote:

> +1 great outcome and great effort behind it
>
> On Tue, Dec 5, 2017 at 7:19 AM, Ian Andrews  wrote:
>
>> Nice work everyone. This is great.
>>
>> _
>> Ian Andrews
>> 571-217-7988 <(571)%20217-7988>
>> iandr...@pivotal.io
>> @ianandrewsdc
>>
>> On Dec 5, 2017, at 6:41 AM, Jacque Istok  wrote:
>>
>> very awesome to see the interest in this project continue to skyrocket!
>> kudos to everyone involved to get it to this point.
>>
>> On Tue, Dec 5, 2017 at 5:58 AM, Scott Yara  wrote:
>>
>>>
>>> Wow. This is super exciting & encouraging. Congratulations!
>>>
>>> - Scott
>>>
>>> Sent from my iPhone
>>>
>>> On Dec 5, 2017, at 12:24 AM, Dormain Drewitz 
>>> wrote:
>>>
>>> Thanks Mike and all! It was amazing to see that room packed! I got a
>>> somewhat shoddy picture from the stage as we kicked off.. but shows how
>>> literally every seat was filled!
>>> Also, you can see some of the moments tweeted out under #GeodeAtS1P
>>> ,
>>> both from today and through the week. The sessions today were filmed and
>>> will be posted!
>>>
>>> Huge thanks to Jag for driving this event
>>>
>>> 
>>>
>>> On Mon, Dec 4, 2017 at 11:05 PM, Akihiro Kitada 
>>> wrote:
>>>
 Great news!


 --
 Akihiro Kitada  |  Staff Customer Engineer |  +81 80 3716 3736
 <+81%2080-3716-3736>
 Support.Pivotal.io   |  Mon-Fri  9:00am to
 5:30pm JST  |  1-877-477-2269 <(877)%20477-2269>
 [image: support]  [image: twitter]
  [image: linkedin]
  [image: facebook]
  [image: google plus]
  [image: youtube]
 


 2017-12-05 16:01 GMT+09:00 Michael Stolz :

> This was really a great day in the history of Apache Geode.
>
> We were concerned that we might not have any turn-out at all for this
> session because it was disjointed from the rest of Spring 1 Platform. The
> rest of the show starts tomorrow.
>
> We were concerned that we had no way to measure the popularity because
> there was no registration page.
> We knew we had way too many great speaker submissions to turn down all
> but 7 of them in the main show starting Tuesday.
> In fact we still had to turn down quite a few (including mine :)
>
>
> Thank goodness the Spring 1 app has an "add to my schedule" feature.
>
>
> We got wind a few days ago that there were 80+ "add to my schedule"
> hits. The room was only configured to handle 65 people.
>
> Jagdish worked with the folks running the conference to reconfigure
> the room to accommodate about 170 people by changing the layout.
>
> We filled all 170 seats 30 minutes before the session started. We
> proactively moved briefcases off of seats, moved people to the sides of 
> the
> room and made every seat count. We still couldn't fit everybody.
>
> I want to thank all the speakers, the PDE team, the event team and
> Jagdish for all of the efforts that went into making this the best 
> attended
> Geode event ever. Clearly Geode is up and coming. More than double last
> year's attendance. And we had to turn some away.
>
> Let's start looking at how we want to continue to grow the community
> next year. And lets not wait until December. We have lots of opportunity 
> to
> grow the Geode community in the early part of 2018.
>
> Thanks all for the great work leading up to this fantastic outcome.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771 <+1%20631-835-4771>
>
> On Mon, Dec 4, 2017 at 10:01 PM, Jagdish Mirani 
> wrote:
>
>> Yes, we promoted the book. As Wes noted, we totally exceeded our
>> expectations. ☺
>>
>> On Mon, Dec 4, 2017 at 1:33 PM, Marshall Presser > > wrote:
>>
>>> Please promote Mike's book if appropriate.
>>> MEP
>>>
>>> On Mon, Dec 4, 2017 at 4:16 PM, Wes Williams 
>>> wrote:
>>>





 Regards,
 Wes Williams
 Sent from mobile phone

>>>
>>>
>>>
>>> --
>>> Marshall Presser
>>> Pivotal Data Engineering
>>> mpresser@pivotal .io
>>> 240.401.1750 <(240)%20401-1750>
>>>
>>>
>>
>>
>> --
>> Regards
>> Jag
>>
>
>

>>>

Re: DISCUSS: Deprecating and replacing current serializable string encodings

[Jacob Barrett]

On Mon, Dec 4, 2017 at 1:52 PM Dan Smith  wrote:

> The new protocol is currently translating from PDX->JSON before sending
> results to the clients so the client doesn't have to understand PDX or
> DataSerializable.
>

For now it does, but we all know that isn't a longer term viable solution.
We will either have to support PDX, at which time all clients will have to
implement nightmare that is PDX, or replace it with something better.


> There is a lot more to DataSerializable than just how a String is
> serialized. And it's not even documented that I am aware of. Just tweaking
> the string format is not going to make that much better. Your hypothetical
> ruby developer is in trouble with or without this proposed change.
>

Very true but just because you can't fix everything doesn't mean you
shouldn't try to fix something.


> Breaking compatibility is a huge PITA for our users. We should do that when
> we are actually giving them real benefits. In this case if we were
> switching to some newer PDX format that was actually easy to implement
> deserialization logic I could see the argument for breaking compatibility.
> Just changing the string format without fixing the rest of the issues
> around DataSerializable isn't providing real benefits.
>

Great, let's prioritize some new PDX format that doesn't have 4 different
string encodings, 2 of which are incorrect and one that isn't a standard.


> You can't assume that a client in one language will only be serializing
> > strings for it's own consumption.
> >
>
> I wasn't making that assumption. The suggestion is that the C++ client
> would have to deserialize all 4 valid formats, but it could just always
> serialize data using the valid UTF-16 format. All other clients should be
> able to deserialize that.
>

Unfortunately its the Java Modified UTF-8 to anything that is the PITA
since it doesn't comply with any standard it must be implemented by hand.
Reading it and not writing it only makes it 1/2 a PITA.


-Jake

Re: DISCUSS: Deprecating and replacing current serializable string encodings

[Jacob Barrett]

On Mon, Dec 4, 2017 at 8:45 PM Michael Stolz  wrote:

> Anything that breaks data on disk is also a big PITA. This change would
> break data on disk.


Changing the on wire serialization format doesn't necessitate changing the
on disk format. While it would be easier or more performant to have them be
the same it isn't necessary. It also isn't necessary that all entries be
stored in the same format.

-Jake

Geode tests completed in pipeline with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/AcceptanceTest/builds/12

Geode tests completed in pipeline with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/FlakyTest/builds/11

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #757 was SUCCESSFUL (with 2264 tests)

[Spring CI]

---
Spring Data GemFire > Nightly-ApacheGeode > #757 was successful.
---
Scheduled
2266 tests in total.

https://build.spring.io/browse/SGF-NAG-757/





--
This message is automatically generated by Atlassian Bamboo

Re: [DISCUSS] FunctionAdapter incompatible serialVersionUID

[Jason Huynh]

I've sent in a pull request for this:
https://github.com/apache/geode/pull/1119

I've added the old serialVersionUID to the FunctionAdapter, under the
assumption that anyone in 1.0-1.3 would have seen that the FunctionAdapter
had been deprecated and not used it.  The 1.0-1.3 users could also easily
change "extends FunctionAdapter" to "implements Function", recompile and
things would work for them.  The users pre-1.0 would have a slightly more
difficult approach.

Anyone interested, please review/accept or reject the pull request.

Thanks,
-Jason

On Wed, Nov 29, 2017 at 9:09 AM Darrel Schneider 
wrote:

> +1 to not removing deprecated apis in minor releases.
> The semver policy Alexander describes seems reasonable.
> In the past of we have had something deprecated for a long time we have
> felt free to remove it whenever we want but I think the semver policy is a
> better way to decide when we are free to remove deprecated external apis.
>
>
> On Wed, Nov 29, 2017 at 8:50 AM, Alexander Murmann 
> wrote:
>
> > Even though the class is deprecated, you should be able to go from one
> > minor version to another without having to worry about anything breaking.
> > The point of semver is to provide information about things breaking or
> not
> > without having to read the changelog. If we remove APIs in a minor
> version
> > because they were previously deprecated we break that contract.
> Semver.org
> > 
> > recommends
> > that the functionality should be marked as deprecated in a minor release
> > and then removed as part of a major release.
> >
> > On Tue, Nov 28, 2017 at 7:03 PM, Jacob Barrett 
> > wrote:
> >
> > > Since the class was deprecated and is technically only there for
> pre-1.0
> > > compatibly then the behavior of this class should be consistent with
> the
> > > pre-1.0 version. This will break 1.0 to 1.3 but anyone coding to a
> > post-1.0
> > > version should not be using this deprecated class.
> > >
> > >
> > > > On Nov 28, 2017, at 5:22 PM, Jason Huynh  wrote:
> > > >
> > > > *With your proposoal 1.0 - 1.3 users would have modify their source
> > code
> > > on
> > > > the client and the server forthe function, correct?*
> > > >
> > > > If they start a new geode server 1.4+ and happened to extend
> > > > functionAdapter (it was deprecated in 1.0) then they would have to
> > > > recompile their client to not use functionAdapter.
> > > >
> > > > This should only affect users that extend FunctionAdapter and execute
> > > > functions by serializing them to the server from the client.  If they
> > > > execute by id it should not run into this problem...
> > > >
> > > >> On Tue, Nov 28, 2017 at 5:20 PM Jason Huynh 
> > wrote:
> > > >>
> > > >> Dan, yeah, the suggested change in the stack overflow answer does
> work
> > > and
> > > >> I was able to put an if with the exact serialVersionUid before
> posting
> > > the
> > > >> proposal, but it is pretty hacky and may affect another class that
> > > somehow
> > > >> generated the same uid.  I can make that change too but I'd prefer
> not
> > > to
> > > >> have to maintain it moving forward...
> > > >>
> > > >>
> > > >>
> > > >>> On Tue, Nov 28, 2017 at 5:09 PM Dan Smith 
> wrote:
> > > >>>
> > > >>> I agree I don't think we can get rid of FunctionAdapter until the
> > next
> > > >>> major version.
> > > >>>
> > > >>> I was thinking FunctionAdapter is rather widely used, but then I'm
> > > >>> surprised no one has hit this yet.
> > > >>>
> > > >>> All of the options kinda suck here - either pre 1.0 users have a
> > > >>> compatibility issue or 1.0-1.3 users do. With your proposoal 1.0 -
> > 1.3
> > > >>> users would have modify their source code on the client and the
> > server
> > > for
> > > >>> the function, correct?
> > > >>>
> > > >>> If we got really fancy we could actually ignore the
> serialVersionUUID
> > > for
> > > >>> this class like this - https://stackoverflow.com/a/1816711/2813144
> .
> > > But
> > > >>> it's pretty messy.
> > > >>>
> > > >>> -Dan
> > > >>>
> > > >>> On Tue, Nov 28, 2017 at 1:59 PM, Alexander Murmann <
> > > amurm...@pivotal.io>
> > > >>> wrote:
> > > >>>
> > >  Anil, I am not sure following. I think FunctionAdapter already is
> > >  deprecated. Isn't it? Anthony is right though that we shouldn't
> > remove
> > >  anything customer facing unless we are doing a major release.
> > > Otherwise
> > > >>> we
> > >  are violating the contract provided by semantic versioning.
> > > 
> > >  On Tue, Nov 28, 2017 at 1:52 PM, Anilkumar Gingade <
> > > aging...@pivotal.io
> > > 
> > >  wrote:
> > > 
> > > > I haven't seen many uses of FunctionAdapter; if its not used
> much,
> > I
> > >  think
> > > > we should deprecate this...
> > > >
> > > > It only provided default implementation for few of the methods;
> > this
> > >  could
> > > > be added in the docs/release notes to help application to move to
> > >

[DISCUSS] Proposal to Deprecate Hash Index

[Jason Huynh]

This is a proposal to deprecate existing Hash Index and deprecate the
create hash index apis.


Currently the Hash Index name causes confusion. It is not a traditional
hash look up index, but more of memory savings index.  The index does not
store index keys in memory and must hash the keys every time.  The index
synchronizes on a backing array and when the backing array needs to be
expanded, it currently needs to rehash all elements in the array.  This can
be very problematic for larger data sets.


There were improvements made to one of the functional indexes (compact
range index) prior to open sourcing.  These improvements helped reduce the
memory consumption of that index and makes it very similar sized to a hash
index, but the keys still are stored in memory.  Probably close enough to
be a replacement for the hash index in most cases.  The read/write
performance on it is also faster than the hash index.


If anyone has any objections, please let us know and why.


Thanks,

- Jason

Re: [DISCUSS] Proposal to Deprecate Hash Index

[Dan Smith]

+1

-Dan

On Tue, Dec 5, 2017 at 4:28 PM, Jason Huynh  wrote:

> This is a proposal to deprecate existing Hash Index and deprecate the
> create hash index apis.
>
>
> Currently the Hash Index name causes confusion. It is not a traditional
> hash look up index, but more of memory savings index.  The index does not
> store index keys in memory and must hash the keys every time.  The index
> synchronizes on a backing array and when the backing array needs to be
> expanded, it currently needs to rehash all elements in the array.  This can
> be very problematic for larger data sets.
>
>
> There were improvements made to one of the functional indexes (compact
> range index) prior to open sourcing.  These improvements helped reduce the
> memory consumption of that index and makes it very similar sized to a hash
> index, but the keys still are stored in memory.  Probably close enough to
> be a replacement for the hash index in most cases.  The read/write
> performance on it is also faster than the hash index.
>
>
> If anyone has any objections, please let us know and why.
>
>
> Thanks,
>
> - Jason
>

Geode tests completed in pipeline with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/DistributedTest/builds/11

Re: [DISCUSS] Proposal to Deprecate Hash Index

[Kirk Lund]

+1

On Tue, Dec 5, 2017 at 4:34 PM, Dan Smith  wrote:

> +1
>
> -Dan
>
> On Tue, Dec 5, 2017 at 4:28 PM, Jason Huynh  wrote:
>
> > This is a proposal to deprecate existing Hash Index and deprecate the
> > create hash index apis.
> >
> >
> > Currently the Hash Index name causes confusion. It is not a traditional
> > hash look up index, but more of memory savings index.  The index does not
> > store index keys in memory and must hash the keys every time.  The index
> > synchronizes on a backing array and when the backing array needs to be
> > expanded, it currently needs to rehash all elements in the array.  This
> can
> > be very problematic for larger data sets.
> >
> >
> > There were improvements made to one of the functional indexes (compact
> > range index) prior to open sourcing.  These improvements helped reduce
> the
> > memory consumption of that index and makes it very similar sized to a
> hash
> > index, but the keys still are stored in memory.  Probably close enough to
> > be a replacement for the hash index in most cases.  The read/write
> > performance on it is also faster than the hash index.
> >
> >
> > If anyone has any objections, please let us know and why.
> >
> >
> > Thanks,
> >
> > - Jason
> >
>