For clarification, the Management REST API <https://github.com/apache/geode/tree/rel/v1.1.1/geode-core/src/main/java/org/apache/geode/management/internal/web> [1] is not the Developer REST API <http://geode.apache.org/docs/guide/11/rest_apps/book_intro.html> [2]. These are 2 separate/different things.
The later is used by developers to perform normal Geode public API style operations (e.g. Region ops, Queries, Function execution) while the later is used by *Gfsh* to tunnel commands over HTTP rather than the default, JMX RMI. The purpose was to be able to interact with a cluster in AWS, or other cloud environments, from localhost. -j [1] https://github.com/apache/geode/tree/rel/v1.1.1/geode-core/src/main/java/org/apache/geode/management/internal/web [2] http://geode.apache.org/docs/guide/11/rest_apps/book_intro.html On Tue, Apr 25, 2017 at 8:25 PM, Wes Williams <wwilli...@pivotal.io> wrote: > A couple of points to interact with John's points. > > GFSH as API > > We agree that GFSH is a DSL, and a really good and useful one. We agree > that we don't want our API tied to GFSH syntax. I view the valuable part of > GemFire admin as the Java code underneath GFSH, or the "Commands." > > For example, to create a JAVA API to "Create Region", why not create a > Java API around CreateAlterDestroyRegionCommands? In this way, GFSH and > the > JAVA API share the same code. It seems too obvious yet I don't see it being > recommended. Why not? (Note: the hard-coded formatting would need to be > removed). > > Once you have the Java/GFSH/REST API as common code, you then refactor it. > What's the objection to this approach? Once you open Java API's to do > everything that GFSH does, then you have now unshackled the power of the > developer (the JAVA API) and the admin (GFSH API). > > > REST API > > I've found that most don't want to use the Dev REST API because it's > attached to a server rather than the cluster. HA? > > > *Wes Williams | Pivotal Advisory **Data Engineer* > 781.606.0325 > http://pivotal.io/big-data/pivotal-gemfire > > On Tue, Apr 25, 2017 at 7:01 PM, Fred Krone <fkr...@pivotal.io> wrote: > > > Good feedback. > > > > This would use the new protocol. I should have mentioned that. > > > > The original driver for this was the Region API needs either an update or > > an alternative. Updating has a few drawbacks: Region wasn't designed > with > > open source in-mind and as Swap mentioned it is naturally tightly > coupled. > > Members of the community are already working to update Region but > something > > gets fixed and it breaks something for someone else. I think it's much > > better to provide a new interface that implements the first part of JSR > 107 > > (javax.cache) and get the ball rolling for the community and, perhaps, > over > > time deprecate Region (although that's not a primary objective). > > > > A Java driver will probably get built regardless just to give the new > > protocol some legs. That driver also needs a decent caching interface. > JSR > > 107 has already solved that part. So let's get started on it. If the > > community wants to go the whole way and continue JSR 107 implementation > > after that that's awesome. Functions can also be added, etc. > > > > I intentionally did not mention anything about PCF as this pertains to > > Geode itself as an open source offering and developer experience. I'm > > writing as a member of the community. Ie: I'm a developer who would like > to > > add some caching to my application -- I can download either Geode or > > Hazelcast for free and right now it's a no brainer. Not that we wouldn't > > keep PCF in-mind but it's out of scope for this thread. I do believe > > getting started on a Java driver for the protocol and a standardized > > caching API are easily leveraged wins across the board. > > > > > > > > > > On Tue, Apr 25, 2017 at 3:20 PM, Swapnil Bawaskar <sbawas...@pivotal.io> > > wrote: > > > > > I had looked at the JCache in the past and here are some of the things > I > > > noted: > > > > > > Naming convention: Geode's region is a Cache in JSR-107, and Geode's > > Cache > > > is a CacheManager. I think it would be too confusing to change the > > meaning > > > of cache. Also, how do you document this given that Cache means > different > > > things if you are talking JCache vs Geode. > > > > > > The way to create a Cache in JSR-107 is: > > > Cache<Integer, Date> cache = manager.createCache(cacheName, > Configuration > > > c); > > > Where it is upto the implementation to extend Configuration. Given > this, > > > users will not be able to switch from an existing implementation to > ours; > > > will have to write new code specially Configuration, making callbacks > > > serializable etc. > > > > > > JSR-107 will not be limited to the client. Server side callbacks like > > > CacheLoader, CacheListener etc. will need handle on jsr-107 “cache”. > > > > > > JSR-107 supports features like an EntryProcessor, which is a function > > > invoked atomically on an entry operation. We will have to make invasive > > > changes to Geode to support this. > > > > > > Given these, I don't think supporting JSR-107 is trivial. > > > > > > On Tue, Apr 25, 2017 at 2:55 PM Dan Smith <dsm...@pivotal.io> wrote: > > > > > > > What transport are you planning on using? REST, or the current binary > > > > protocol? Or is this just a wrapper around the existing java client > > APIs? > > > > > > > > If this about creating a new API, I agree with what John is saying > that > > > we > > > > need reduce the number of APIs we have to do the same thing in java. > It > > > > seems especially confusing if we end up with different APIs that > > support > > > > distinct features - like being able to create a region on the server > > with > > > > this new API but only being able to invoke a function with the old > API. > > > > > > > > The other thing I think we need to avoid is being able to do things > > from > > > > the client (or gfsh, or xml, ...) that don't have a java API on the > > > server. > > > > I'm assuming your getOrCreateRegion region is supposed behave like > the > > > gfsh > > > > command and update the cluster configuration? +++1 to Wes's > suggestion > > > have > > > > a public API for executing these gfsh operations. > > > > > > > > I really think we need to work on having *one* authoritative public > API > > > > that contains everything that we support on the server. The protocols > > we > > > > support (REST, binary, ...) and the client drivers that use those > > > protocols > > > > should just be ways of accessing that API. The Java API on the server > > > right > > > > now the closest thing we have to this, but there are things you can > do > > > with > > > > gfsh and things you can do with the current client that have no java > > API > > > > equivalent. I think we really need to fix that! > > > > > > > > Also, preferably we won't have to hand code a bunch of stuff in every > > > > client driver and protocol whenever we add a new feature. For example > > if > > > > were to add a geode function to invoke each new API we add, that new > > API > > > > would be accessible from REST, gfsh, the java client etc. Later we > > could > > > > add syntatic sugar to make the new API prettier, but if we had a low > > > level > > > > API that automatically exposed all new features that would make > adding > > > new > > > > features much less painful. > > > > > > > > I do like the idea of adding a reactive API. > > > > > > > > Supporting JSR-107 might be nice - but that could probably be > written > > > just > > > > as a wrapper around the current API without too much work. I don't > > think > > > we > > > > should do anything for JSR-107 other than provide a JSR-107 > compatible > > > > wrapper - if someone wants additional geode specific features they > > should > > > > drop into the existing API. > > > > > > > > I also like the idea of a smaller client jar and dependencies, but I > > > think > > > > there is a huge amount of room for improvement in our existing client > > > just > > > > by refactoring the code a bit more and shinking geode-core into a > bare > > > > minimum. > > > > > > > > -Dan > > > > > > > > On Mon, Apr 24, 2017 at 8:20 PM, Fred Krone <fkr...@pivotal.io> > wrote: > > > > > > > > > That's great, Wes. I will take a look. Thanks. > > > > > > > > > > John -- good feedback to consider and I was hoping this would come > > up. > > > > > > > > > > "In my mind, there really are only 2 approaches... *Spring* and > > > > > non-*Spring*, > > > > > or rather PCF and non-PCF, and since PCF is primarily based on Boot > > > > (given > > > > > Microservices/applications are the new concurrency), then I am > really > > > > > saying the same thing." > > > > > > > > > > This would be cover non spring and attempt to give the community > > > > something > > > > > that had the same standardized experience as JSR 107 -- starting > with > > > the > > > > > Cache interface itself. Although we don't necessarily have to > > > implement > > > > > Cache, it's method signatures are essentially a (non spring) > caching > > > > > standard for Java developers at this point. We considered full > > blown > > > > JSR > > > > > 107 implementation but thought it was too robust for the needs > > > mentioned > > > > > (that's not to say we couldn't get there incrementally). I think > > those > > > > > needs still exist open-source outside of PCF and don't cannibalize. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Apr 24, 2017 at 7:44 PM, Real Wes <thereal...@outlook.com> > > > > wrote: > > > > > > > > > > > > > > > > > Here is a simple Java client _for enterprise use_ that I > developed > > > for > > > > > > Geode and distributed to several enterprises. It has similarities > > and > > > > > > differences for your goal. This project creates both server and > > > client > > > > > > regions dynamically. It lists regions, alters regions… really > > > anything > > > > > > that GFSH can do. It differs in that it calls GFSH to do its work > > > > rather > > > > > > than a Java interface. You can think of this as a Java API on top > > of > > > > > GFSH. > > > > > > > > > > > > You can use this model and keep the similarities while adjusting > > for > > > > the > > > > > > Java interface. > > > > > > > > > > > > Similarities > > > > > > - Client uses SDG and complies with JSR-107 > > > > > > [the Cache Manager is here: https://github.com/Pivotal- > > > > > > Data-Engineering/ClusterManagement/tree/master/ > > > > > > cluster-management-client/src/main/java/com/geode/cache/manager > ] > > > > > > - After the server creates its region, client creates its region > > > > > > [ see createRegions at: https://github.com/Pivotal- > > > > > Data-Engineering/ > > > > > > ClusterManagement/blob/master/cluster-management- > > > > > > > > > > client/src/main/java/com/geode/cache/manager/ > > RegionCreator.java<https:// > > > > > > github.com/Pivotal-Data-Engineering/ClusterManagement/ > > > > > blob/master/cluster- > > > > > > management-client/src/main/java/com/geode/cache/manager/ > > > > > RegionCreator.java> > > > > > > ] > > > > > > > > > > > > Differences > > > > > > Server dynamically calls GFSH to execute commands > > > > > > > > > > > > How it works > > > > > > - Whenever the client calls a region that does not exist, the > > client > > > > > looks > > > > > > for a <region name>.properties file. If the properties file > exists, > > > the > > > > > > region is created with the dynamic properties. > > > > > > - If <region name>.properties does not exist, default region > > > properties > > > > > > are loaded and used to create the region. > > > > > > - properties are formed into a GFSH string and passed to a server > > > > > function > > > > > > - The server function calls GFSH using internal calls. It calls > the > > > > > > GfshParser, executes the command, and then returns the parsed > > > results. > > > > > > [see https://github.com/Pivotal-Data-Engineering/ > > > > > > ClusterManagement/blob/master/cluster-management-server/src/ > > > > > > main/java/com/geode/gfsh/GfshCommand.java] > > > > > > > > > > > > Limitations > > > > > > It uses internal calls to call GFSH. I have made requests to make > > > this > > > > a > > > > > > stable public interface. Andrew Baker had a good idea to return > > gfsh > > > > > > results in JSON format with a new —results=json property to pass > to > > > > GFSH. > > > > > > > > > > > > Strengths > > > > > > That is uses GFSH can be viewed as a strength to keep a > consistent > > > API, > > > > > > whether Java or GFSH > > > > > > > > > > > > TESTS > > > > > > You can start by running server tests at: > > > https://github.com/Pivotal- > > > > > > Data-Engineering/ClusterManagement/tree/master/ > > > > > > cluster-management-server/src/test/java/com/geode/gfsh > > > > > > > > > > > > Client tests are at: https://github.com/Pivotal- > Data-Engineering/ > > > > > > ClusterManagement/tree/master/cluster-management-client/src/ > > > > > > test/java/com/geode/management/client > > > > > > > > > > > > > > > > > > Regards, > > > > > > Wes Williams > > > > > > > > > > > > On Apr 24, 2017, at 6:51 PM, Kirk Lund <kl...@apache.org<mailto: > > > klund > > > > > > @apache.org>> wrote: > > > > > > > > > > > > A couple things I'd like to see: > > > > > > > > > > > > 1) completely new API that doesn't reuse the old API classes (or > at > > > > least > > > > > > not the giant classes such as Cache and Region interfaces) > > > > > > 2) separation of API and Impl so that users can compile their > code > > > > > against > > > > > > a dedicated client API jar > > > > > > > > > > > > On Mon, Apr 24, 2017 at 3:03 PM, Fred Krone <fkr...@pivotal.io > > > <mailto: > > > > > fkro > > > > > > n...@pivotal.io>> wrote: > > > > > > > > > > > > In an effort to improve Java developer ease of use for caching > with > > > > > Geode I > > > > > > am looking for feedback on going forward with creating a Java > > client. > > > > > This > > > > > > client will allow for server-side region creation and distributed > > > data > > > > > > caching. This would allow for a thin client that fits with > > > > microservice > > > > > > caching patterns and also abstracts a cleaner client-server > > > experience > > > > > > driven interface. > > > > > > > > > > > > Initially we were going to update the Region interface but were > > > > concerned > > > > > > with breaking existing applications. We also would like to > provide > > > > > Region > > > > > > creation to a client application and so propose here solving both > > of > > > > > these > > > > > > areas with a Java client. > > > > > > > > > > > > It would have new project repo for the client. > > > > > > > > > > > > It would provide new Region interface for clients. The specifics > > of > > > > the > > > > > > API design are too lengthy for this conversation but > implementation > > > > will > > > > > > resemble JSR 107 Cache interface. > > > > > > > > > > > > It would use the new security framework. > > > > > > > > > > > > > > > > > > *An example*, > > > > > > > > > > > > The client application simply creates an instance of client and > > > points > > > > it > > > > > > to the locator: > > > > > > > > > > > > > > > > > > org.apache.geode.client.Client client = > Client.create(locatorHost, > > > > > > locatorPort); > > > > > > > > > > > > > > > > > > Client has the following methods: > > > > > > > > > > > > package org.apache.geode.client; > > > > > > > > > > > > > > > > > > public interface GeodeClient { > > > > > > > > > > > > /** > > > > > > > > > > > > * creates the region on the servers, or gets the region if it > > > exits, > > > > > > returns a PROXY region > > > > > > > > > > > > */ > > > > > > > > > > > > public Region getOrCreateRegion(RegionAttributes attributes, > > String > > > > > > name); > > > > > > > > > > > > > > > > > > /** > > > > > > > > > > > > * Returns a PROXY region if the region exists on the server > > > > > > > > > > > > */ > > > > > > > > > > > > public Region getRegion(String name); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > MVP > > > > > > > > > > > > The smallest set of features to test this idea, learn and > iterate, > > > and > > > > > get > > > > > > the client into the communities hands for future iterations is: > > > > > > > > > > > > > > > > > > Create a server side Region from a client > > > > > > > > > > > > Region interface has CRUD on par with javax.cache.Cache (from JSR > > > 107) > > > > > > > > > > > > Calls are asynchronous -- futures > > > > > > > > > > > > > > > > > > > > > > > > Also would like feedback on which future functionality would be > > most > > > > > useful > > > > > > from a thin client: > > > > > > > > > > > > Function execution > > > > > > > > > > > > Durable clients > > > > > > > > > > > > User defined serialization > > > > > > > > > > > > Register interest > > > > > > > > > > > > Queries > > > > > > > > > > > > CQ > > > > > > > > > > > > Near side caching > > > > > > > > > > > > Create disk stores from client > > > > > > > > > > > > Region group membership > > > > > > > > > > > > Client subscription load balancing > > > > > > Transactions > > > > > > > > > > > > > > > > > > Thanks, > > > > > > -Fred > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- -John john.blum10101 (skype)
