As far as I can see, we are utilizing the streaming capability provided by the rmi-io, the content is going over the wire on whatever port that was port before. When streaming content from the gfsh to the jmxManager, it's using the jmx port; when getting jars between locator/servers, it's using the FunctionService, so it's whatever communication channel that FunctionService is using.
All the FileContent are saved in temp folder, and get cleaned up after each deployment. On Tue, Jan 23, 2018 at 3:17 PM, Dan Smith <dsm...@pivotal.io> wrote: > I don't have an issue with the dependency. But if we are opening up new > ports for RMI connections, that seems like a potential security risk. If > someone has enabled cluster SSL we shouldn't be opening up an insecure port > for RMI connections. > > We should also make sure this is not leaking open sockets/file decriptors. > How does this SimpleRemoteInputStream we are creating get shutdown and > cleaned up? > > -Dan > > On Tue, Jan 23, 2018 at 2:36 PM, Jens Deppe <jensde...@apache.org> wrote: > > > Apologies that this was not raised earlier in discussion but I'm happy to > > describe it now. > > > > *Background:* > > > > When deploying jars into Geode they are moved through the system as > simple > > byte[] blobs. This obviously consumes memory. The various affected areas > > are: > > > > - gfsh reads the jars into memory > > - the jars are pushed to the locator (via a jmx call) - again creating a > > byte[] blob on the locator > > - from the locator, the jars are pushed to all servers via a function > call > > (also sending the jars as byte[] blobs). > > > > Obviously if the jar is small this would not be a problem, however in > > memory constrained systems or with large jars this is obviously going to > > put pressure on memory and possibly result in OOM situations. In fact, > the > > reason this came up was that some folks were unable to deploy a 40Mb jar > to > > a 512Mb (heap) locator. > > > > *rmi-io:* > > > > After doing some research, it seemed that the ideal solution would be > > something that allows for serializing Input/OutputStreams. Java doesn't > > provide anything natively. > > > > One library that stood out as being robust and feature complete was > rmi-io > > [1]. This allows for serializing a remote Input/OutputStream object which > > then lets us completely avoid having to pull deploying jars into memory > > everywhere. Under the covers it uses RMI and allows for either 'pulling' > or > > 'pushing' data. The reference page [2] has nice sequence diagrams. > > > > If anyone sees any issues with this, please do raise them. The current > > usage of this has not changed any user-facing interaction so ultimately > > changing the actual implemented fix for this problem (if we needed to) > > would not have any external effect. > > > > Thanks > > --Jens > > > > [1] http://openhms.sourceforge.net/rmiio > > [2] http://openhms.sourceforge.net/rmiio/class_reference.html > > > -- Cheers Jinmei
