> > the content is going over the wire on whatever port that was port before.
>From what I see, DownloadJarFunction is calling SimpleRemoteInputStream.export() which will call UnicastRemoteObject.exportObject. That's an RMI call to start a tcp server socket listening for connections to interact with that object. -Dan On Tue, Jan 23, 2018 at 6:15 PM, Jinmei Liao <jil...@pivotal.io> wrote: > As far as I can see, we are utilizing the streaming capability provided by > the rmi-io, the content is going over the wire on whatever port that was > port before. When streaming content from the gfsh to the jmxManager, it's > using the jmx port; when getting jars between locator/servers, it's using > the FunctionService, so it's whatever communication channel that > FunctionService is using. > > All the FileContent are saved in temp folder, and get cleaned up after each > deployment. > > On Tue, Jan 23, 2018 at 3:17 PM, Dan Smith <dsm...@pivotal.io> wrote: > > > I don't have an issue with the dependency. But if we are opening up new > > ports for RMI connections, that seems like a potential security risk. If > > someone has enabled cluster SSL we shouldn't be opening up an insecure > port > > for RMI connections. > > > > We should also make sure this is not leaking open sockets/file > decriptors. > > How does this SimpleRemoteInputStream we are creating get shutdown and > > cleaned up? > > > > -Dan > > > > On Tue, Jan 23, 2018 at 2:36 PM, Jens Deppe <jensde...@apache.org> > wrote: > > > > > Apologies that this was not raised earlier in discussion but I'm happy > to > > > describe it now. > > > > > > *Background:* > > > > > > When deploying jars into Geode they are moved through the system as > > simple > > > byte[] blobs. This obviously consumes memory. The various affected > areas > > > are: > > > > > > - gfsh reads the jars into memory > > > - the jars are pushed to the locator (via a jmx call) - again creating > a > > > byte[] blob on the locator > > > - from the locator, the jars are pushed to all servers via a function > > call > > > (also sending the jars as byte[] blobs). > > > > > > Obviously if the jar is small this would not be a problem, however in > > > memory constrained systems or with large jars this is obviously going > to > > > put pressure on memory and possibly result in OOM situations. In fact, > > the > > > reason this came up was that some folks were unable to deploy a 40Mb > jar > > to > > > a 512Mb (heap) locator. > > > > > > *rmi-io:* > > > > > > After doing some research, it seemed that the ideal solution would be > > > something that allows for serializing Input/OutputStreams. Java doesn't > > > provide anything natively. > > > > > > One library that stood out as being robust and feature complete was > > rmi-io > > > [1]. This allows for serializing a remote Input/OutputStream object > which > > > then lets us completely avoid having to pull deploying jars into memory > > > everywhere. Under the covers it uses RMI and allows for either > 'pulling' > > or > > > 'pushing' data. The reference page [2] has nice sequence diagrams. > > > > > > If anyone sees any issues with this, please do raise them. The current > > > usage of this has not changed any user-facing interaction so ultimately > > > changing the actual implemented fix for this problem (if we needed to) > > > would not have any external effect. > > > > > > Thanks > > > --Jens > > > > > > [1] http://openhms.sourceforge.net/rmiio > > > [2] http://openhms.sourceforge.net/rmiio/class_reference.html > > > > > > > > > -- > Cheers > > Jinmei >
