temporal commit, forgot to switch to the branch
Project: http://git-wip-us.apache.org/repos/asf/incubator-ignite/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ignite/commit/d5fdbf49 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ignite/tree/d5fdbf49 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ignite/diff/d5fdbf49 Branch: refs/heads/ignite-789 Commit: d5fdbf49bfaad085b6929d409de7e3a94a2a54c8 Parents: 0ee2799 Author: Denis Magda <dma...@gridgain.com> Authored: Thu Apr 23 17:03:16 2015 +0300 Committer: Denis Magda <dma...@gridgain.com> Committed: Thu Apr 23 17:03:16 2015 +0300 ---------------------------------------------------------------------- modules/cloud/README.txt | 32 +++ modules/cloud/licenses/apache-2.0.txt | 202 ++++++++++++++++ modules/cloud/pom.xml | 67 ++++++ .../cloud/TcpDiscoveryCloudNodesIpFinder.java | 234 +++++++++++++++++++ .../tcp/ipfinder/cloud/package-info.java | 21 ++ pom.xml | 1 + 6 files changed, 557 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/d5fdbf49/modules/cloud/README.txt ---------------------------------------------------------------------- diff --git a/modules/cloud/README.txt b/modules/cloud/README.txt new file mode 100644 index 0000000..c57a5da --- /dev/null +++ b/modules/cloud/README.txt @@ -0,0 +1,32 @@ +Apache Ignite Cloud Module +------------------------ + +Apache Ignite Cloud module provides Apache jclouds implementations of IP finder for TCP discovery. + +To enable Cloud module when starting a standalone node, move 'optional/ignite-cloud' folder to +'libs' folder before running 'ignite.{sh|bat}' script. The content of the module folder will +be added to classpath in this case. + +Importing Cloud Module In Maven Project +------------------------------------- + +If you are using Maven to manage dependencies of your project, you can add Cloud module +dependency like this (replace '${ignite.version}' with actual Ignite version you are +interested in): + +<project xmlns="http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 + http://maven.apache.org/xsd/maven-4.0.0.xsd";> + ... + <dependencies> + ... + <dependency> + <groupId>org.apache.ignite</groupId> + <artifactId>ignite-cloud</artifactId> + <version>${ignite.version}</version> + </dependency> + ... + </dependencies> + ... +</project> http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/d5fdbf49/modules/cloud/licenses/apache-2.0.txt ---------------------------------------------------------------------- diff --git a/modules/cloud/licenses/apache-2.0.txt b/modules/cloud/licenses/apache-2.0.txt new file mode 100644 index 0000000..d645695 --- /dev/null +++ b/modules/cloud/licenses/apache-2.0.txt @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/d5fdbf49/modules/cloud/pom.xml ---------------------------------------------------------------------- diff --git a/modules/cloud/pom.xml b/modules/cloud/pom.xml new file mode 100644 index 0000000..61bdf22 --- /dev/null +++ b/modules/cloud/pom.xml @@ -0,0 +1,67 @@ +<?xml version="1.0" encoding="UTF-8"?> + +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +<project xmlns="http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + + <modelVersion>4.0.0</modelVersion> + + <parent> + <groupId>org.apache.ignite</groupId> + <artifactId>ignite-parent</artifactId> + <version>1</version> + <relativePath>../../parent</relativePath> + </parent> + + <artifactId>ignite-cloud</artifactId> + <version>1.0.3-SNAPSHOT</version> + + <properties> + <jcloud.version>1.9.0</jcloud.version> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.ignite</groupId> + <artifactId>ignite-core</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.jclouds</groupId> + <artifactId>jclouds-allcompute</artifactId> + <version>${jcloud.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.jclouds.driver</groupId> + <artifactId>jclouds-sshj</artifactId> + <version>${jcloud.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.jclouds.driver</groupId> + <artifactId>jclouds-log4j</artifactId> + <version>${jcloud.version}</version> + </dependency> + + </dependencies> + +</project> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/d5fdbf49/modules/cloud/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/cloud/TcpDiscoveryCloudNodesIpFinder.java ---------------------------------------------------------------------- diff --git a/modules/cloud/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/cloud/TcpDiscoveryCloudNodesIpFinder.java b/modules/cloud/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/cloud/TcpDiscoveryCloudNodesIpFinder.java new file mode 100644 index 0000000..0680595 --- /dev/null +++ b/modules/cloud/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/cloud/TcpDiscoveryCloudNodesIpFinder.java @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.spi.discovery.tcp.ipfinder.cloud; + +import com.google.common.collect.ImmutableSet; +import com.google.inject.Module; +import org.apache.ignite.internal.IgniteInterruptedCheckedException; +import org.apache.ignite.internal.util.tostring.GridToStringExclude; +import org.apache.ignite.internal.util.typedef.internal.U; +import org.apache.ignite.spi.IgniteSpiConfiguration; +import org.apache.ignite.spi.IgniteSpiException; +import org.apache.ignite.spi.discovery.tcp.TcpClientDiscoverySpi; +import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi; +import org.apache.ignite.spi.discovery.tcp.ipfinder.*; +import org.jclouds.ContextBuilder; +import org.jclouds.compute.ComputeService; +import org.jclouds.compute.ComputeServiceContext; +import org.jclouds.compute.domain.ComputeMetadata; +import org.jclouds.compute.domain.NodeMetadata; +import org.jclouds.sshj.config.SshjSshClientModule; +import org.jclouds.logging.log4j.config.*; + +import java.net.InetSocketAddress; +import java.util.Collection; +import java.util.LinkedList; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * IP finder for automatic lookup of nodes running in a cloud. + * <p> + * Implementation is based on Apache jclouds multi-cloud toolkit. + * For information about jclouds visit <a href="https://jclouds.apache.org/";>jclouds.apache.org</a>. + * <h1 class="header">Configuration</h1> + * <h2 class="header">Mandatory</h2> + * <ul> + * <li>Cloud provider (see {@link #setProvider(String)})</li> + * <li>Identity (see {@link #setIdentity(String)})</li> + * </ul> + * <h2 class="header">Optional</h2> + * <ul> + * <li>Credential (see {@link #setCredential(String)})</li> + * <li>Discovery port (see {@link #setDiscoveryPort(Integer)})</li> + * </ul> + * </p> + * <p> + * The finder forms nodes addresses, that possibly running Ignite, by getting private and public IPs of all + * VMs in a cloud and adding {@link #discoveryPort} to them. + * Both {@link #registerAddresses(Collection)} and {@link #unregisterAddresses(Collection)} has no effect. + * </p> + * <p> + * Note, this finder is only workable when it used directly by a cloud VM. + * Choose another implementation of {@link org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinder} for local + * or home network tests. + * </p> + */ +public class TcpDiscoveryCloudNodesIpFinder extends TcpDiscoveryIpFinderAdapter { + /* Cloud provider. */ + private String provider; + + /* Cloud specific identity (user name, email address, etc.). */ + private String identity; + + /* Cloud specific credential (password, secrets key, etc.). */ + @GridToStringExclude + private String credential; + + /* Port to use to connect to nodes across a cluster. */ + private Integer discoveryPort; + + /** Init guard. */ + @GridToStringExclude + private final AtomicBoolean initGuard = new AtomicBoolean(); + + /** Init latch. */ + @GridToStringExclude + private final CountDownLatch initLatch = new CountDownLatch(1); + + /* JCloud compute service. */ + private ComputeService computeService; + + /** + * Constructor. + */ + public TcpDiscoveryCloudNodesIpFinder() { + setShared(true); + } + + /** {@inheritDoc} */ + @Override public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException { + initComputeService(); + + Collection<InetSocketAddress> addresses = new LinkedList<>(); + + try { + for (ComputeMetadata node : computeService.listNodes()) { + NodeMetadata metadata = computeService.getNodeMetadata(node.getId()); + + for (String addr : metadata.getPrivateAddresses()) + addresses.add(new InetSocketAddress(addr, discoveryPort)); + + for (String addr : metadata.getPublicAddresses()) + addresses.add(new InetSocketAddress(addr, discoveryPort)); + } + } + catch (Exception e) { + throw new IgniteSpiException("Failed to get registered addresses for the provider: " + provider); + } + + return addresses; + } + + /** {@inheritDoc} */ + @Override public void registerAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException { + // No-op + } + + /** {@inheritDoc} */ + @Override public void unregisterAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException { + // No-op + } + + /** + * Sets the cloud provider to use. + * + * <a href="https://jclouds.apache.org/reference/providers/#compute";>Apache jclouds providers list</a> from + * ComputeService section contains names of all supported providers. + * + * @param provider Provider name. + */ + @IgniteSpiConfiguration(optional = false) + public void setProvider(String provider) { + this.provider = provider; + } + + /** + * Sets the identity that is used as a user name during a connection to the cloud. + * Depending on a cloud platform it can be an email address, user name, etc. + * + * Refer to <a href="http://jclouds.apache.org/guides/";>Apache jclouds guide</a> to get concrete information on + * what is used as an identity for a particular cloud platform. + * + * @param identity Identity to use during authentication on the cloud. + */ + @IgniteSpiConfiguration(optional = false) + public void setIdentity(String identity) { + this.identity = identity; + } + + /** + * Sets credential that is used during authentication on the cloud. + * Depending on a cloud platform it can be a password, path to a secrets file, etc. + * + * Refer to <a href="http://jclouds.apache.org/guides/";>Apache jclouds guide</a> to get concrete information on + * what is used as an credential for a particular cloud platform. + * + * @param credential Credential to use during authentication on the cloud. + */ + @IgniteSpiConfiguration(optional = true) + public void setCredential(String credential) { + this.credential = credential; + } + + /** + * Sets the port that is used to discover other nodes running Apache Ignite in the cloud. + * If doesn't set, a default port number is used. + * + * Note that all cloud nodes must accept connection on the same port number. + * + * @param discoveryPort Port number to use for discovering of other nodes. + */ + @IgniteSpiConfiguration(optional = true) + public void setDiscoveryPort(Integer discoveryPort) { + this.discoveryPort = discoveryPort; + } + + /** + * Initializes Apache jclouds compute service. + */ + private void initComputeService() { + if (initGuard.compareAndSet(false, true)) + try { + if (provider == null) + throw new IgniteSpiException("Cloud provider is not set."); + + if (identity == null) + throw new IgniteSpiException("Cloud identity is not set."); + + try { + ContextBuilder ctxBuilder = ContextBuilder.newBuilder(provider); + ctxBuilder.credentials(identity, credential); + ctxBuilder.modules(ImmutableSet.<Module>of(new Log4JLoggingModule(), new SshjSshClientModule())); + + computeService = ctxBuilder.buildView(ComputeServiceContext.class).getComputeService(); + } + catch (Exception e) { + throw new IgniteSpiException("Failed to connect to the provider: " + provider, e); + } + + if (discoveryPort == null) + discoveryPort = TcpDiscoverySpi.DFLT_PORT; + + } finally { + initLatch.countDown(); + } + else + { + try { + U.await(initLatch); + } + catch (IgniteInterruptedCheckedException e) { + throw new IgniteSpiException("Thread has been interrupted.", e); + } + + if (computeService == null) + throw new IgniteSpiException("Ip finder has not been initialized properly."); + } + } +} http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/d5fdbf49/modules/cloud/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/cloud/package-info.java ---------------------------------------------------------------------- diff --git a/modules/cloud/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/cloud/package-info.java b/modules/cloud/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/cloud/package-info.java new file mode 100644 index 0000000..87fc19a --- /dev/null +++ b/modules/cloud/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/cloud/package-info.java @@ -0,0 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** + * <!-- Package description. --> + * Contains Apache jclouds based IP finder. + */ +package org.apache.ignite.spi.discovery.tcp.ipfinder.cloud; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/d5fdbf49/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 8810b17..6585c37 100644 --- a/pom.xml +++ b/pom.xml @@ -75,6 +75,7 @@ <module>modules/jcl</module> <module>modules/schema-import</module> <module>modules/codegen</module> + <module>modules/cloud</module> </modules> <profiles>
