Hi Jochen Thanks for getting back to me. Actually I started with the default settings, but when they didn't work, I started investigating the elasticsearch settings. Turns out the "problem" was that my elasticsearch cluster was not listening on loop back. As per my output in the original post, it was listening on the hosts RFC1918 address. To "solve" the problem I set elasticsearch back to listening on loop back, and hey presto, graylog could see it. Since I need my elasticsearch nodes to talk to each other, I need them to listen on a routable IP. As such I opted for this in my elasticsearch config:
bddvgl1:~ # grep network.host /etc/elasticsearch/elasticsearch.yml network.host: [ "_local_", "_site_" ] bddvgl1:~ # Which makes it listen on BOTH RFC1918 addresses, and the loopback. Graylog is happy because it can now access elasticsearch on loopback. It does make me wonder however what elasticsearch_discovery_zen_ping_unicast_hosts Actually does, if (as it appears) its not for pointing graylog at the address(es) of the elasticsearch cluster. On Tuesday, 6 December 2016 16:34:22 UTC, Jochen Schalanda wrote: > > Hi Dylan, > > it seems like you're overthinking this. > > If you run Elasticsearch and Graylog on the same node, you don't need to > configure anything except for the cluster name really. > > Keep the defaults for everything except cluster.name in the Elasticsearch > configuration and read > http://docs.graylog.org/en/2.1/pages/configuration/elasticsearch.html#configuration > > for more details. > > Cheers, > Jochen > > On Thursday, 24 November 2016 16:29:57 UTC+1, Dylan Humphreys wrote: >> >> Yes, its this one again. Sadly none of the other cases of this seem to >> fix my problem, although I have an idea of where things are going wrong. >> >> bddvgl1:~ # dpkg -l elasticsearch graylog-server | grep ii | awk '{print >> $1,$2,$3}' >> ii elasticsearch 2.3.5 >> ii graylog-server 2.1.2-1 >> bddvgl1:~ # >> >> My Elasticsearch cluster (currently one node to make things simple) is >> hale and hearty: >> >> bddvgl1:~ # curl -s bddvgl1:9200/_cluster/health?pretty >> >> { >> "cluster_name" : "graylog", >> "status" : "green", >> "timed_out" : false, >> "number_of_nodes" : 1, >> "number_of_data_nodes" : 1, >> "active_primary_shards" : 0, >> "active_shards" : 0, >> "relocating_shards" : 0, >> "initializing_shards" : 0, >> "unassigned_shards" : 0, >> "delayed_unassigned_shards" : 0, >> "number_of_pending_tasks" : 0, >> "number_of_in_flight_fetch" : 0, >> "task_max_waiting_in_queue_millis" : 0, >> "active_shards_percent_as_number" : 100.0 >> } >> bddvgl1:~ # >> >> >> I think the "problem" is with (or with my understanding of) the >> elasticsearch node graylog uses to communicate with the cluster. I >> *believe* I have that binding to 127.0.0.1 >> (from my server.conf) >> >> elasticsearch_cluster_name = graylog >> elasticsearch_node_name_prefix = graylog- >> elasticsearch_transport_tcp_port = 9300 >> elasticsearch_network_host = 127.0.0.1 >> elasticsearch_network_bind_host = 127.0.0.1 >> elasticsearch_http_enabled = true >> >> Sure enough, when I start graylog, I get this: >> >> bddvgl1:~ # netstat -tulpn | grep -E "(9300|9200)" >> tcp6 0 0 127.0.0.1:9200 :::* >> LISTEN 15627/java >> tcp6 0 0 10.0.34.128:9200 :::* >> LISTEN 15281/java >> tcp6 0 0 127.0.0.1:9300 :::* >> LISTEN 15627/java >> tcp6 0 0 10.0.34.128:9300 :::* >> LISTEN 15281/java >> bddvgl1:~ # >> >> >> The 9200 & 9300 on the RFC1918 addresses are my elasticsearch cluster >> that shows healthy above. >> I enabled http on the graylog elasticsearch node to help diagnose my >> problem. Sure enough: >> >> bddvgl1:~ # curl -s localhost:9200/_cluster/health?pretty >> { >> "error" : { >> "root_cause" : [ { >> "type" : "master_not_discovered_exception", >> "reason" : null >> } ], >> "type" : "master_not_discovered_exception", >> "reason" : null >> }, >> "status" : 503 >> } >> >> >> Which is essentially what Im getting in the logs. >> >> I have tried this: >> elasticsearch_discovery_zen_ping_unicast_hosts = bddvgl1:9300 >> and this >> elasticsearch_discovery_zen_ping_unicast_hosts = 10.0.34.128:9300 >> and this >> elasticsearch_discovery_zen_ping_unicast_hosts = 127.0.0.1:9300 >> >> But it never seems to connect. In fact If I comment that out entirely, I >> still get the same error mentioned in the subject. >> >> Primarily, where am I going wrong? also, does elasticsearch_blah refer to >> the graylog elasticsearch instance, or (in the case >> of elasticsearch_discovery_zen_ping_unicast_hosts) does that ask for the >> ACTUAL data storing nodes of my elasticsearch cluster? >> >> Thanks in advance! >> >> Dylan >> >> >> >> >> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/26f76749-7aae-4967-b3cc-db815a4901ab%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
