0

I'm looking to setup a development testing environment and while I have things mostly setup I'm having difficulty connecting from the host machine (which is running Visual Studio) to the Cassandra cluster in docker (using Docker Desktop for Windows). I'm guessing I'm just messing up the configuration settings and/or missing a setting; but cannot determine how to make this work (it does work for single node; just not multiple nodes, which is what I want to work with)

What I have setup:

docker run --name tnode1 -d -e CASSANDRA_CLUSTER_NAME=tcluster -e CASSANDRA_DC=TDC1 -e CASSANDRA_RACK=TRAC1 -e CASSANDRA_BROADCAST_ADDRESS=10.0.75.2 -e CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch cassandra

I've also tried with the -p switch

docker run --name tnode1 -d -p 9042:9042 -e CASSANDRA_CLUSTER_NAME=tcluster -e CASSANDRA_DC=TDC1 -e CASSANDRA_RACK=TRAC1 -e CASSANDRA_BROADCAST_ADDRESS=10.0.75.2 -e CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch cassandra

This seems to work in terms of if I put data in this and then connect to 10.0.75.2 from Visual Studio, everything works as I'd expect it to. The issue comes when I go to add another node.

I've tried a number of ways, but it seems like they all end up with the second note starting up and then exiting and never joining the cluster. If I leave out the CASSANDRA_BORADCAST_ADDRESS when setting up tnode1 then the cluster works, but I cannot get to it from Visual Studio.

Second Node (General Information)

docker inspect -f '{{ .NetworkSettings.IPAddress }}' tnode1 returns 172.17.0.2 docker exe -it tnode1 nodetool status has the address as 10.0.75.2

Second Node (Attempt 1)

docker run --name tnode2 -d -e CASSANDRA_CLUSTER_NAME=tcluster -e CASSANDRA_DC=TDC1 -e CASSANDRA_RACK=TRAC1 -e CASSANDRA_SEEDS=172.17.0.2 ca ssandra result: running nodetool status doesn't show tnode2 ... running docker ps -a shows "Exited (3) 30 seconds ago" for status

Second Node (Attempt 2)

docker run --name tnode2 -d -e CASSANDRA_CLUSTER_NAME=tcluster -e CASSANDRA_DC=TDC1 -e CASSANDRA_RACK=TRAC1 -e CASSANDRA_SEEDS=10.0.75.2 ca ssandra result: running nodetool status doesn't show tnode2 ... running docker ps -a shows "Exited (3) 28 seconds ago" for status

It seems as if the seed value isn't connecting and the new node then stops as a result. Again if I take out the broadcast address then the node creation works, but I cannot connect from the host machine; I've attempted to add listener address, and -p parameter on tnode1 creation, but similar results.

Any assistance would be greatly appreciated.

veben
  • 19,637
  • 14
  • 60
  • 80
ChrisHDog
  • 4,473
  • 8
  • 51
  • 77
  • I'm happy to take out all the broadcast items if I know how to connect to the cluster; when I do that and attempt 10.0.75.2 or 127.0.0.1 or 172.17.0.2 they all fail with NoHostAvailableException and then I put in the broadcast that goes away and enables the connection. – ChrisHDog Sep 02 '17 at 15:16
  • This is not cassandra specific, but when creating clusters it's generally easier to run the nodes all on a [user defined network](https://docs.docker.com/engine/userguide/networking/#user-defined-networks) so they can all communicate like they would on a physical network, with no Docker in between. Using a [docker compose v2+ definition](https://docs.docker.com/compose/compose-file/) for the cluster will set the network up for you by default. – Matt Sep 03 '17 at 00:39
  • @Matt thanks, that was what I initially had for all the clusters, but then I couldn't connect to it from the host machine (where the visual studio/code instance was) ... so I added the broadcast flag to get that connection work and with that I lost the ability to connect nodes ... so it seems close; just need to know where/how to point the nodes so they all see each other and I can connect to from outside – ChrisHDog Sep 03 '17 at 06:09

1 Answers1

2

The following compose cluster definition works for me, I can connect with cqlsh from the Docker host and run the test cql.

The memory is adjusted down from the default 1.5G as every time I brought up a new node, the previous node would exit due to a lack of memory in the Docker VM.

version: "2.1"

services:
  cassandra-1:
    image: cassandra
    environment:
      CASSANDRA_CLUSTER_NAME: tcluster
      CASSANDRA_DC: TDC1
      MAX_HEAP_SIZE: 600M
      HEAP_NEWSIZE: 100M
    ports:
      - '9042:9042'
      - '9160:9160'
    networks:
      cassclus:
        ipv4_address: 10.0.75.11

  cassandra-2:
    image: cassandra
    environment:
      CASSANDRA_CLUSTER_NAME: tcluster
      CASSANDRA_DC: TDC1
      CASSANDRA_SEEDS: 10.0.75.11
      MAX_HEAP_SIZE: 600M
      HEAP_NEWSIZE: 100M
    networks:
      cassclus:
        ipv4_address: 10.0.75.12

  cassandra-3:
    image: cassandra
    environment:
      CASSANDRA_CLUSTER_NAME: tcluster
      CASSANDRA_DC: TDC1
      CASSANDRA_SEEDS: 10.0.75.11
      MAX_HEAP_SIZE: 600M
      HEAP_NEWSIZE: 100M
    networks:
      cassclus:
        ipv4_address: 10.0.75.13

networks:
  cassclus:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 10.0.75.0/24
          gateway: 10.0.75.1
Matt
  • 68,711
  • 7
  • 155
  • 158