0

I'm running a WSO2 container with all products together(apim-is-as-km-with-analytics) using mysql as database and I'm facing an error when the docker compose starts. My problem is on wso2-is server, it shows the following message:

[2021-02-26 21:38:17,531] []  INFO {org.wso2.carbon.mex2.internal.DynamicCRMCustomMexComponent} - DynamicCRMSupport MexServiceComponent bundle activated successfully.
 [2021-02-26 21:38:19,923] []  INFO {org.apache.jasper.servlet.TldScanner} - At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
 [2021-02-26 21:38:20,098] []  INFO {org.wso2.carbon.identity.authenticator.x509Certificate.internal.X509CertificateServiceComponent} - X509 Certificate Servlet activated successfully..
 [2021-02-26 21:38:23,807] [] ERROR {org.wso2.carbon.user.core.common.DefaultRealm} - nullType class java.lang.reflect.InvocationTargetException org.wso2.carbon.user.core.UserStoreException: nullType class java.lang.reflect.InvocationTargetException
    at org.wso2.carbon.user.core.common.DefaultRealm.createObjectWithOptions(DefaultRealm.java:397)
    at org.wso2.carbon.user.core.common.DefaultRealm.initializeObjects(DefaultRealm.java:224)
    at org.wso2.carbon.user.core.common.DefaultRealm.init(DefaultRealm.java:129)
    at org.wso2.carbon.user.core.common.DefaultRealmService.initializeRealm(DefaultRealmService.java:276)
    at org.wso2.carbon.user.core.common.DefaultRealmService.<init>(DefaultRealmService.java:102)
    at org.wso2.carbon.user.core.common.DefaultRealmService.<init>(DefaultRealmService.java:115)
    at org.wso2.carbon.user.core.internal.Activator.startDeploy(Activator.java:72)
    at org.wso2.carbon.user.core.internal.BundleCheckActivator.start(BundleCheckActivator.java:61)
    at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:842)
    at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:1)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at org.eclipse.osgi.internal.framework.BundleContextImpl.startActivator(BundleContextImpl.java:834)
    at org.eclipse.osgi.internal.framework.BundleContextImpl.start(BundleContextImpl.java:791)
    at org.eclipse.osgi.internal.framework.EquinoxBundle.startWorker0(EquinoxBundle.java:1013)
    at org.eclipse.osgi.internal.framework.EquinoxBundle$EquinoxModule.startWorker(EquinoxBundle.java:365)
    at org.eclipse.osgi.container.Module.doStart(Module.java:598)
    at org.eclipse.osgi.container.Module.start(Module.java:462)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel$1.run(ModuleContainer.java:1820)
    at org.eclipse.osgi.internal.framework.EquinoxContainerAdaptor$2$1.execute(EquinoxContainerAdaptor.java:150)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1813)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1770)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.doContainerStartLevel(ModuleContainer.java:1735)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1661)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:345)
 Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at org.wso2.carbon.user.core.common.DefaultRealm.createObjectWithOptions(DefaultRealm.java:351)
    ... 25 more
 Caused by: org.wso2.carbon.user.core.UserStoreException: DB error occurred while persisting domain : PRIMARY & tenant id : -1234
    at org.wso2.carbon.user.core.util.UserCoreUtil.persistDomain(UserCoreUtil.java:871)
    at org.wso2.carbon.user.core.common.AbstractUserStoreManager.persistDomain(AbstractUserStoreManager.java:8595)
    at org.wso2.carbon.user.core.ldap.ReadOnlyLDAPUserStoreManager.<init>(ReadOnlyLDAPUserStoreManager.java:243)
    at org.wso2.carbon.user.core.ldap.UniqueIDReadOnlyLDAPUserStoreManager.<init>(UniqueIDReadOnlyLDAPUserStoreManager.java:148)
    at org.wso2.carbon.user.core.ldap.UniqueIDReadWriteLDAPUserStoreManager.<init>(UniqueIDReadWriteLDAPUserStoreManager.java:122)
    ... 30 more
 Caused by: java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.

The last line on log shows a message

 Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

It's estrange because I already put the mysql driver on Dockerfile from wso2-is:

# copy MySQL JDBC connector to server home as a third party library
COPY --chown=wso2carbon:wso2 /binary/mysql-connector-java-8.0.17.jar ${WSO2_SERVER_HOME}/repository/components/dropins/

Does anybody know what am I missing?

I already check all jdbc address on toml files.
wso2am:3.2.0-alpine
wso2is:5.10.0-alpine
mysql:5.7.33
wso2am-analytics-dashboard:3.2.0-alpine
wso2am-analytics-worker:3.2.0-alpine

Community
  • 1
  • 1
Aldo Inácio da Silva
  • 824
  • 2
  • 14
  • 38

1 Answers1

0

My problem was with mysql container. Even though the container's health was ok it was not ready yet:

mysql:5.7.33   "docker-entrypoint.s…"   ... Up 5 minutes (healthy)    0.0.0.0:3306->3306/tcp, 33060/tcp

When I try to access using workbench it showed the message:

...Error Code: 2013 Lost connection to MySQL server at 'reading initial communication packet', system error: 0

Solution:

I added a script (wait-for-it) in all Dockerfiles(wso2am,wso2is,wso2am-analytics-worker) to check the availability of mysql before start wso2 server.

Dockerfile:

# install required packages. Need bash to wait-for-it
RUN apk add --no-cache netcat-openbsd \
        bash

# Add wait-for-it 
COPY --chown=wso2carbon:wso2 wait-for-it.sh ${USER_HOME}/

# initiate container after check if mysql is available and start WSO2 Carbon server
ENTRYPOINT ["/home/wso2carbon/wait-for-it.sh" , "cup-mysql:3306" , "--strict" , \
"--timeout=300" , "--" ,"/home/wso2carbon/docker-entrypoint.sh"]

And also increased the start_period of all containers to more than 60s since mysql spends 85s to start.

docker container logs wso2is:

wait-for-it.sh: waiting 300 seconds for cup-mysql:3306
wait-for-it.sh: cup-mysql:3306 is available after 85 seconds
JAVA_HOME environment variable is set to /opt/java/openjdk
CARBON_HOME environment variable is set to /home/wso2carbon/wso2is-5.10.0

docker-compose.yml:

  mysql:
    container_name: cup-mysql
    image: mysql:5.7.33
    ports:
      - "3306:3306"
    networks:
      - wso2-network
    environment:
      MYSQL_ROOT_PASSWORD: root
    volumes:
      - ./conf/mysql/scripts:/docker-entrypoint-initdb.d
      - ./conf/mysql/conf/my.cnf:/etc/mysql/my.cnf
    ulimits:
      nofile:
        soft: 20000
        hard: 40000
    command: [--ssl=0]
    healthcheck:
      test: ["CMD", "mysqladmin" ,"ping", "-uroot", "-proot"]
      interval: 30s
      timeout: 60s
      retries: 5
      start_period: 80s
  is-as-km:
    container_name: cup-is-as-km
    build: ./dockerfiles/is-as-km/closeup
    image: wso2is:5.10.0-alpine
    healthcheck:
      test: ["CMD", "nc", "-z","localhost", "9443"]
      interval: 30s
      start_period: 180s
      retries: 20
    depends_on:
      mysql:
        condition: service_healthy
    volumes:
      - ./conf/is-as-km:/home/wso2carbon/wso2-config-volume
    ports:
      - "9444:9443"
    ....
Aldo Inácio da Silva
  • 824
  • 2
  • 14
  • 38