0

We are doing Web farming (Tomcat) for scalability and to increase throughput. But we are not getting results as expected.

Scenario 1- (Standalone Tomcat)

Load Generator       Jmeter (One instance)
Server               Tomcat(One instance)
Throughput           21k per second

Scenario 2- (Web Farm Scenario)

Load Generator         Jmeter  [Three instances]  
Load Balancer          Nginx 
Server                 Tomcat [3 tomcats on three different machines]
Session Replication    Delta Manager
Throughput             30k per second

Jmeter configuration:

No of Threads (users)         1000
Ramp up period (in seconds)   1
Loop count                    forever

Now we don't understand where we are doing wrong, because if one tomcat can give 21K per second then 3 Tomcats should give at least 50K logically.Our tomcat configuration is as given below

Server.xml:

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" 
                        channelSendOptions="8">
                <Channel className="org.apache.catalina.tribes.group.GroupChannel">
                <Membership   className="org.apache.catalina.tribes.membership.McastService"
                                address="228.0.0.4"
                                port="45564"
                                frequency="500"
                                dropTime="3000"/>

                <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                </Sender>
                <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                                address="XXX.XXX.XXX.90"
                                port="4000"       
                                autoBind="100"
                                selectorTimeout="5000"
                                maxThreads="6"/>
                <Interceptor  className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/></Channel>
                <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
       <!--<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>-->
                <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
                </Cluster>

Context.xml:

<Manager className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>
Has QUIT--Anony-Mousse
  • 76,138
  • 12
  • 138
  • 194
Krishnaveer
  • 53
  • 3

1 Answers1

0

I would suggest you add some reasonable ramp-up instead of keeping it 1 second.

  • Can you check if your load balancer is able to handle all the incoming requests in case of web farming?
  • Can you also confirm if both the tests were executed in same network?

In some cases I observed that the web server/ loadbalancer is not able to handle incoming requests and may throttle the throughput. Also keep eye on resource utilization in case of web farming. If the requests are coming in the CPU utilization should be comparable with single instance.

rachna bafna
  • 501
  • 3
  • 7
  • Thnx for you valuable suggestion. We have tested it with Nginx as well as Kemp load balancer but we found same result. and executed in same network. – Krishnaveer May 08 '16 at 04:15
  • Can you share resource utilization details? It seems your systems are saturating in that case. Also check the network utilization for single/web farming. This will give you a better idea of requests that are coming in. Check the JMeter reports for number of samples and response time. This will give you better idea. – rachna bafna May 11 '16 at 04:57
  • Network -- 1 GB n/w bandwidth. When running with 1 Tomcat instance CPU utilization is ~ 75% and memory used 1.5 GB of 16 GB. While running with 3 Tomcat instances CPU utilization is 45 %, 50% and 45% respectively. and memory utilization is 1.4GB of 16 GB, 1.5 GB of 16 GB and 1.5 GB of 16 GB respectively. – Krishnaveer May 11 '16 at 11:19
  • Can you also share the resource utilization details of load balancer? Also try and take thread dumps to see if something is blocking the threads? – rachna bafna May 11 '16 at 21:45
  • Less than 5% resource utilization of load balancer. – Krishnaveer May 12 '16 at 13:06
  • Now we are doing it in very simple way. We have exposed a spring REST service and trying to scale using delta manager as explained above. Still throughput is not getting scaled as it should be. – Krishnaveer May 15 '16 at 03:54
  • Are you still using load balancer? Have you been observing all the logs to look for any errors during the tests? – rachna bafna May 17 '16 at 21:12