How to send a message via Kafka.Producer from a java servlet to Kafka

Question

I want to get the data from a HTTP Post, so in servlet I am implementing doPost like following

package PostHandler;

import java.io.IOException;
import java.util.Enumeration;
import java.util.Properties;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import kafka.producer.ProducerConfig;
import kafka.javaapi.producer.ProducerData;

@WebServlet(description = "This is to handle the posted data on HTTP", urlPatterns = { "/ClsPostHandler" })
public class ClsPostHandler extends HttpServlet 
{
protected void doPost(HttpServletRequest request,
            HttpServletResponse response) throws ServletException, IOException 
      {
            final kafka.javaapi.producer.Producer<Integer, String> producer;
            final Properties props = new Properties();
            props.put("serializer.class", "kafka.serializer.StringEncoder");
            props.put("zk.connect", "localhost:2181");
            producer = new kafka.javaapi.producer.Producer<Integer, String>(
                    new ProducerConfig(props));
            String messageStr = new String("Testing to publish on testing topic");
            producer.send(new ProducerData<Integer, String>("testing",
                    messageStr));
    }
}

Its throwing an runtime error as follows:

Jul 26, 2013 11:57:08 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [PostHandler.ClsPostHandler] in context with path [/JavaRest] threw exception [Servlet execution threw an exception] with root cause
java.lang.NoClassDefFoundError: kafka/javaapi/producer/Producer
    at PostHandler.ClsPostHandler.doPost(ClsPostHandler.java:105)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:562)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:395)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:250)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:188)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:166)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:302)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:679)

What is going wrong can some body help me in understanding?

Advance Thanks for your time.

Update: Below is my .classpath file content

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
    <classpathentry kind="src" path="src"/>
    <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/java-6-openjdk-amd64">
        <attributes>
            <attribute name="owner.project.facets" value="java"/>
        </attributes>
    </classpathentry>
    <classpathentry kind="con" path="org.eclipse.jst.server.core.container/org.eclipse.jst.server.tomcat.runtimeTarget/Apache Tomcat v7.0">
        <attributes>
            <attribute name="owner.project.facets" value="jst.web"/>
        </attributes>
    </classpathentry>
    <classpathentry kind="con" path="org.eclipse.jst.j2ee.internal.web.container"/>
    <classpathentry kind="con" path="org.eclipse.jst.j2ee.internal.module.container"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/lib_managed/scala_2.8.0/compile/apache-rat-0.8.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/lib_managed/scala_2.8.0/compile/apache-rat-core-0.8.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/lib_managed/scala_2.8.0/compile/apache-rat-tasks-0.8.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/lib_managed/scala_2.8.0/compile/commons-cli-1.2.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/lib_managed/scala_2.8.0/compile/commons-collections-3.2.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/lib_managed/scala_2.8.0/compile/commons-compress-1.0.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/lib_managed/scala_2.8.0/compile/commons-io-1.4.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/lib_managed/scala_2.8.0/compile/commons-lang-2.1.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/core/lib_managed/scala_2.8.0/compile/zookeeper-3.3.4.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/core/lib_managed/scala_2.8.0/compile/zkclient-0.1.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/core/lib_managed/scala_2.8.0/compile/snappy-java-1.0.4.1.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/core/lib_managed/scala_2.8.0/compile/log4j-1.2.15.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/core/lib_managed/scala_2.8.0/compile/jopt-simple-3.2.jar"/>
    <classpathentry kind="lib" path="/usr/local/kafka/kafka-0.7.2-incubating-src/core/target/scala_2.8.0/kafka-0.7.2.jar"/>
    <classpathentry kind="lib" path="/usr/local/scala-2.9.1.final/lib/jline.jar"/>
    <classpathentry kind="lib" path="/usr/local/scala-2.9.1.final/lib/scala-library.jar"/>
    <classpathentry kind="output" path="build/classes"/>
</classpath>

om-nom-nom · Accepted Answer · 2013-07-26T08:25:17.810

1

You have to place kafka jar on your runtime classpath

edited Jul 26 '13 at 08:25

answered Jul 26 '13 at 06:57

om-nom-nom

62,329
13
183
228

In my project I have added to libraries i.e. kafka-0.7.2 – Amol M Kulkarni Jul 26 '13 at 07:03
1

@AmolMKulkarni the thing is that you must have it on classpath both at compile and runtime. If you don't have lib on compile time your code just won't compile, if you don't have lib at run time you got the error you've got (NoClassDefFound) – om-nom-nom Jul 26 '13 at 08:14
So how should I get to know which location is set to CLASSPATH.. I am running tomcat7 from eclipse Kepler.. – Amol M Kulkarni Jul 26 '13 at 08:33
In case you need an example for setting up a kafka producer, here is an example : https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example – Myna Sep 19 '14 at 14:19

How to send a message via Kafka.Producer from a java servlet to Kafka

1 Answers1

Linked