maven - KafkaSpout BrokerHosts error -
i trying compile , run storm-kafka-starter project @ https://github.com/thehydroimpulse/storm-kafka-starter
the main function kafkatopology looks like:
public class kafkatopology { public static void main(string[] args) throws exception { list<string> hosts = new arraylist<string>(); hosts.add("localhost"); spoutconfig kafkaconf = new spoutconfig(statichosts.fromhoststring(hosts,1), "test-topic","/kafkastorm","discovery"); kafkaconf.scheme = new schemeasmultischeme(new stringscheme()); kafkaconf.forcestartoffsettime(-2); topologybuilder builder = new topologybuilder(); builder.setspout("spout", kafkaspout, 2); builder.setbolt("printer", new printerbolt()).shufflegrouping("spout"); config config = new config(); config.setdebug(true); if(args!=null && args.length > 0) { config.setnumworkers(3); stormsubmitter.submittopology(args[0], config, builder.createtopology()); } else { config.setmaxtaskparallelism(3); localcluster cluster = new localcluster(); cluster.submittopology("kafka", config, builder.createtopology()); thread.sleep(10000); cluster.shutdown(); } } }
the jar compiles using maven. on running topology, error:
exception in thread "main" java.lang.noclassdeffounderror: storm/kafka/kafkaconfig$brokerhosts @ java.lang.class.getdeclaredmethods0(native method) @ java.lang.class.privategetdeclaredmethods(class.java:2451) @ java.lang.class.getmethod0(class.java:2694) @ java.lang.class.getmethod(class.java:1622) @ sun.launcher.launcherhelper.getmainmethod(launcherhelper.java:494) @ sun.launcher.launcherhelper.checkandloadmain(launcherhelper.java:486) caused by: java.lang.classnotfoundexception: storm.kafka.kafkaconfig$brokerhosts @ java.net.urlclassloader$1.run(urlclassloader.java:366) @ java.net.urlclassloader$1.run(urlclassloader.java:355) @ java.security.accesscontroller.doprivileged(native method) @ java.net.urlclassloader.findclass(urlclassloader.java:354) @ java.lang.classloader.loadclass(classloader.java:423) @ sun.misc.launcher$appclassloader.loadclass(launcher.java:308) @ java.lang.classloader.loadclass(classloader.java:356) ... 6 more
the local repository have brokerhosts in storm-kafka jar , have imported kafkaconfig library in java file. cannot figure out cause of error. suggestions appreciated.
i had similar issues using 0.9.2_incubating version of apache storm.
the isssue caused because actual storm distribution doesn't have kafka libraries in /lib folder. able resolve error copying following libraries (that used compile & build topology) /lib folder ran storm
- storm-kafka-0.9.2-incubating.jar
- kafka_2.10-0.8.1.1.jar
- scala-library-2.10.1.jar
remember actual versions in case might vary. take ones use build storm topology (i.e. .m2 or .gradle dependencies folder)
note: not using exact same starter project mentioned above fix similar.
Comments
Post a Comment