Using msgpack with storm 1.0.1 throws KeyError #35
Description
From @amitripshtos on July 22, 2016 13:35
hey,
im trying to use msgpack with streamparse using Dan`s guide on one of the issues around this repo.
Im using:
streamparse 3.0.0 dev3
apache storm 1.0.1
python 3.5.1
java 1.8
It seems that it working on using msgpack in java, but I get KeyError in the "python" side of msgpack:
java.lang.RuntimeException: Error when launching multilang subprocess Traceback (most recent call last): File "/usr/local/bin/streamparse_run", line 9, in <module> load_entry_point('streamparse==3.0.0.dev3', 'console_scripts', 'streamparse_run')() File "/usr/local/lib/python3.5/site-packages/streamparse/run.py", line 37, in main cls(serializer=args.serializer).run() File "/usr/local/lib/python3.5/site-packages/pystorm/component.py", line 474, in run storm_conf, context = self.read_handshake() File "/usr/local/lib/python3.5/site-packages/pystorm/component.py", line 303, in read_handshake pid_dir, _conf, _context = msg['pidDir'], msg['conf'], msg['context'] KeyError: 'pidDir' at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:89) at org.apache.storm.spout.ShellSpout.open(ShellSpout.java:88) at org.apache.storm.daemon.executor$fn__7885$fn__7900.invoke(executor.clj:601) at org.apache.storm.util$async_loop$fn__625.invoke(util.clj:482) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.EOFException at org.msgpack.io.StreamInput.readByte(StreamInput.java:60) at org.msgpack.unpacker.MessagePackUnpacker.getHeadByte(MessagePackUnpacker.java:66) at org.msgpack.unpacker.MessagePackUnpacker.trySkipNil(MessagePackUnpacker.java:396) at org.msgpack.template.MapTemplate.read(MapTemplate.java:59) at org.msgpack.template.MapTemplate.read(MapTemplate.java:27) at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31) at org.msgpack.MessagePack.read(MessagePack.java:527) at org.msgpack.MessagePack.read(MessagePack.java:496) at com.yelp.pyleus.serializer.MessagePackSerializer.readMessage(MessagePackSerializer.java:205) at com.yelp.pyleus.serializer.MessagePackSerializer.connect(MessagePackSerializer.java:67) at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:85) ... 5 more
I could not fix that in any cases and I hope you guys have any solution
thank you very much!
Copied from original issue: pystorm/streamparse#278