-
-
Notifications
You must be signed in to change notification settings - Fork 16.2k
New decompressor API #15667
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 4.2
Are you sure you want to change the base?
New decompressor API #15667
Conversation
codec-compression/src/main/java/io/netty/handler/codec/compression/Decompressor.java
Outdated
Show resolved
Hide resolved
codec-compression/src/main/java/io/netty/handler/codec/compression/DefensiveDecompressor.java
Show resolved
Hide resolved
codec-compression/src/test/java/io/netty/handler/codec/compression/Bzip2DecoderTest.java
Show resolved
Hide resolved
codec-compression/src/main/java/io/netty/handler/codec/compression/Decompressor.java
Outdated
Show resolved
Hide resolved
...ssion/src/main/java/io/netty/handler/codec/compression/BackpressureDecompressionHandler.java
Outdated
Show resolved
Hide resolved
...ssion/src/main/java/io/netty/handler/codec/compression/BackpressureDecompressionHandler.java
Outdated
Show resolved
Hide resolved
...ssion/src/main/java/io/netty/handler/codec/compression/BackpressureDecompressionHandler.java
Outdated
Show resolved
Hide resolved
...ssion/src/main/java/io/netty/handler/codec/compression/BackpressureDecompressionHandler.java
Show resolved
Hide resolved
...ssion/src/main/java/io/netty/handler/codec/compression/BackpressureDecompressionHandler.java
Outdated
Show resolved
Hide resolved
...ssion/src/main/java/io/netty/handler/codec/compression/BackpressureDecompressionHandler.java
Outdated
Show resolved
Hide resolved
codec-compression/src/main/java/io/netty/handler/codec/compression/BrotliDecompressor.java
Outdated
Show resolved
Hide resolved
codec-compression/src/main/java/io/netty/handler/codec/compression/Bzip2Decompressor.java
Outdated
Show resolved
Hide resolved
codec-compression/src/main/java/io/netty/handler/codec/compression/DefensiveDecompressor.java
Outdated
Show resolved
Hide resolved
|
@normanmaurer I've added new logic that allows a number of buffers to be forwarded before a readComplete, and a test. I hope that works for you. The logic is getting complicated though. PTAL. I've also changed the handler API to create the decompressor lazily, in handlerAdded. This looks a bit odd (you pass the decompressor builder, not the decompressor, to the handler) but I think it works as an API and it's safer. |
|
I've added a HttpContentDecoder and -decompressor replacement, based on the BackpressureDecompressionHandler. I've also deprecated all the old decoders. I still need to migrate DelegatingDecompressorFrameListener. It is a bit more difficult because it uses a different flow control mechanism. Probably it needs an independent Decompressor-based implementation, not based on BackpressureDecompressionHandler. Finally, I changed the decompressor API a little bit so that the alloc is required by the .build call, not when creating the builder. This complements the change in 9279fc0. Now, the BackpressureDecompressionHandler will use the alloc from the context, and you don't need to manually pass in the alloc at all before that. |
codec-http/src/main/java/io/netty/handler/codec/http/BackpressureHttpContentDecoder.java
Fixed
Show fixed
Hide fixed
codec-http/src/main/java/io/netty/handler/codec/http/BackpressureHttpContentDecoder.java
Outdated
Show resolved
Hide resolved
codec-http/src/main/java/io/netty/handler/codec/http/HttpDecompressionHandler.java
Dismissed
Show dismissed
Hide dismissed
…mal decompressing handler
|
Okay, I rebuilt the decompression codecs again. HTTP/1.1 pipelining has a lot of edge cases to handle. I decided to build a central I've also built a Http2FrameListener based on the new decompressors. It appears to be a lot simpler than the ChannelHandler approach, unless I missed some edge cases. I'm going on vacation for a few days so I might be slow to answer now. |
Motivation:
The current decompression decoders have a few issues:
Modification:
DecompressorAPI not based onChannelHandler. It is similar in design to the brotli API, with separate operations for adding input and removing output.BackpressureDecompressionHandlerthat uses the newDecompressorAPI.Result:
Lower overhead, better backpressure handling.
Work yet to be done:
BackpressureDecompressionHandlerHttpContentDecoderbased on the new decompressors