Ode to SharpZipLib

Not much technical stuff expected in this post, nor any sample code or screenshots. I simply wanted to share an experience with different compression solutions under high loads. Disclaimer may be required though: Nothing in this post is intended to slander any compression solution whatsoever. 

Different compression solutions for developers exist for years now, within the .NET framework and outside it. I had several different projects which required compression solutions in various scenarios. I always preferred and looked for free solutions over a paid-for solution, and I always preferred a managed solution which was easy to work with, over other alternatives, such as older COM solutions or simply Dll external function calls. Amongst the different compression solutions I used are .NET’s GZipStream and Deflate, SharpZipLib and 7zip. Possibly others that I don’t recall.

I guess that when a .NET developer is looking for a compression solution, there are several features which are essential, these are:

1. Reliability – uncompressed data must be identical to the original input; trivial request, but one cannot do without.
2. Stability – must not fail under high stress loads.
3. Decent compression rate/percentage.

Arguable, but I find the following necessary but not mandatory:
4. Performance – Good compression speed.
5. A free solution or at least cheap.
6. Managed code and general ease of use.

Lately I was looking again for updated compression solutions. I googled and found posts like Peter Bromberg’s compression comparison. which compared different compression solutions, and performed some tests of my own. The drive to make this search arose from an application using 7zip compression, which under stress tests began to fail with Out of Memory exceptions. Searching through the web I found that I wasn’t the only one, and this seems like a known issue. Although 7-Zip is my favorite client application for zipping, and although it was great in previous solutions, in this particular development many compressions are made simulatenously and under a heavy load, so I had to find an alternate solution which would be 100% stable. Attempts to use different 7zip “helpers” and reuse the LZMA demo code supplied with the SDK to solve this failed altogether, which led to this search in the first place.

.NET Framework’s GZipStream and Deflate were tested and found to be both reliable and stable, but provided  relatively poor compression (if at all). Other solutions were either not free, or they were tested but found unreliable, or they weren’t tested for various reasons. However, the good ol’ SharpZipLib proved the best of all worlds. Although it’s compression rate isn’t as good as 7zip’s, it was way better than GZipStream/Deflate and other alternatives, and it proved reliable as far as data integrity or memory consumption.

I know, I know. SharpZipLib has an “old fashioned” API which is a little cumbersome; it’s not the best compression in the world, and it’s speed isn’t the fastest out there. BUT, it’s licensed so that it can be used freely in closed source applications; it provides a compression rate and speed which are reasonable; it has managed code which is “not-so-bad”; but most importantly – it was stable. Under high loads it didn’t crash, not even once. The memory consumption seemed reasonable. The CPU didn’t go crazy comparing to stress test without compression at all.

So, that’s my latest experience with compression solutions for the .NET. I reckon that there are alternatives which may be better – feel free to share your thoughts and experience.