ArgumentException while Saving TIFF as JPEG

In one of project I'm involving, we are doing some kind of picture processing. If picture comes from the digital camera as .tiff-file, we are doing rotation (if this specified in exif-header) and saving it as high-quality jpeg to conserve disk space. We've discovered that for some pictures we were getting ArgumentException while doing Image.Save(string, ImageCodecInfo, EncoderParameters) MSDN does not mention anything about that, but since the whole image class is just a wrapper around GDI+, we are basically getting error from the corresponding WinAPI function. Funny enough I was able to save exactly this image as bmp, but of cause it does not help us to minimize space usage.
So I started to trace what else we are doing with the image. I found out that we are also stripping all exif information via Image.RemovePropertyItem() because sometimes thumbnails which are stored there can be quite large. And it appears that if we are preserving certain items in exif (namely TIFFTAG_EXIFIFD and TIFFTAG_PHOTOSHOP), image can be saved without any problems.