MIP File SDK 1.18.124 buffers the entire decrypted output in native memory - is there a streaming / bounded-memory way to decrypt large protected files?

Question

MIP File SDK 1.18.124 buffers the entire decrypted output in native memory - is there a streaming / bounded-memory way to decrypt large protected files?

박진식 0

When decrypting a large protected file (PFILE / .pzip, 2,272,066,451 bytes, > int.MaxValue) with the

MIP File SDK 1.18.124 (Windows x64), private commit peaks at ~the full file size (~2,190 MB) for every decryption.

The decrypted OUTPUT appears to be fully buffered in native (mip_file_sdk.dll), which causes severe transient memory spikes under concurrency.

Is there a supported way to decrypt to disk with bounded memory (streaming), or is this by design?

What I measured (same file, sampled private bytes; both C++ and .NET File SDK):

 1. IFileHandler.GetDecryptedTemporaryFileAsync (path input)       : ~2,191 MB peak

 2. GetDecryptedTemporaryFileAsync (file-backed input Stream)      : ~2,191 MB peak (input is NOT the cause)

 3. GetDecryptedTemporaryStreamAsync + chunked read               : ~4,362 MB peak (output double-buffered; worse)

 4. RemoveProtection + CommitAsync                                : ~2,191 MB peak

Managed heap stays ~7 MB, so the buffer is native, inside mip_file_sdk.dll.

Additional issue on .NET: GetDecryptedTemporaryStreamAsync returns a fully-materialized MemoryStream,

which fails for files > int.MaxValue (2,147,483,647) with a read-not-supported error.

Is there any supported streaming/bounded-memory file decryption path we're missing?
Can ProtectionHandler::CreateProtectedStream be used over a PFILE payload directly? The File SDK does not expose the protected payload's start offset / length, so we can't bypass to the Protection SDK.
On .NET, is there a way to avoid the MemoryStream materialization so decryption works for files > 2 GB? Environment: Windows 11 x64, MIP File SDK 1.18.124 (Windows NuGet + native mip_*.dll),
```
                   authenticated via client credentials (app-only).
```

Himaja Y 375 Reputation points Microsoft External Staff Moderator

2026-09-04T13:46:32.04+00:00
Hi @박진식 ,

Thank you for reaching out to the Microsoft Q&A forum.

Thank you for providing the detailed test results. Based on the behavior observed, the memory increase is occurring in the native MIP SDK layer and is not caused by the managed heap or input stream.

Large-file decryption with the MIP File SDK requires available memory approximately equal to the file size. Therefore, the native-memory usage you observed is consistent with the SDK’s current large-file processing behavior.

Answers to your questions

Streaming or bounded-memory decryption There is currently no documented File SDK API that provides true chunk-based decryption directly to disk with bounded memory for PFILE/PZIP files.

Using ProtectionHandler::CreateProtectedStream PFILE/PZIP is handled as a File SDK container. Because the SDK does not expose the protected payload offset and length, passing the internal payload directly to the Protection SDK is not a documented or supported workaround.

Files larger than 2 GB in .NET Avoid GetDecryptedTemporaryStreamAsync for multi-GB files because the returned in-memory stream can encounter .NET size limitations. Use GetDecryptedTemporaryFileAsync instead, and then process the resulting temporary file using a disk-based FileStream. This avoids an additional managed-memory copy, but it will not eliminate the native-memory peak during decryption.

Recommended mitigation

Use GetDecryptedTemporaryFileAsync.

Limit the number of concurrent decryptions based on available memory.

Avoid GetDecryptedTemporaryStreamAsync for files near or above 2 GB.

Ensure sufficient disk space and securely delete temporary decrypted files after processing.

Test with the latest supported MIP SDK version.

Since this directly affects large-file and concurrent-processing scenarios, I recommend opening a Microsoft Support request with the reproduction project, MIP SDK logs, native-memory trace, exact file type, and peak private-byte measurements. This will allow the product team to confirm whether the behavior is expected in version 1.18.124 or whether a fix is available in a later build.

And also please share the requested details in private chat
박진식 0 Reputation points

2026-09-04T15:06:54.4866667+00:00
Hi @Himaja Y

Thank you for the detailed reply and for confirming that the native memory usage is consistent with the current large-file processing behavior of the File SDK.

To make the behavior concrete, here are measurements from a standalone reproduction that uses only the public Microsoft.InformationProtection.File SDK (1.18.124) plus MSAL — no third-party code. Private bytes were sampled at 100 ms around GetDecryptedTemporaryFileAsync (peak delta over baseline):

A 2.36 GB PFILE (larger than int.MaxValue = 2,147,483,647 bytes):

success = True inputSize = 2,361,903,950 bytes outputSize = 2,361,860,146 bytes privateBytes peak = 2,263 MB (absolute 2,290 MB) <- about file size, native workingSet peak = 2,273 MB managedHeap peak = 2 MB <- managed heap unaffected elapsed = 65,387 ms

A 0.56 GB PFILE (faster run):

inputSize = 590,489,571 bytes privateBytes peak = 570 MB (absolute 597 MB) <- about file size, native managedHeap peak = 0 MB

The private-commit peak tracks the input file size while the managed heap is unaffected, which points to buffering in the native layer (mip_file_sdk.dll). Under concurrency this scales as file_size x concurrent_decryptions.

Since Q&A is a public forum, I've opened an Azure support case (case number 2609040060002898) with the full standalone reproduction project and details, to keep tenant/credential information out of the public thread.

For the public record, my remaining forward-looking questions are:

Is the file-size native memory peak during GetDecryptedTemporaryFileAsync considered expected/by-design in 1.18.124, or a defect being tracked?

Is a bounded-memory or streaming file-decryption path planned for a future GA release (writing decrypted output to disk incrementally without buffering the whole content in native memory)?

Is there any supported low-memory decryption path available today that we may have missed?

I'm also happy to file this on the Microsoft feedback portal as a feature request if that helps prioritization. Thanks again.
Himaja Y 375 Reputation points Microsoft External Staff Moderator

2026-09-04T17:21:19.6133333+00:00

Hi @박진식 ,

Thank you for your patience. Let me check your case with our internal team, and I will get back to you once I have an update.

Your answer

Himaja Y 375 Reputation points Microsoft External Staff Moderator

2026-09-04T17:21:19.6133333+00:00

Hi @박진식 ,

Thank you for your patience. Let me check your case with our internal team, and I will get back to you once I have an update.