ESENT durable commit: log namespace and application-created directories on NTFS

Question

ESENT durable commit: log namespace and application-created directories on NTFS

Corrie Janse van vuuren 0

I am assessing the supported durability contract for an offline Windows application using the in-box ESENT engine on local NTFS, from a non-administrator account. This is a documentation question; no data-loss bug is being alleged.

Assume supported Windows 11, a correctly configured storage stack that honors flush/write-through requests, recovery/logging enabled, no lazy commit, and a successful outermost JetCommitTransaction. Hardware that falsely acknowledges persistence, media destruction and administrator tampering are outside scope.

The ESE transaction documentation promises persistent acknowledged commits, and the file-lifecycle documentation describes log rollover. I understand those guarantees to cover engine-managed rollover under supported configuration. Please confirm whether the application owes any extra namespace synchronization beyond successful documented ESE calls; I am not seeking a separate proof of each internal rename.

My unresolved bootstrap question is that JetCreateDatabase normally requires an existing directory. When the application has just created that directory and any ancestor directories, what supported non-administrator API sequence durably establishes the complete containing directory chain before the first committed transaction is relied on? Does ESENT cover this, or must the application establish it independently?

Please identify the documented APIs/configuration and supported Windows/ESENT versions. We cannot rely on privileged volume flushes or private JET APIs. A process-restart test alone is not being treated as a power-loss guarantee.

References: ESE transactions, JetCreateDatabase, FlushFileBuffers.

0 comments

1 answer

Your answer

Answer 1

Hi Corrie,

Thank you for laying out the scenario and its assumptions so clearly. I reviewed the available Microsoft documentation. Here are the direct answers to your questions. First, I could not find any documented requirement for the application to perform extra namespace synchronization for ESE-managed log rollover. ESE documents the durability of an acknowledged outermost transaction commit when lazy commit is not used. I also found no documented application-side step for separately synchronizing log-file creation, renaming, or rollover performed internally by ESE. However, the public documentation does not describe the persistence contract of each individual namespace operation performed inside the engine. For that reason, I can confirm only the published ESE transaction guarantee, not a separate guarantee for every internal file-system operation. Second, Yes. If JET_paramCreatePathIfNotExist is enabled before JetInit, ESE can create missing folders in a path used by the database engine. See:

Third, the official documentation confirms that the missing folders can be created. It does not state that JET_paramCreatePathIfNotExist, JetCreateDatabase, or the first successful transaction commit separately makes every newly created ancestor-directory entry durable against power loss. And the FlushFileBuffers function documents flushing a file handle and states that the handle must have GENERIC_WRITE access. It also describes flushing all open files through a volume handle, but that operation requires administrative privileges. The documentation does not provide a non-administrator sequence that explicitly guarantees persistence of a newly created directory chain as a separate namespace operation.

The ESE transactions documentation also states that, after the database engine acknowledges that a transaction has been committed, its changes are persistent in the database. The JetCommitTransaction documentation also explains that JET_bitCommitLazyFlush allows the call to return without waiting for the transaction to be flushed to the transaction log, at the cost of durability. For a normal outermost commit without lazy flush, the documented ESE transaction-durability guarantee applies. That guarantee addresses the committed database transaction. The documentation does not explicitly say that the same commit also establishes the durability of an application-created or newly created parent-directory chain.

Therefore, while the documented ESE calls cover transaction durability and can create the required path, I cannot point to a published Microsoft API contract that provides the additional directory-chain durability guarantee you are asking for.

As you noted, confirming that the path remains present after a process or system restart would show observed behavior, but it would not establish the requested sudden-power-loss guarantee.

Although the public documentation does not establish the additional directory-chain guarantee discussed above, if you have a specific reproducible scenario, please share the API sequence, Windows version, expected result, and actual result, and I can help you investigate the observed behavior.

Hope these information help. If you found my response helpful or informative, I would greatly appreciate it if you could follow this guide for your confirmation.

Thank you.