I’ve tested this many times, and yes, your understanding is correct.
Owner/Contributor can browse and modify blobs in the Azure Portal because the portal uses account keys, not data-plane RBAC. Those roles have the listKeys permission, which lets the portal pull the storage keys and act as if it was the storage account itself.
Users who only have data-plane roles cannot use the portal UI because the portal won’t use AAD-based data-plane RBAC for browsing. But they can still read/write blobs through CLI/SDK because those tools use proper AAD authentication.
So, the behavior you described is exactly how Azure works today