Azure AI Video Indexer frequently asked questions

Azure AI Video Indexer is a cloud application, part of Azure AI services, enables you to extract the insights from your videos using Azure AI Video Indexer video and audio models. For more information, see Overview.

Some of the operations that Azure AI Video Indexer can perform on media files include:

• Identifying and extracting speech and identify speakers.
• Identifying and extracting on-screen text in a video.
• Detecting objects in a video file.
• Identify brands (for example: Microsoft) from audio tracks and on-screen text in a video.
• Detecting and recognizing faces from a database of celebrities and a user-defined database of faces.
• Extracting topics discussed but not necessarily mentioned in audio and video content.
• Creating closed captions or subtitles from the audio track.

For more information and more Azure AI Video Indexer features, see Overview.

Azure AI Video Indexer includes a trial offering that provides you with 2,400 free indexing minutes when using the Azure AI Video Indexer website or the Azure AI Video Indexer API (see developer portal).

To index videos and audio flies at scale, you can connect Azure AI Video Indexer to a paid Microsoft Azure subscription. You can find more information on pricing on the pricing page.

You can find more information on getting started in Get started.

You can use the Azure AI Video Indexer web-based interface to evaluate, configure, and manage your account with no coding required. When you are ready to develop more complex applications, you can use the Azure AI Video Indexer API to integrate Azure AI Video Indexer into your own applications, web sites, or custom workflows using serverless technologies like Azure Logic Apps or Azure Functions.

Face Identify and Celebrity Recognition features of Azure AI Video Indexer are Limited Access features that require registration.

Find the registration form and more information about Limited Access in this article.

Also, see the following: the announcement blog post and investment and safeguard for facial recognition blog post.

No, Azure AI Video Indexer provides the integration of multiple machine learning models into one pipeline. Indexing a video or audio file via Azure AI Video Indexer retrieves a full set of insights extracted on one shared timeline without any machine learning skills or knowledge on algorithms needed on the customer's part.

Azure AI Video Indexer supports most common media formats. See Azure AI Video Indexer support matrix and service limits.

In the Azure AI Video Indexer web-based portal, you can upload a media file using the file upload dialog or by pointing to a URL that directly hosts the source file. Any URL that hosts the media content using an iFrame or embed code will not work (see example).

Yes. To access the raw video file from video indexer you can use the Get Video Source File Download Url method to get the MP4 file URL and put it in the video tag.

In Azure AI Video Indexer you can create a project and add multiple files to stitch and render as a new file. The number of source files is set to 10 in the website and 100 in the API.

The amount of time it takes to index a video or audio file, both using the Azure AI Video Indexer API and the Azure AI Video Indexer web-based interface, depends on multiple parameters such as the file length and quality, the number of insights found in the file, among other things. We recommend that you run a few test files with your own content and take an average to get a better idea.

Yes, you can integrate Azure AI Video Indexer into serverless technologies like Logic Apps, Flow, and Azure Functions. You can find more details on the Logic App and Flow connectors for Azure AI Video Indexer here. You can see some automation projects done by partners in the Azure AI Video Indexer Samples repo.

You can see which Azure regions Azure AI Video Indexer is available on the regions page.

Yes. In Azure AI Video Indexer you can customize some of the available models to better fit your needs.

For example, our Person model supports out-of-the-box 1,000,000 faces of celebrity recognition, but you can also train it to recognize other faces which are not in that database.

For details, see articles about customizing Person, Brands, and Language models.

Yes. Press the edit video button from the library display or the open in editor button from the player display to get to the Projects tab. You can create a new project and add more videos from your library to edit them together, once you are done you can render your video and download.

If you want to get insights on your new video, index it with Azure AI Video Indexer and it will appear in your library with its insights.

If there are multiple audio streams, Azure AI Video Indexer takes the first one it encounters and will process only this stream. In any audio stream Azure AI Video Indexer processes, it takes the different channels (if present) and processes them together as mono. For streams/channels manipulation you can use ffmpeg commands on the file before indexing it.

For detailed description on how to switch to the correct domain, see Switch between tenants.

An application running in Arc-enabled Kubernetes containers that can be deployed outside of Azure to extract and consume audio insights from your media files. Future versions will provide video insights as well.

Azure Arc is a bridge that extends the Azure platform to help you build applications and services with the flexibility to run across datacenters, at the edge, and in multi-cloud environments. Develop cloud-native applications with consistent development, operations, and security models.

To participate in the preview, you need an active Azure subscription, a connection to Azure, an environment that is capable of running the extension (see [requirements in the quickstart, and permission to join the preview. You can submit a request to join the preview here.

You can either follow the quickstart and experience Azure Arc-enabled Kubernetes automated scenarios with Azure Arc Jumpstart.

During the installation of the extension, machine learning containers are downloaded to the edge compute. Some of these containers can be several gigabytes in size, and the download time will depend on network conditions. In some cases, the download may take longer than 15 minutes.

Video Indexer enabled by Arc requires a storage class configured with readwritemany. If a storage class is already set as the cluster’s default during installation, then it will be used. Otherwise, if you need to configure different storage, you can pass it as a property during the extension installation.

Yes. The Video Indexer extension’s API is available via the frontend extension. You can restrict access to either an IP address or a FQDN. Also, as all communication is done over secure traffic protocols, using a valid CA certificate, and setting a security layer such as NSG or firewall to limit the access to this endpoint is recommended.

Consult the troubleshooting guide.

Communication between a client application and an Azure AI Video Indexer account is encrypted using Transport Layer Security (TLS). TLS is a standard cryptographic protocol that ensures privacy and data integrity between clients and services over the Internet. For more information about TLS, see Transport Layer Security. Video Indexer enforces a minimum of TLS v1.2. Customers can't control the cipher suits, we will pick the first match between the client and server going from strong to weaker.

Yes, unless you delete the file from Azure AI Video Indexer, either using the Azure AI Video Indexer website or API, your video and audio files are stored. For the trial, the video and audio files that you index are stored in the Azure region East US. Otherwise, your video and audio files are stored in the storage account of your Azure subscription.

By default, your data is replicated to the paired region into which you deployed the Azure AI Video Indexer instance. For region with data residency requirements, your data is always kept within the same region. For more related information, check out other Privacy Questions in this section.

Yes, you can always delete your video and audio files as well as any metadata and insights extracted from them by Azure AI Video Indexer. Once you delete a file from Azure AI Video Indexer, the file and its metadata and insights are permanently removed from Azure AI Video Indexer. However, if you have implemented your own backup solution in Azure storage, the file remains in your Azure storage.

Yes, only account admins can invite and uninvite people to their accounts, as well as assign who has editing privileges and who has read-only access.

Your video or audio content that have public as its privacy setting can be accessed by anyone who has the link to your video or audio content and its insights. Your video or audio content that have private as its privacy setting can only be accessed by users that were invited to the account of the video or audio content. The privacy setting of your content also applies to the metadata and insights that Azure AI Video Indexer extracts. You assign the privacy setting when you upload your video or audio file. You can also change the privacy setting after indexing.

Per the Azure Online Services Terms (OST), you completely own your content, and Microsoft will only access your content and the metadata and insights that Azure AI Video Indexer extracts from your content according to the OST and the Microsoft Privacy Statement.

No, the custom models that you create in your account are not available to any other account. Azure AI Video Indexer currently allows you to build custom brands, language, and person models in your account. These models are only available in the account in which you created the models.

Yes, the content and its insights are kept within the Azure region (except for Singapore and Brazil South regions) unless you have a manual configuration in your Azure subscription that uses multiple Azure regions.

Customer data in a region is replicated for BCDR reasons to the paired region.

Azure AI Video Indexer is covered by the Microsoft Privacy Statement. The privacy statement explains the personal data Microsoft processes, how Microsoft processes it, and for what purposes Microsoft processes it. To learn more about privacy, visit the Microsoft Trust Center.

Azure AI Video Indexer currently has the SOC certification. To review Azure AI Video Indexer's certification, please refer to the Microsoft Trust Center.

When videos are uploaded to Azure AI Video Indexer, you can choose from two privacy settings: private and public. Public videos are accessible for anyone, including anonymous and unidentified users. Private ones are restricted solely to the account members.

When uploading a video to Azure AI Video Indexer, an automatic content analysis is done by algorithms and models in order to make sure no inappropriate content will be presented publicly. If a video is found to be suspicious as containing explicit content, it will not be possible to set it as public. However, the account members can still access it as a private video (view it, download the insights and extracted artifacts, and perform other operations available to account members).

In order to set the video for public access, you can either:

• Build your own interface layer (such as app or website) and use it to interact with the Azure AI Video Indexer service. This way the video remains private in our website and your users can interact with it through your interface. For example, you can still get the insights or allow viewing of the video in your own interface.

• Request a human review of the content, which would result in removing of the restriction assuming the content is not explicit.

  This option can be explored if the Azure AI Video Indexer website is used directly by your users as the interface layer, and for public (unauthenticated) viewing.

Azure AI Video Indexer's APIs allows for indexing, extracting metadata, asset management, translation, embedding, customization of models and more. To find more detailed information on using the Azure AI Video Indexer API, refer to the Azure AI Video Indexer developer portal.

There are currently no client SDKs offered. The Azure AI Video Indexer team is working on the SDKs and plans to deliver them soon.

Follow Tutorial: get started with the Azure AI Video Indexer API.

The Azure AI Video Indexer API contains an Authorization API and an Operations API. The Authorizations API contains calls that give you access token. Each call to the Operations API should be associated with an access token, matching the authorization scope of the call.

Access tokens are needed to use the Azure AI Video Indexer APIs for security purposes. This ensures that any calls are coming from you or those who have access permissions to your account.

• Account level – account level access tokens let you perform operations on the account level or the video level. For example, upload a video, list all videos, get video insights.
• User level - user level access tokens let you perform operations on the user level. For example, get associated accounts.
• Video level – video level access tokens let you perform operations on a specific video. For example, get video insights, download captions, get widgets, etc.

Access tokens expire every hour, so you need to generate a new access token every hour.

See a release note regarding login information.

Once you register your email account using an identity provider, you cannot use this email account with another identity provider.

They are stored by the Azure AI Video Indexer service and can be fetched with the Get Video Index and Get Video Artifact Download Url APIs.

There are no additional charges from Azure AI Video Indexer for calling the API.

120 requests per minute are standard but can go up to 600 requests per minute.

Azure AI Video Indexer uses a simple pay-as-you-go pricing model based on the duration of the content input that you index. Additional charges may apply for encoding, streaming, storage, network usage, and media reserved units. For more information, see the pricing page.

When sending a video to be indexed, the user will define the indexing to be video analysis, audio analysis or both. This will determine which SKUs will be charged. If there is a critical level error during processing, an error code will be returned as a response. In such a case, no billing occurs. A critical error can be caused by a bug in our code or a critical failure in an internal dependency the service has. Errors such as wrong identification or insight extraction are not considered as critical and a response is returned. In any case where a valid (non-error code) response is returned, billing occurs.

Yes, Azure AI Video Indexer offers a trial account that that provides you with 2,400 free indexing minutes when using the Azure AI Video Indexer website or the Azure AI Video Indexer API (see developer portal).

Send a detailed explanation of your request to "Video Indexer Customer Support" [email protected]. Including: AI model you are referring to, the result you'd like to opt out, the reason we should opt out the results and an explanation of why you are asking this.

Processing the request might take few weeks.