How to Use Azure Vision in Foundry Tools for Image Analysis and OCR?

Abhinava Maddha 105 Reputation points Microsoft Employee
2026-04-28T21:07:48.87+00:00

Question: I’m exploring Azure Vision in Foundry Tools and want to understand how to effectively use it for analyzing images and extracting insights.

Details:

  • Working within Azure AI Foundry
  • Interested in capabilities like image analysis, OCR, and object detection
  • No prior machine learning experience

Questions:

  1. What core features does Azure Vision provide in Foundry Tools (e.g., image tagging, OCR, spatial analysis)?
  2. How can I integrate Azure Vision into my application without building custom ML models?
  3. What are the typical use cases for Azure Vision, such as text extraction or real-time video analysis?
  4. Are there any limitations or prerequisites (formats, size, region support) I should be aware of?
Azure AI Custom Vision
Azure AI Custom Vision

An Azure artificial intelligence service and end-to-end platform for applying computer vision to specific domains.

0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. SRILAKSHMI C 19,725 Reputation points Microsoft External Staff Moderator
    2026-04-29T09:59:52.67+00:00

    Hello Abhinava Maddha,

    Thank you for Reaching out to Microsoft Q&A.

    Azure AI Vision provides a rich set of prebuilt computer vision capabilities that allow you to analyze images, extract text, and derive meaningful insights without requiring any prior machine learning experience or the need to build custom models. It is designed for easy integration into applications through Azure AI Foundry, REST APIs, and SDKs.

    Core Capabilities of Azure AI Vision

    Image Analysis

    Azure AI Vision can automatically analyze images and provide:

    • Image tagging – generates descriptive keywords for image content
    • Object detection – identifies and locates common objects and people
    • Image captions – creates natural-language descriptions of entire images
    • Dense captions – generates detailed descriptions for specific regions within an image
    • People detection – detects and localizes people in an image
    • Brand and logo detection
    • Smart cropping – creates content-aware thumbnails
    • Color analysis and image metadata extraction

    Optical Character Recognition (OCR)

    Azure AI Vision offers robust OCR capabilities for extracting text from images and documents:

    • Extract printed and handwritten text
    • Support for multiple languages
    • Preserve text structure, including lines, words, and bounding boxes

    Two OCR options are available:

    • Read API – ideal for text in natural images such as signs, labels, posters, and screenshots
    • Document Intelligence Read Model – best suited for multi-page documents, forms, invoices, and scanned files

    Spatial Analysis and Video Understanding

    For real-time scenarios, Azure AI Vision can provide:

    • People counting
    • Occupancy monitoring
    • Zone entry and exit detection
    • Foot traffic analysis
    • Space utilization insights

    These capabilities are especially useful for retail analytics, workplace monitoring, and smart building solutions.

    Optional Face Capabilities

    Azure AI Vision also supports:

    • Face detection
    • Facial attribute analysis (such as accessories or expressions)

    Please note that face identification features are subject to Responsible AI access requirements.

    How to Integrate Azure AI Vision

    You can use Azure AI Vision without training any machine learning models.

    Integration Options

    • Azure AI Foundry tools and Prompt Flows
    • REST APIs
    • Azure SDKs for Python, C#, JavaScript, and Java
    • Foundry pipelines and agent workflows

    Typical Workflow

    1. Create or open an Azure AI Foundry project.
    2. Connect an Azure AI Vision resource.
    3. Select the required capability (Image Analysis, OCR, or Spatial Analysis).
    4. Submit an image via file upload, URL, or base64-encoded content.
    5. Receive structured JSON output containing tags, captions, detected objects, extracted text, bounding boxes, and confidence scores.

    All models are prebuilt and fully managed by Microsoft.

    Common Use Cases

    Document and Text Extraction

    • Invoice and receipt processing
    • Form digitization
    • Document indexing and search

    Image Understanding

    • Digital asset management
    • Automated image tagging
    • Accessibility captions
    • Brand and logo detection

    Object Detection

    • Inventory monitoring
    • Retail shelf analysis
    • Manufacturing quality inspection
    • Security and surveillance

    Real-Time Analytics

    • Queue monitoring
    • Occupancy tracking
    • Customer footfall analysis
    • Smart building and workplace insights

    Content Optimization

    • Automated thumbnail generation
    • Smart image cropping for web and mobile applications

    Supported Formats and Limits

    Supported Image Formats

    • JPEG
    • PNG
    • BMP
    • GIF
    • TIFF
    • WEBP
    • ICO
    • MPO

    Size and Dimension Limits

    Image Analysis API:

    • Up to 20 MB
    • Supported dimensions: 50 × 50 to 16,000 × 16,000 pixels

    OCR / Read API:

    • Supports images up to 10,000 × 10,000 pixels

    For best results, use high-quality, properly oriented images.

    Regional Availability and Prerequisites

    • Feature availability may vary by Azure region.
    • Some advanced capabilities, such as Spatial Analysis, may only be available in select regions or may require container deployment.
    • Real-time video analysis scenarios may require additional compute resources.

    Authentication Options

    • API key
    • Microsoft Entra ID
    • Managed Identity

    Key Benefits

    • No machine learning expertise required
    • No data labeling or model training needed
    • Pretrained, enterprise-ready models
    • Easy integration through APIs and SDKs
    • Scalable and secure
    • Seamless integration with Azure AI Foundry and Azure OpenAI

    If you are new to Azure AI Vision, we recommend starting with:

    1. Image Analysis for tags, captions, and object detection
    2. Read OCR API for text extraction
    3. Testing with sample images in Azure AI Foundry
    4. Integrating into your application using the Azure SDK or REST API

    Azure AI Vision in Azure AI Foundry enables you to:

    • Analyze image content
    • Detect objects and people
    • Generate captions
    • Extract printed and handwritten text
    • Perform spatial and video analytics
    • Build intelligent, vision-powered applications

    All of this can be accomplished using Microsoft's prebuilt AI models, without any need for custom machine learning development.

    Please refer this

    Overview of Image Analysis 4.0 API (features, limits) https://learn.microsoft.com/azure/ai-services/computer-vision/overview-image-analysis?tabs=4-0

    Quickstart: Image Analysis 4.0 with SDKs https://learn.microsoft.com/azure/ai-services/computer-vision/quickstarts-sdk/image-analysis-client-library-40

    OCR Overview & Quickstart https://learn.microsoft.com/azure/ai-services/computer-vision/overview-ocr https://learn.microsoft.com/azure/ai-services/computer-vision/quickstarts-sdk/client-library

    What is Azure Vision in Foundry Tools? (service summary & image limits) https://learn.microsoft.com/azure/ai-services/computer-vision/overview?wt.mc_id=knowledgesearch_inproduct_azure-cxp-community-insider

    Tutorial: Azure Vision in Foundry Tools with SynapseML https://learn.microsoft.com/azure/synapse-analytics/machine-learning/tutorial-computer-vision-use-mmlspark?wt.mc_id=knowledgesearch_inproduct_azure-cxp-community-insider

    I Hope this helps. Do let me know if you have any further queries.

    If this answers your query, please do click Accept Answer and Yes for was this answer helpful.

    Thank you!

    Was this answer helpful?

  2. AI answer

    2026-04-28T21:08:06.73+00:00

    Azure Vision in Foundry Tools provides ready-made computer vision capabilities that can be used directly from applications without building or training custom machine learning models.

    1. Core features in Azure Vision (Foundry Tools)

    Azure Vision includes several major capabilities:

    • Image Analysis
      • Extracts visual features from images such as:
        • Objects
        • Faces
        • Adult/racy content
        • Auto-generated tags
        • Auto-generated natural-language descriptions
      • The Image Analysis 4.0 API is the latest version and adds:
        • Synchronous OCR (Read OCR) for images
        • People detection
      • Can be accessed via SDKs or the REST Analyze Image API.
    • OCR (Optical Character Recognition)
      • Extracts printed and handwritten text from images, including:
        • Photos of street signs, products, posters, screenshots
        • Text-heavy images and multi-page PDFs (via the Document Intelligence Read OCR model for documents)
      • Azure Vision OCR in Image Analysis 4.0:
        • Fast, synchronous API optimized for non-document, “in-the-wild” images
        • Supports printed and handwritten text in many languages using script-based models
        • Returns text plus structure via the readResult section in the Analyze Image response.
    • Face
      • Detects, recognizes, and analyzes human faces in images.
      • Supports scenarios such as identification, touchless access control, and face blurring.
    • Thumbnail generation
      • Analyzes an image to find the area of interest and generates a high-quality thumbnail, optionally with a different aspect ratio than the original.
    1. Integrating Azure Vision without custom ML models

    Azure Vision is exposed as managed APIs and SDKs, so integration does not require ML expertise:

    • Use client SDKs or REST APIs
      • Image Analysis 4.0 can be called via:
        • Client library SDKs (for example, .NET, Python, etc.)
        • Direct REST calls using the Analyze Image API (https://aka.ms/vision-4-0-ref).
      • For OCR with Image Analysis 4.0:
        • Call the Analyze Image API and include Read in the features query parameter.
        • Parse the readResult section of the JSON response for extracted text.
    • Use SynapseML / Synapse Analytics (no ML model building)
      • In Azure Synapse Analytics with SynapseML, Vision can be used as Spark transformers:
        • AnalyzeImage – extract tags, objects, faces, adult content, descriptions.
        • OCR / RecognizeText – read text from images.
        • DescribeImage – generate human-readable descriptions.
        • ReadImage – used in the tutorial to run OCR and return results in a Spark DataFrame.
      • Example (OCR in SynapseML): 
            df = spark.createDataFrame([
        ("<replace with your file path>/ocr.jpg", )
    ], ["url", ])
    
    ri = (ReadImage()
        .setLinkedService(ai_service_name)
        .setImageUrlCol("url")
        .setOutputCol("ocr"))
    
    display(ri.transform(df))
      • This pattern lets applications use Vision from data pipelines without training or deploying models.
    1. Typical use cases
    • Text extraction (OCR)
      • Extract printed and handwritten text from:
        • Product labels, street signs, posters, screenshots
        • Business documents, invoices, receipts, reports, letters, whiteboards (for text-heavy documents, use Document Intelligence Read OCR model).
      • Power scenarios like:
        • Knowledge mining and search
        • Business process automation (invoice processing, form intake)
        • Accessibility (screen-reading of images)
    • Image analysis and tagging
      • Automatically tag images with objects and concepts.
      • Detect faces and adult/racy content.
      • Generate natural-language descriptions of images.
      • Example from SynapseML tutorial: analyzing a dog image returns tags like "dog", "outdoor", "fence", "bench".
    • Thumbnail generation and content-aware cropping
      • Generate thumbnails that focus on the most important region (e.g., a person’s face) while fitting a required aspect ratio.
    • Face-based scenarios
      • Detect, recognize, and analyze human faces for:
        • Identification and verification
        • Touchless access control
        • Privacy-preserving face blurring.
    • Domain-specific content detection
      • Detect and identify domain-specific content in images (as demonstrated in the SynapseML tutorial).

    Note: Real-time video analysis is not a primary Azure Vision scenario in this context. For advanced video analysis (transcription, translation, summarization), Video Indexer is recommended instead.

    1. Limitations and prerequisites

    From the provided context, the key considerations are:

    • Service scope and when not to use Azure Vision
      • Use Azure Vision for:
        • Extracting printed and handwritten text from images and documents via OCR.
        • Analyzing images for objects, faces, and descriptions.
        • Detecting and analyzing human faces.
      • Do not use Azure Vision for:
        • Advanced video analysis (use Video Indexer).
        • Content safety moderation (use Content Safety in Foundry Control Plane).
        • Tasks already covered by large multimodal foundation models like GPT‑4o.
    • API behavior and performance
      • Image Analysis 4.0 OCR:
        • Synchronous, optimized for non-document images and near real-time experiences.
      • Document Intelligence Read OCR:
        • Asynchronous, optimized for text-heavy PDFs and scanned documents.
        • Requires polling via a “Get Read Results” operation to retrieve results.
    • Deprecation notice
      • Image Analysis 4.0 in Azure Vision in Foundry Tools is marked as deprecated and will be retired on September 25, 2028. After that date, calls to this service will fail.
      • Migration to alternatives outlined in the migration guide is recommended.
    • Language support
      • OCR supports several languages for printed text.
      • Handwritten OCR is currently supported exclusively for English (for the legacy OCR described in the transparency note).
    • General prerequisites
      • An Azure AI Foundry / Azure Vision resource and keys.
      • For SynapseML usage:
        • Azure Synapse Analytics workspace.
        • SynapseML library configured.
        • Linked service to the Azure AI service (ai_service_name in examples).

    File formats, size limits, and region-specific availability are not detailed in the provided context; those details are typically documented per API in the service reference and should be checked there when designing production solutions.

    References:

    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.