Language and Vision APIs in Practice

The Computer Vision API (/vision/v3.2/analyze) analyses an image URL or binary stream and returns a rich JSON response containing detected objects, a scene description in natural language, dominant colours, and whether the image contains adult content. You specify which visual features you want — Categories, Description, Objects, Tags, Faces, Color, Adult — to control what the API computes and pay only for what you use.

The Read API (/vision/v3.2/read/analyze) is the recommended OCR endpoint for extracting text from images and PDFs. Unlike the older /ocr endpoint, the Read API is asynchronous — you submit the image, get back an operation URL, poll it until complete, and then retrieve the result. It supports printed and handwritten text in 164 languages, preserves the reading order of text blocks, and handles rotated or low-resolution images better than the synchronous OCR endpoint.

The Sentiment Analysis endpoint (/text/analytics/v3.1/sentiment) returns a sentiment label (positive, negative, neutral, or mixed) and a confidence score between 0 and 1 for each document submitted. It also provides opinion mining (aspect-based sentiment) that identifies which specific aspects of a product or service a reviewer is commenting on and the sentiment towards each aspect — for example, detecting that a restaurant review is positive about food but negative about service.

Key phrase extraction identifies the main topics and concepts in a body of text, returning them as a list of phrases. For a customer support email, it might return phrases like delivery delay, order confirmation, and refund request. This is useful for automatically tagging support tickets, summarising documents for search indexing, or prioritising content in a knowledge base. The API processes up to 1,000 documents per request and supports 10+ languages.

Named Entity Recognition (NER) identifies and categorises mentions of entities in text — such as Person, Organisation, Location, DateTime, Quantity, URL, and Email. The PII detection variant (Personal Identifiable Information) specifically finds and can redact sensitive data like credit card numbers, social security numbers, and phone numbers. PII detection is useful for automatically anonymising data before storing or processing it for analytics.

The Translator API (/translate) converts text between 100+ languages in a single REST call. You pass the source text, specify one or more target language codes, and receive the translated text with detected source language. The API also provides transliteration (converting script, e.g. Arabic to Latin characters) and dictionary lookup for word-level translation alternatives. Translator supports batching up to 100 documents per request, making it efficient for large-scale content localisation.

Azure AI Custom Vision allows you to train an image classification or object detection model using your own labelled images without writing ML code. In the Custom Vision portal, you upload images, tag them, and click Train. The service trains a model and returns performance metrics (Precision, Recall, AP). You then test the model with new images and publish it as a prediction endpoint. Models can also be exported as ONNX or CoreML for on-device inference.

Conversational Language Understanding (CLU) is the successor to LUIS. You define intents (what the user wants, e.g. BookFlight, GetWeather) and entities (data to extract, e.g. destination city, travel date), provide example utterances for each intent, train the model, and deploy it. Applications call the CLU prediction endpoint to classify user input into the correct intent and extract entities. CLU powers the understanding layer of virtual assistants and chatbots.

Azure AI Services APIs return standard HTTP status codes. 400 Bad Request means malformed input (e.g. document too long — Text Analytics supports up to 5,120 characters per document). 401 Unauthorized means an invalid or missing API key. 429 Too Many Requests means you have exceeded your service quota — implement exponential backoff with jitter and retry. For production, consider a dedicated resource (not the free tier) and request quota increases via the Azure portal if you need higher throughput.

Azure AI Services can be integrated with Azure Logic Apps using built-in connectors, enabling no-code AI workflows. For example: when a new email arrives → extract attachments → call the Computer Vision Read API to OCR text from attachments → call Text Analytics to extract key phrases → write results to a SharePoint list. Logic Apps includes connectors for Cognitive Services (Language, Vision, Translator) that manage authentication and API calls without writing REST request code.

Microsoft publishes official Python packages for each AI Service. Install azure-ai-textanalytics for Language APIs or azure-cognitiveservices-vision-computervision for Vision. The SDK handles authentication, serialisation, and retry logic automatically. Authenticate using an AzureKeyCredential or a DefaultAzureCredential (managed identity aware). Using the SDK is preferred over raw HTTP requests in production code as it provides type safety and easier upgrades when API versions change.

Test your understanding of Microsoft Azure Fundamentals (AZ-900) concepts from this lesson.

In this lesson you learned: the Computer Vision API analyses images for objects, descriptions, and OCR text (using the async Read API), Text Analytics APIs extract sentiment, key phrases, and PII from text, and Translator API converts text between 100+ languages in a single call. Next up we explore Azure Machine Learning Studio for training and deploying custom ML models.