Azure Cognitive Services Overview

Azure AI Services (formerly Azure Cognitive Services) is a family of pre-built AI APIs that let developers add intelligent features to applications without training machine learning models from scratch. The portfolio covers four domains: Vision (analyse images and video), Language (understand and generate text), Speech (convert between audio and text), and Decision (personalise content and detect anomalies). All are accessible via REST APIs, SDKs, or the Azure portal.

You can provision AI Services in two ways. A multi-service resource (called Azure AI Services in the portal) provides access to most APIs through a single endpoint and API key — convenient for development and experimentation. A single-service resource (e.g. Computer Vision or Text Analytics) gives access to only one API with dedicated billing and quota limits per service. For production, single-service resources are often preferred to control spend and apply service-specific security policies.

The Azure AI Vision category includes several APIs. Computer Vision analyses images for objects, scene descriptions, text (OCR), and adult content. Custom Vision lets you train your own image classification or object detection model with just a few dozen labelled images. Face API detects faces and emotions (with responsible-use access controls). Azure AI Video Indexer extracts insights from video files including speaker recognition, transcription, and scene detection.

The Azure AI Language category provides text analysis capabilities. Text Analytics offers sentiment analysis, key phrase extraction, named entity recognition (NER), and language detection. Azure AI Translator translates text between 100+ languages with a single API call. Azure AI Language Understanding (CLU) classifies user intents from natural language for building conversational applications. Azure AI Question Answering builds a knowledge base from FAQs and documents to answer natural language questions.

The Azure AI Speech category bridges audio and text. Speech to Text (STT) transcribes spoken audio in real time or batch mode, supporting 100+ languages and custom acoustic models for domain-specific terminology. Text to Speech (TTS) converts text to natural-sounding audio using neural voices in 140+ languages. Speaker Recognition identifies or verifies individuals by their voice. Speech Translation combines STT with Translator to provide real-time speech-to-speech translation.

The Decision category includes services for personalisation and anomaly detection. Azure AI Anomaly Detector analyses time-series data (e.g. server metrics, sales figures) and automatically identifies anomalies without requiring a labelled training dataset — it learns normal patterns from historical data. Azure AI Personalizer (now part of Content Safety in some views) uses reinforcement learning to select the best content item to show a user based on contextual features. Both are useful for operational and business intelligence scenarios.

All Azure AI Services APIs follow the same pattern: obtain the endpoint URL and an API key (or use managed identity) from your resource, then send an HTTP POST request with the payload and authentication header. The API returns a JSON response with the results. Microsoft provides client SDKs for .NET, Python, JavaScript, Java, and Go that wrap the REST calls with typed objects and retry logic, making integration straightforward without manual HTTP handling.

Microsoft builds Azure AI Services around six Responsible AI principles: Fairness (AI should not discriminate), Reliability and Safety (AI should perform as expected), Privacy and Security (data should be protected), Inclusiveness (AI should empower everyone), Transparency (AI should be understandable), and Accountability (humans should be responsible for AI decisions). Some capabilities like Face API's identification features require a formal application to Microsoft before access is granted.

Azure AI Services are priced on a pay-per-use model: you pay per API call, per image analysed, per character translated, or per audio minute transcribed. Each service and tier has its own pricing unit. A free tier (F0) exists for most services with limited call quotas (e.g. 5,000 calls/month for Text Analytics), making experimentation cost-free. The Standard tier (S0) has higher quotas and is suitable for production. Costs scale linearly with usage — there are no fixed monthly commitments.

AI Services resources should be secured beyond the default API key authentication. Best practices include: storing API keys in Azure Key Vault rather than in code, using managed identity to access the resource without any key, restricting network access using private endpoints so the resource is only reachable from your VNet, and enabling diagnostic logging to audit all API calls for compliance. API keys should be rotated regularly — each resource has two keys to enable zero-downtime rotation.

Many Azure AI Services can be deployed as Docker containers for scenarios requiring data residency (processing stays on-premises or in a specific region) or disconnected environments. You pull the container image from the Microsoft Container Registry, run it with your API key and endpoint configuration, and call the service's REST API at localhost. The container phones home to Azure for billing — all usage is still tracked and charged through your Azure subscription, but the AI processing occurs locally.

Test your understanding of Microsoft Azure Fundamentals (AZ-900) concepts from this lesson.

In this lesson you learned: Azure AI Services provides pre-built AI capabilities across Vision, Language, Speech, and Decision domains accessible via REST APIs, a multi-service resource simplifies access with a single endpoint and key, and all services should be secured with Key Vault and managed identity and can be deployed as containers for data residency requirements. Next up we explore calling the Language and Vision APIs in practice.