GenerativeAI Python Libaries

https://www.pexels.com/photo/two-women-looking-at-the-code-at-laptop-1181263/
https://www.pexels.com/photo/two-women-looking-at-the-code-at-laptop-1181263/

After deploying GenerativeAI chat completion models from Azure GenerativeAI catalog, we need to know which Python SDK to used.

OpenAI Models

After deploying Azure OpenAI model, we use openai Python library. Either

pip install openai

or

poetry add openai

and we suggest that you use the asynchronous client.

from openai import AsyncAzureOpenAI

client = AsyncAzureOpenAI(
    api_key=<azure_openai_key>,
    api_version=<azure_openai_api_version>,
    azure_endpoint=<azure_openai_endpoint>,
)

or (even better, use the DefaultAzureCredential)

from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import AsyncAzureOpenAI

azure_credential = DefaultAzureCredential()
token_provider = get_bearer_token_provider(
	azure_credential, "https://cognitiveservices.azure.com/.default"
)
client = AsyncAzureOpenAI(
    api_version=<azure_openai_api_version>,
    azure_endpoint=<azure_openai_endpoint>,
    azure_ad_token_provider=token_provider,
)

then we can use the client's chat.completions.create accordingly (with await).


Other GenerativeAI Models deployed in Azure

Models such as Phi3 (our small and mighty model :-)), Llama (mighty from Meta), etc. We use azure-ai-inference

pip install azure-ai-inference

or

poetry add azure-ai-inference

Similarly, we recommend its asynchronous client

from azure.ai.inference.aio import ChatCompletionsClient
from azure.core.credentials import AzureKeyCredential

client = ChatCompletionsClient(
    endpoint=<azure_phi3_endpoint>,
    credential=AzureKeyCredential(<azure_phi3_key>),
)

or (even better, use the DefaultAzureCredential)

from azure.ai.inference.aio import ChatCompletionsClient
from azure.identity import DefaultAzureCredential

client = ChatCompletionsClient(
    endpoint=<azure_phi3_endpoint>,
    credential=DefaultAzureCredential(),  # type: ignore
)

Two things worth mentioning are

  1. We do not need to fetch bearer token like we did for OpenAI SDK.
  2. There is an issue with the typing declaration of credential parameter, it does take DefaultAzureCredential when it claims that it does not. Hence add a ignore typing checking comment is required.

All the above code can be found in https://github.com/dennisseah/llm-operational-metrics where we collect operational metrics for Phi3 and OpenAI models. This can be extended to collect more metrics and other models.





Comments

Post a Comment

Popular posts from this blog

Happy New Year, 2024 from DALL-E

Image
generated by DALL-E Wishing everyone a happy new year. Year 2023 is great for me because I learned much about OpenAI. I am involved in a live deployment with it, and involved in a handful of interesting user scenarios around it. On top of this, a copilot hackathon. The image above is generated by DALL-E and following is the code. 1. Dependencies python = "^3.10" pydantic-settings = "^2.1.0" openai = "^1.3.8" 2. Environment Parameters OPENAI_AZURE_ENDPOINT="<your azure openai endpoint>" AZURE_OPENAI_API_KEY="<your azure openai secret>" OPENAI_API_VERSION="2023-12-01-preview" OPENAI_DALLE_MODEL="dalle" Note: OPENAI_API_VERSION has to be 2023-12-01-preview otherwise you will get an exception openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} dalle  is the name of my dall-e-3 deployment model. You need to change it to
Read more

Bing Search as tool for OpenAI

Image
  Image from https://www.pexels.com/@skitterphoto/ Many times, we need to search for information in the web when we are working with OpenAI. One of the reasons is that the dataset for training for OpenAI is never up to date. For instance, the OpenAI GPT-4 Turbo has knowledge of events up to April 2023. Here is the Python implementation with Pydantic model. Dependencies python = "^3.10" pydantic-settings = "^2.1.0" pydantic = "^2.5.2" azure-cognitiveservices-search-websearch = "^2.0.0" Environment These are the environment parameters needed. Have the following in a .env file. azure_bing_search_endpoint="https://api.bing.microsoft.com" azure_bing_search_key="<key>" We look under "Keys and Endpoint" for these values. Pydantic Setting Model from pydantic_settings import BaseSettings class BingSearchSettings(BaseSettings): azure_bing_search_endpoint: str azure_bing_search_key: str class Config:
Read more

OpenAI: Functions Feature in 2023-07-01-preview API version

Image
  Image from https://www.pexels.com/@pavel-danilyuk/ One of the cool features of OpenAI version 2023-07-01 is its custom Functions feature. This feature is close to me because I am working on a project where we require OpenAI to generate JSON responses. We always want the generated JSON to be consistent, otherwise, we need to do some post-processing tasks to get the JSON object in the correct schema. In the blog, we observe the completion with and without this new feature. Development Setup python -m venv .venv poetry init poetry shell poetry add openai poetry add python-dotenv poetry add pydantic I am using Python version 3.11 Create .env file OPENAI_API_TYPE="azure" OPENAI_API_BASE="<OpenAI endpoint>" OPENAI_API_KEY="<OpenAI secret>" OPENAI_API_VERSION="2023-07-01-preview" OPENAI_DEPLOYMENT_ID="<deployment identifier>" Without OpenAI functions feature We use two sample texts for experimentation. We want to get the na
Read more