Mastering ChatGPT: From Zero to Hero (3/3)
As machines push their way into the domain of creative work, it is obvious that the days of the human workforce are numbered. This prediction has led to fear and panic in most of the industry. This series of blogs will help you build your skills — not just to retain your job — but to help you rule the new era of Generative LLMs.
Topics covered
This blog starts with a detailed theoretical introduction, then jumps into practical implementation and code examples. We will cover the following topics in the blog series.
- Introduction to ChatGPT, LLM, and Prompt Engineering
- Using Open AI API in your apps. Host your own Chatbot on AWS
- Host your own LLM on AWS, Amazon Bedrock
I am sure you are excited to continue the journey to the next step.
Hosting an LLM on AWS
ChatGPT and OpenAI are great. However, we don’t have to depend on them. We can deploy our own deployment LLM on AWS. It is not as difficult as it sounds. In fact, it is quite simple when we work with Sagemaker Studio.
Hugging face is a large repository of open-source models that we can use in our applications. Their license terms are a bit stringent. So make sure you understand it well before making a commercial product out of them.
Once the infrastructure is setup, the code is quite simple:
from transformers import T5ForConditionalGeneration, AutoTokenizer
import torch
import os
def model_fn(model_dir):
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl",
load_in_8bit=True, device_map="auto", cache_dir="/tmp/model_cache/")
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-xxl")
return model, tokenizer
def predict_fn(data, model_and_tokenizer):
model, tokenizer = model_and_tokenizer
text = data.pop("inputs", data)
inputs = tokenizer(text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(inputs, **data)
return tokenizer.decode(outputs[0], skip_special_tokens=True)We can deploy the inference script in a single command
predictor = huggingface_model.deploy(
initial_instance_count=1,
instance_type="ml.g5.4xlarge",
endpoint_name=endpoint_name,
)This can be exposed easily with Streamlit.
import streamlit as st
import boto3
import json
# Create a low-level client representing Amazon SageMaker Runtime
session = boto3.Session()
sagemaker_runtime = session.client('sagemaker-runtime', region_name=session.region_name)
# The name of the endpoint. The name must be unique within an AWS Region in your AWS account.
endpoint_name='<your_endpoint>'
st.sidebar.title("Flan-T5 Parameters")
stop_word = st.sidebar.text_input("Stop word")
min_length, max_length = st.sidebar.slider("Min/Max length", 0, 200, (30, 100))
temperature = st.sidebar.slider("Temperature", min_value=0.0, max_value=1.0, value=0.3)
rep_penalty = st.sidebar.slider("Repetition Penalty", min_value=0.9, max_value=1.2, value=1.0)
def generate_text(prompt):
do_sample = temperature > 0
payload = {
"inputs": prompt,
"min_length": min_length,
"max_length": max_length,
"temperature": temperature,
"repetition_penalty": rep_penalty,
"do_sample": do_sample,
}
response = sagemaker_runtime.invoke_endpoint(
EndpointName=endpoint_name,
ContentType='application/json',
Body=json.dumps(payload)
)
result = json.loads(response['Body'].read().decode())
return result
st.header("Flan-T5-XXL Playground")
prompt = st.text_area("Enter your prompt here:")
if st.button("Run"):
generated_text = generate_text(prompt)
if len(stop_word) > 0:
generated_text = generated_text[:generated_text.rfind(stop_word)]
st.write(generated_text)For more details, refer to this blog by Heiko Hotz.
Despite its simplicity, some of us are averse to sagemaker. It seems to be clumsy. Don’t worry! AWS has you covered. Check out this blog to see how we can quickly build a custom LLM based application using Amazon Kendra and Langchain.
Amazon Bedrock
This is the upcoming big bang in the world of AI. We are all eagerly waiting for the day when we will get access to the amazing service on AWS. I asked ChatGPT to write something on the topic. But I got a disappointing response.
As of my last training data in September 2021, there is no specific reference to “Amazon Bedrock” in relation to the company Amazon or the Amazon Rainforest. “Bedrock” typically refers to the solid rock layer under loose material like soil or other surface material, and Amazon is a well-known e-commerce company and the name of the world’s largest rainforest.
Hmm. No worries! Looks like we will have to dig up ourselves. Well, I don’t give up so easily. After some effort, I could get ChatGPT to write this for me. Try to do that for yourselves. And write your experience in the comments.
Amazon Bedrock is a comprehensive managed service offering access to numerous Function Modules (FMs) from leading AI startups and Amazon, all readily available via an API. This broad assortment of FMs ensures you can select the ideal model suited for your specific use case. Bedrock’s serverless architecture allows rapid initialization and private customization of FMs with your data, significantly reducing your time to deployment. Integration into your applications is seamless, utilizing AWS tools and services that you’re already accustomed to. This includes compatibility with Amazon SageMaker ML features such as Experiments for model testing and Pipelines for scalable FM management, relieving you from the burden of infrastructure management.
This is the big bang we are impatiently waiting for.
Ethical AI
With great power comes great responsibility. When we’re talking about this cool AI stuff, we can’t forget that we’ve got to play it fair and safe. Being ethical with AI means being clear about what it does, making sure it’s fair to everyone, and standing up and taking responsibility if things go wrong.
Legally, we’ve got to be careful not to step on toes with things like privacy, copyright, and data protection laws. AI is like a super powerful tool, but it’s up to us to use it right. We’re heading towards an AI-filled future, so let’s make sure it’s a future that respects everyone and their rights.
