langchain-模型和prompt快速筛选平台
大模型开发/技术交流
- LLM
- Prompt
2023.11.142541看过
构建您的语言模型应用程序可能需要在多种提示、模型甚至Chain（这是Langchain的概念）之间进行选择。在这个过程中，您需要以简单、灵活且直观的方式比较不同选项在不同输入上的表现。
LangChain提供了一个名为ModelLaboratory的概念，用于测试并尝试不同的模型。
实验环境是colab，但是还没有成功，模型加载问题，贴出来先看看
安装依赖
!pip install cohere!pip install huggingface_hub
配置访问key
cohere 的api_key 和 huggingface_hub的access_token,是两个托管模型的地方
%env COHERE_API_KEY=FdHkBHjF3B7ynaCbCs0fxxx%env HUGGINGFACEHUB_API_TOKEN=hf_MADVhnPvDxxxxTVuYjKbjAxxx
模型效果比较
from langchain import LLMChain, OpenAI, Cohere, HuggingFaceHub, PromptTemplatefrom langchain.model_laboratory import ModelLaboratoryllms = [OpenAI(temperature=0),Cohere(model="command-xlarge-20221108", max_tokens=20, temperature=0),HuggingFaceHub(repo_id="google/flan-t5-xl", model_kwargs={"temperature":1})]model_lab = ModelLaboratory.from_llms(llms)model_lab.compare("What color is a flamingo?")
输出
Input:What color is a flamingo?OpenAIParams: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}Flamingos are pink.CohereParams: {'model': 'command-xlarge-20221108', 'max_tokens': 20, 'temperature': 0.0, 'k': 0, 'p': 1, 'frequency_penalty': 0, 'presence_penalty': 0}PinkHuggingFaceHubParams: {'repo_id': 'google/flan-t5-xl', 'temperature': 1}pink
prompt效果比较
prompt = PromptTemplate(template="What is the capital of {state}?", input_variables=["state"])model_lab_with_prompt = ModelLaboratory.from_llms(llms, prompt=prompt)model_lab_with_prompt.compare("New York")
输出
Input:New YorkOpenAIParams: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}The capital of New York is Albany.CohereParams: {'model': 'command-xlarge-20221108', 'max_tokens': 20, 'temperature': 0.0, 'k': 0, 'p': 1, 'frequency_penalty': 0, 'presence_penalty': 0}The capital of New York is Albany.HuggingFaceHubParams: {'repo_id': 'google/flan-t5-xl', 'temperature': 1}st john s
prompt 比较
from langchain import SelfAskWithSearchChain, SerpAPIWrapperopen_ai_llm = OpenAI(temperature=0)search = SerpAPIWrapper()self_ask_with_search_openai = SelfAskWithSearchChain(llm=open_ai_llm, search_chain=search, verbose=True)cohere_llm = Cohere(temperature=0, model="command-xlarge-20221108")search = SerpAPIWrapper()self_ask_with_search_cohere = SelfAskWithSearchChain(llm=cohere_llm, search_chain=search, verbose=True)chains = [self_ask_with_search_openai, self_ask_with_search_cohere]names = [str(open_ai_llm), str(cohere_llm)]model_lab = ModelLaboratory(chains, names=names)model_lab.compare("What is the hometown of the reigning men's U.S. Open champion?")
输出
Input:What is the hometown of the reigning men's U.S. Open champion?OpenAIParams: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}> Entering new chain...What is the hometown of the reigning men's U.S. Open champion?Are follow up questions needed here: Yes.Follow up: Who is the reigning men's U.S. Open champion?Intermediate answer: Carlos Alcaraz.Follow up: Where is Carlos Alcaraz from?Intermediate answer: El Palmar, Spain.So the final answer is: El Palmar, Spain> Finished chain.So the final answer is: El Palmar, SpainCohereParams: {'model': 'command-xlarge-20221108', 'max_tokens': 256, 'temperature': 0.0, 'k': 0, 'p': 1, 'frequency_penalty': 0, 'presence_penalty': 0}> Entering new chain...What is the hometown of the reigning men's U.S. Open champion?Are follow up questions needed here: Yes.Follow up: Who is the reigning men's U.S. Open champion?Intermediate answer: Carlos Alcaraz.So the final answer is:Carlos Alcaraz> Finished chain.So the final answer is:Carlos Alcaraz
本文为稀土掘金博主「x2lab」的原创文章如有侵权，请联系千帆社区进行删除
评论
