Ryan Lingo

AI Builders' Favorite Tools: A Hive Mind Survey Summary I recently asked AI builders about their favorite go-to libraries and frameworks for AI projects. Here is a summary of the first ~63 replies, categorized. Agent Frameworks: • AgentOpsAI: This tool simplifies the debugging and monitoring of agent operations, making it easier to read through prompts and visualize spend. (@AlexReibman) • AgentForge: Designed for rapid iteration of cognitive architectures with support for multiple LLM APIs per prompt and easy VectorDB implementation via ChromaDB. (@JohnSmith4Reel) • crewAIInc: A flexible and powerful framework for enabling multi-agent task completion with support for any OpenAI API-compatible endpoints. (@whoabuddydev, @as_cybersamurai) • craftgen: Offers flexibility with its actor model and event-driven architecture, making human-in-the-loop workflows easy. (@Necmttn) • Lyzr: An agent framework mentioned for its capabilities in building multi-agent systems. (@theAIsailor) • Policy Synth: An open-source multiscale AI agent library. (@robertbjarnason) • GraphAI: Used for data flow programming to build genetic apps, enabling LLMs to generate genetic applications. (@snakajima) Libraries for Data Management and Vector Databases: • Pinecone: Used for vector databases and praised for its serverless capabilities. (@EricBDelisle, @jjacky, @theAIsailor) • ChromaDB: Integrated with AgentForge for easy VectorDB implementation. (@JohnSmith4Reel) • Redis, Qdrant, Postgres: Commonly mentioned databases for AI projects. (@who_mansu) • Greenhouse ECS: An ECS server framework built to be programmable by LLMs. (@EricBDelisle) Web and UI Frameworks: • Streamlit: Popular for creating web apps quickly from ideas, making it a go-to tool for many. (@josephs_tez, @SaidAitmbarek, @theAIsailor, @hTrapVader) • Gradio: Useful for creating a UI in a few lines of Python, often used to showcase work in team meetings. (@AI_NewsWaltz) • Wordware: Not a traditional framework but appreciated for building AI agents quickly and easily, even for less technical users. (@unable0_) • PreactJS: Used for front-end application interfaces. • NodeJS: Often paired with PreactJS for front-end development. Language Models and NLP Tools: • GPT-4o and other OpenAI tools: Widely used for their robust performance in various AI applications. (@TheHamedMP, @SaidAitmbarek, @theAIsailor) • Langchain and Llama Index: Common starting points for fast iterations in AI projects. (@TheHamedMP, @topmass) • DeepgramAI and AssemblyAI: Mentioned for their NLP capabilities. (@traviscorrigan) • Instruct: A lightweight library for structured outputs with LLMs. (@who_mansu, @HamzaFarhan) • Litellm: Another library for structured outputs with LLMs, often used alongside Instruct. (@who_mansu, @yoheinakajima) • DSPy: Highlighted for its power and versatility in AI projects. (@mysticaltech) • Magentic: A thin layer over LLM providers to simplify structured outputs and function calling. (@MichaelNStruwig, @jackmpcollins) • Openrouter: Offers multiple language models under the OpenAI API definition for easy experimentation. • Funcchain: An integration of Langchain with simpler typing and usage. (@akatzzzzz) Developer Tools and Platforms: • Replit: Favored as an IDE for its versatility and integration with various tools. (@theAIsailor) • Deno: Used in the craftgen tool for its code interpreter capabilities. (@Necmttn) • Imprompt: Highlighted for its ease of use in generative AI projects. (@jeffrschneider) • Vapi_AI, usebland, vocodehq, retellai: New tools currently being explored. (@traviscorrigan) • Hacknote: Recently added a feature called reactor creator, simplifying prompt writing and model selection. (@dbqsun) Evaluation and Testing: • LangSmith: Used for evaluation, debugging, and testing of LLM applications. (@as_cybersamurai) • Promptfoo: A testing framework to evaluate prompts and iterate on LLMs faster. (@Yossi_Dahan_) • Ragas: Used for evaluation in building RAG applications. (@AI_NewsWaltz) Miscellaneous Tools: • NATS / Socket IO / Redis: Used for messaging and caching in AI applications. (@EricBDelisle) • Tailwinds: A CSS framework used alongside NodeJS and PreactJS for front-end development. (@EricBDelisle) • BBScript: Acts as the glue between data, frontend, and backend. (@EricBDelisle) • Obsidian: A personal knowledge management tool that integrates with OpenAI APIs. (@BrianAndrenMA) • Convokit: An NLP toolkit from Cornell for conversational analysis. • Google Cloud's Vertex AI: Mentioned for its multimodal embedding model. (@jjacky) • PEFT: A tool used frequently for various AI projects. (@actualrealyorth) • Blacksmith: Provides fine-grained automation for agents and flow generation. (@MoMe36806866) • Trafilatura: Useful for processing web data, mentioned for its utility in LLM projects. (@TommyFalkowski) *Summarized by GPT-4o - sorry if it missed any! Of course this isn't a comprehensive list of great tools - but hopefully you find some interesting thing ones you didn't know of - and you now know someone you can ping about it :)