Thursday, I will be presenting my open-source LLM tooling work at the London Clojurians online event. I'll go over the components needed to build LLM apps and discuss whether or not these components are already a part of LLM models and APIs.
meetup.com/london-clojurians…
4
214
A demo of my latest work on Bosquet - an LLM tooling to work with Gen AI
github.com/zmedelis/bosquet
youtu.be/ywlNGiD9gCg
2
194
Žygimantas Medelis retweeted
The worst take that I have seen these past few days is that long context models like Gemini 1.5 will replace RAG.
This couldn't be further from the truth.
Let's explore one scenario. Imagine you had data that has a complex structure, changes regularly, and has an important temporal dimension (e.g., code edits/changes and web logs), meaning one data point might be connected to other historical data points and potentially many more in the future. You can't use today's long-context LLMs (on their own) to address use cases that depend on such type of data. Not only could this data be too complex for LLMs to handle but today's max context windows are just not feasible for this type of data. I mean, you will at the end of the day need some type of clever retrieval mechanism when dealing with such type of data.
Today's long-context LLMs work great with static information (books, video recordings, PDFs, etc.) but they are yet to be battle-tested on highly evolving information and knowledge. We will make progress towards addressing some of the challenges like "lost in the middle" and handling more complex structured and dynamic data but we still have a long way to go.
To tackle these types of problems you could potentially combine RAG and long-context LLMs to build a robust system that effectively and efficiently retrieves and performs large-scale analysis of key historical information. And not even this is enough for lots of scenarios. AI-based agents add even more complexity especially because a lot of data could be changing rapidly. Properly combining all these ideas is highly likely what will happen with complex use cases as opposed to general-purpose or long-context LLMs replacing everything.
And there are plenty of examples like this. You can't forget the fact that not all data is static, a lot of it is dynamic. When thinking of these applications it's always good to remember the 3 Vs of big data: velocity, volume, and variety. This is a lesson I learned while working at, wait for it, a search company. In my opinion, different families of LLMs will help solve different types of problems. We need to move on from this idea that there will be one LLM that will rule all.
I welcome other opinions about this but most of the takes I've seen are super dismissive without concrete examples or evidence.
Lastly, I would like to end this post with a quote from @OriolVinyalsML (VP of Research & DL Lead at Google Deepmind): "RAG ("retrieval-augmented generation") isn’t done for, even though we can do 1M or more tokens in context now. In fact, RAG has some nice properties that can enhance (and be enhanced by) long context. RAG allows you to find relevant information, but the way the model accesses it may be too restrictive due to compression. Long context may help bridge that gap, similar to how L1/L2 cache & main memory work together in modern CPUs."
An exploration into LLM Red Teaming:
1. How is LLM jailbreaking defined?
2. What are the motivations and goals of those seeking ways to make LLMs misbehave?
3. What strategies are used to red team?
arxiv.org/abs/2311.06237
2
172
Want to know if a phrase is present in one of the big data sets used for AI training? AllenAI (now making the news with OLMo) has 'What's In My Big Data?' WIMBD toolset that can search for phrases or domains in those datasets. wimbd.apps.allenai.org
1
1
129
Žygimantas Medelis retweeted
Small rant about LLMs and how I see them being put, rather thoughtlessly IMO, into productivity tools. 📄
TL;DR — Most knowledge work isn't a text-generation task, and your product shouldn't ship an implementation detail of LLMs as the end-user interface
stream.thesephist.com/update…
My talk on implementing 'Large Language Models as Optimizers' paper. It covers dealing with embeddings, Qdrant as a vector store, large documents, content splitting, evaluation, and setting up complex LLM flows.
youtube.com/watch?v=2JGQt9iW…
Done with Clojure LLM tooling available at github.com/zmedelis/bosquet
Žygimantas Medelis retweeted
On Friday, we'll have the next talk of our #LLM meetup series. @zzgmm will demonstrate how recent research papers can be elegantly implemented using #Bosquet in #Clojure.
clojureverse.org/t/scicloj-l…
Sunday tinkering to help with HuggingFace 🤗 dataset access from Clojure github.com/zmedelis/hfds-clj
Following the CoVe implementation another chain this time on text summarization task. Chain of Density is a neat technique to produce good summaries. It is interesting to track how summarization density increases as the LLM produces versions of summaries. zmedelis.github.io/bosquet/n…
Chain of Verification is a multi-step prompting technique addressing the issue of LLM hallucinations. CoVe can be declared and executed in Bosquet without custom coding and piping zmedelis.github.io/bosquet/n…
1
1
189
Chain of Verification is a multi-step prompting technique addressing the issue of LLM hallucinations. CoVe can be declared and executed in Bosquet without custom coding and piping zmedelis.github.io/bosquet/n…
LLMs are not great at generating complex structured data. They will often be a lot or a bit off. Even having 50.0% instead of 50 in a financial doc might be misleading. But finetuned smaller models like Llama-7B will outperform big ones in this task. arxiv.org/pdf/2309.08963.pdf
3
96
Clojure LLM meetup #5 is tomorrow 📅 Join us to hear about and discuss the LLM tools we are building 💪 clojureverse.org/t/scicloj-l…
I'll present Bosquet's approach to LLM memory 🧠 handling github.com/zmedelis/bosquet
1
100
LLM hallucinates when it adds fabricated info. This is not always bad. Precision parsing or completing contracts with LLM is useful but not that spectacular. Going with hallucinated what-ifs in the world of the contract is more interesting. arxiv.org/abs/2309.05922
Žygimantas Medelis retweeted
"I did this thing and automated 90% of grunt work on your side, help me fix the hard parts." - is how we're trying to design our AI interfaces. Why?
I'm increasingly convinced that our existing UX primitives don't hold up for modern AI workflows.
Most our systems designed for human input move in one direction. Input ➡️ Output.
Generative flows are much better served by AI Output ➡️ Corrective Input ➡️ Output.
This fundamentally flows in the backward direction - and provides more affordance for the user.
Instead of saying "give me some small input, I'll do the whole thing for you", the better method is "I did this thing and automated 90% of grunt work on your side, help me fix the hard parts."
Copilot is the only one that's figured this out.
This isn't a new thought - hell we've been thinking about corrective interfaces for a long time: olickel.com/one-click-crew-c…
But so far I can't find any more examples out in the wild - do you know any?
🧠Generative Agents paper has a nice LLM memory approach, showing how memory handling should be added to an LLM-reliant app. It does not focus on how large content is stored. VectorDB, or whatnot. Instead, it defines how memory is produced and consumed arxiv.org/pdf/2304.03442.pdf
1
126
Bosquet - a Clojure LLM Ops lib. Supports multiple LLM API providers, agents, and prompt chaining github.com/zmedelis/bosquet
1
9
175
I am delighted to receive Clojurists Together support to continue my work on Bosquet - a Clojure LLM lib for building AI apps. Will be expanding agents and adding memory support.
github.com/zmedelis/bosquet
Clojurists Together is excited to announce our next round of funding for Q3 202 projects. Congratulations all! clojuriststogether.org/news/…
3
150












