
Software Architect @wescocorp • Ex - @Siemens • Creator @InsightVeda • Sharing engineering notes
Bengaluru
Joined January 2021
- Tweets3.7K
- Following331
- Followers1.6K
- Likes2.1K
If you're aiming for an AI engineering job in 2027, get comfortable with these 50 terms.
1/ Agents
• Agent Handoff
• Idempotency Key
• Agent-to-Agent Communication
• MCP (Model Context Protocol)
• Irreversible Steps
• Tool Allowlist
• Checkpointed State
• Loop Detection Fingerprint
• Escalation Policy
• Tool Side-effect Class
2/ Workflows
• Durable Execution
• Compensating Transaction
• Saga Orchestration
• Human-in-the-Loop Gate
• Replay vs Resume
• Side-effect Isolation
• Exactly-once Semantics
• Workflow Determinism
• Approval Boundary
• Working vs Durable State
3/ RAG
• Hybrid Retrieval (RRF)
• ACL-aware Retrieval
• Groundedness Score
• Lost-in-the-Middle
• Query Rewrite
• Reranker Cutoff
• Chunk Boundary Leakage
• Embedding Drift
• Negative Document Filtering
• Citation Grounding
4/ Evals
• Golden Set Drift
• LLM-as-Judge Calibration
• Pass@k Reliability
• Faithfulness vs Relevance
• Online Eval Signals
• CI Regression Gate
• Trajectory Eval
• Inter-rater Agreement
• Eval Slice Analysis
• Regression Attribution
5/ Production
• Retry Budget
• Prompt Injection Boundary
• Model Routing
• Trace Attribution
• Per-request Cost Budget
• Circuit Breaker for Tools
• Shadow Prompt Rollout
• Observability Span
• Token Budget Guardrail
• Context Compaction
GPT-5.2 scored 58.7% with the OpenAI Agents SDK, compared with 53.5% with the Claude Agent SDK.
That's a 5.2 percentage-point difference using the same model.
But the more surprising result comes from Claude Opus 4.5, which scored 71.3% with the Claude Agent SDK versus 47.1% with the OpenAI Agents SDK.
In AGENCYBENCH, researchers compared three agent scaffolds across 10 representative scenarios.
Here are the results:
GPT-5.2:
- Custom scaffold: 57.4%
- Claude Agent SDK: 53.5%
- OpenAI Agents SDK: 58.7%
Claude Opus 4.5:
- Custom scaffold: 50.8%
- Claude Agent SDK: 71.3%
- OpenAI Agents SDK: 47.1%
Kimi K2 Thinking:
- Custom scaffold: 50.5%
- Claude Agent SDK: 44.6%
- OpenAI Agents SDK: 37.7%
The differences are worth looking at.
Claude Opus 4.5 improved by 20.5 percentage points with the Claude Agent SDK compared with the custom scaffold.
GPT-5.2 performed best with the OpenAI Agents SDK, while Kimi K2 Thinking performed best with the custom scaffold.
There isn't one framework that produces the best results across all three models.
This has important implications for agent architecture.
The framework influences how models interact with tools, manage context, process feedback and execute multi-step workflows.
That means selecting an LLM and selecting an agent framework cannot be treated as completely independent decisions.
The combination needs to be evaluated against the actual tasks the system must perform.
A model's benchmark score alone doesn't tell you how well it will perform inside a production agent.
Source research paper link in the first comment.
1
1
85
Research paper: AGENCYBENCH: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts
aclanthology.org/2026.acl-lo…
33
Everyone talks about Human-in-the-Loop (HITL) for AI agents.
But there's one important question:
At what point should a human actually intervene?
Consider an agentic workflow processing 10,000 operational actions.
If every action requires human approval, and each review takes just 30 seconds, that's:
10,000 approvals × 30 seconds = 83.3 hours of human review.
Now imagine only 1% of those actions require escalation.
That's 100 approvals, or just 50 minutes of review time.
The difference is significant.
In e-commerce, routine operations such as checking order status, validating delivery details or processing eligible low-value refunds can happen autonomously within predefined policies and limits.
In cybersecurity, agents can continuously analyse alerts, correlate events and perform pre-authorized containment actions.
Waiting for human approval at every operational step defeats much of the speed that agentic systems are designed to provide.
A better approach is to introduce human intervention at different levels of risk:
1. Low-risk operations: Agents execute routine, reversible actions within predefined policies.
2. Exception handling: Uncertainty, conflicting information, unusual transactions or policy violations trigger escalation.
3. Critical decisions: High-impact financial actions, permanent account restrictions and significant security decisions require human approval.
4. Governance: Humans periodically review agent behaviour, failures and autonomy boundaries.
The architecture also needs clear escalation thresholds, timeout handling and audit trails so that an agent doesn't continue with a risky action simply because human approval is unavailable.
Human oversight should be based on the consequence and risk of an action, not on every individual tool call.
The goal is to let agents operate at machine speed within defined boundaries, while bringing humans into decisions where their judgment actually matters.
Your AI agent reduces 50,000 tokens of conversation history to a 5,000-token summary.
That's a 90% reduction in context.
But what if it also removes the one instruction that changed the entire task?
Consider this scenario:
An agent is executing a 20-step workflow.
- Step 3: The user asks it to contact a supplier.
- Step 12: The user cancels that instruction.
- Step 18: The agent summarizes its conversation history.
- Step 20: The agent resumes execution.
If the summary retains the original instruction but loses the cancellation, the agent may contact the supplier anyway.
The context is smaller, but the task state is now incorrect.
This is a subtle risk in long-running AI agents.
Four design practices can help prevent this:
1. Current state: Maintain the latest task status separately from conversational summaries.
2. Instruction history: Track changes, cancellations and approvals as explicit events.
3. Decision priority: Preserve the latest valid instruction rather than relying on frequency or recency alone.
4. Action validation: Before executing an external action, verify that the instruction is still valid and authorized.
There is an important distinction between summarizing conversation history and maintaining workflow state.
A summary helps the model understand what happened.
Structured state tells the system what is currently true.
For agents that execute long-running workflows or interact with external systems, both are necessary.
Saving 90% of the tokens is useful. Preserving the correct decision is essential.
3
4
202
Claude Opus 4.5 fell from 96% to 14.7% as the context grew from 8K to 256K tokens.
That's an 81.3 percentage-point drop.
ChatGPT-5.2 Medium also fell, from 72% to 21.3%.
And other models show similarly steep declines.
The LOCA-bench report evaluates how AI agents perform as their working context grows while completing tasks involving tools and external services.
Here is how some models performed:
Claude Opus 4.5:
8K: 96.0%
16K: 84.0%
32K: 84.0%
64K: 65.3%
96K: 45.3%
128K: 34.0%
256K: 14.7%
ChatGPT-5.2 Medium:
8K: 72.0%
16K: 70.7%
32K: 60.0%
64K: 52.0%
96K: 44.0%
128K: 38.7%
256K: 21.3%
Other models, from 8K to 256K:
Gemini 3 Flash: 64.0% → 17.3%
DeepSeek V3.2 Thinking: 78.7% → 6.7%
MiniMax M2.1: 69.3% → 5.3%
GLM-4.7: 76.0% → 5.3%
Kimi K2 Thinking: 74.7% → 2.7%
There are two things worth noticing.
First, a larger context window doesn't guarantee that an agent can use all that information effectively.
Second, model performance isn't the whole story. How the agent manages its growing context also matters.
For production AI systems, this has architectural implications.
Instead of continuously appending every tool response, conversation and intermediate result to the context, systems need to manage what stays relevant.
Some approaches worth considering:
- Summarising completed interactions.
- Keeping structured workflow state outside the context window.
- Retrieving relevant history when needed.
- Removing redundant tool outputs.
- Separating persistent memory from temporary working context.
The interesting engineering question is how much context an agent should carry forward, not simply how much its model can accept.
One caveat: these are LOCA-bench agent task accuracy results under controlled environment growth. They are not a universal measure of long-context performance.
Source report link in the first comment.
6
3
148
Source: LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth
Research paper:
arxiv.org/abs/2602.07962
21
One strong post can do the work of 100. But you’ll usually need to create 100 posts before you hit that one.
Follow me on Instagram if you’re there.
instagram.com/bhuvnesharyaof…
3
121
The biggest problem with vibe coders is that they get instantly excited by how many lines of code have been written.
1
3
198
Your RAG system gives the correct answer.
Does that mean retrieval is working?
Not necessarily.
An LLM may already know the answer from its training data or infer it from incomplete context. So a correct final response doesn't automatically prove that your retrieval pipeline found the right evidence.
This creates a blind spot when evaluating RAG applications.
Consider two separate questions:
Did retrieval find the right information?
Did the LLM generate a correct answer grounded in that information?
These need different evaluation signals.
For retrieval, metrics such as Recall@K and MRR help measure whether relevant documents are being found and how highly they rank.
For generation, evaluate answer correctness, faithfulness to the retrieved context and citation accuracy.
There is another useful test: remove or deliberately alter the retrieved evidence and observe whether the answer changes appropriately.
If the system keeps producing the same confident answer, it may be relying on its parametric knowledge rather than the evidence supplied by retrieval.
I would evaluate the retrieval and generation stages independently, then measure end-to-end task success.
Otherwise, a good-looking answer quality score can hide a retrieval problem that will surface later when the system encounters information the model doesn't already know.
3
4
207
Your RAG system retrieves 5 relevant documents.
Four say the customer is on the Enterprise plan.
One says the customer is on the Professional plan.
Which answer should the LLM generate?
This is a common problem when retrieval systems work with data from multiple sources, document versions or update cycles.
Increasing the number of retrieved chunks won't solve it. In fact, it can give the model more conflicting information to reason over.
A few design decisions can make a difference:
1. Preserve document metadata such as source, version, tenant and last-updated timestamp during ingestion.
2. Define source authority. A current customer database record may take precedence over an older PDF or an archived support document.
3. Apply freshness and version filters before passing retrieved chunks to the model.
4. Detect conflicting values for important fields instead of expecting the LLM to silently resolve every contradiction.
5. When the conflict cannot be resolved using source rules, let the system return an uncertainty or request clarification.
There is another subtle issue here: relevance and authority are different things.
A document can be semantically very relevant and still contain outdated information.
For production RAG systems, retrieval quality needs to account for both.
1
1
58
An AI agent can call an API, query a database, create a support ticket and update a customer record.
But whose permissions is it using?
This becomes an important design question when agents move from prototypes into enterprise applications.
Giving an agent a shared service account with broad access is easy to implement, but it creates problems:
- Every agent may inherit the same permissions.
- Audit logs may show the application identity instead of the user who initiated the action.
- Revoking one user's access may not stop an already-running agent.
- A compromised agent could use permissions far beyond what its task requires.
A better approach is to treat an agent as a first-class identity in the security architecture.
Depending on the workflow, this can involve:
1. Workload identity for authenticating the agent itself.
2. Delegated user authorization when an agent acts on someone's behalf.
3. Short-lived tokens with narrowly scoped permissions.
4. Policy checks at the tool or API boundary, not just when the agent starts.
5. Audit records that connect the user, agent, requested action and actual resource access.
There is also an important distinction between authenticating an agent and authorizing what it can do. A valid agent identity should never automatically mean unrestricted access.
This gets particularly interesting in multi-agent systems, where one agent delegates work to another. Identity, permission inheritance and auditability need to remain clear across the entire execution chain.
Your AI agent has completed 18 steps of a 20-step workflow.
Then the application crashes.
Do you execute all 18 steps again?
This is a real design problem with long-running agents, especially when workflows involve external APIs, document processing or human approval.
LangGraph provides checkpointing through its persistence layer. It can save graph state as execution progresses and resume a workflow from a saved checkpoint.
But there is an implementation detail worth understanding.
A checkpoint doesn't guarantee that every operation before the failure will never execute again.
Imagine a workflow that creates a support ticket, sends a notification and then updates a customer record.
If execution fails before the relevant progress is checkpointed, some operations may run again during recovery.
This is why external side effects need careful handling. Use idempotency keys for operations that must not be duplicated, and isolate work into appropriately scoped nodes or tasks so completed results can be recovered.
LangGraph also supports interrupting workflows for human approval and resuming them later using persisted state.
If you're building agents that run for minutes or hours, pay attention to checkpoint boundaries, replay behaviour and how external operations behave during recovery.
9
6
300
A product price changes in PostgreSQL at 10:00 AM.
At 10:01, the AI assistant still tells a customer the old price.
The database is correct. The model is working as expected. The answer is still wrong.
This can happen when different parts of an application maintain their own copies of data. Redis may still have the cached response, an event may be waiting to be processed, or the search index may not have received the latest update.
One way to reduce this risk is to make data updates traceable across the systems that consume them.
For example, when a product price changes, the application can publish an update through an event or CDC pipeline. Consumers can use that change to refresh the cache and update the search index. Version numbers or update timestamps can help identify whether the retrieved record is older than the source.
There is another important consideration: not every piece of data needs the same freshness guarantee.
A product description might tolerate a few minutes of delay. A price, account balance or order status may need a fresh check against the source system before the AI application uses it to answer or take action.
That distinction should influence how we design caching, event propagation and retrieval.
The goal is to make stale data detectable and prevent time-sensitive decisions from relying on outdated values.
1
3
257
Retries are supposed to make distributed systems more reliable.
But what happens when the operation you're retrying has already succeeded?
Imagine an AI agent initiating a ₹5,000 payment.
The payment service processes it successfully, but the response times out before reaching the agent.
The agent sees a timeout, assumes the operation failed, and retries.
Now the customer may have been charged twice.
This isn't necessarily a model problem. It's a distributed systems problem.
A few things need to be designed into the application:
→ 𝐈𝐝𝐞𝐦𝐩𝐨𝐭𝐞𝐧𝐜𝐲 𝐤𝐞𝐲𝐬: Make repeated requests with the same operation identifier safe.
→ 𝐒𝐭𝐚𝐭𝐞 𝐭𝐫𝐚𝐜𝐤𝐢𝐧𝐠: Know whether an operation is pending, completed, failed or needs reconciliation.
→ 𝐑𝐞𝐭𝐫𝐲 𝐩𝐨𝐥𝐢𝐜𝐲: Distinguish transient failures from permanent ones.
→ 𝐑𝐞𝐬𝐮𝐥𝐭 𝐯𝐞𝐫𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧: Check the original operation's status before attempting it again.
This becomes even more important with AI agents.
An agent may retry a tool call because it didn't receive a response. But the underlying operation might already have completed.
The model can decide to retry. The application must ensure that retrying doesn't repeat the side effect.
𝐑𝐞𝐭𝐫𝐲 𝐥𝐨𝐠𝐢𝐜 𝐢𝐬 𝐧𝐨𝐭 𝐚 𝐬𝐮𝐛𝐬𝐭𝐢𝐭𝐮𝐭𝐞 𝐟𝐨𝐫 𝐢𝐝𝐞𝐦𝐩𝐨𝐭𝐞𝐧𝐜𝐲.
1
4
155
Want to build an AI engineering portfolio in 2026?
Here are 5 projects I'd recommend building. Each one demonstrates a different engineering capability that matters when moving AI applications towards production.
𝐎𝟏. 𝐌𝐮𝐥𝐭𝐢-𝐀𝐠𝐞𝐧𝐭 𝐑𝐞𝐬𝐞𝐚𝐫𝐜𝐡 𝐒𝐲𝐬𝐭𝐞𝐦
Build a workflow where researcher, writer and critic agents collaborate to produce a report with citations.
Dataset: huggingface.co/datasets/hotp…
What it demonstrates: agent orchestration, task delegation and source verification.
𝐎𝟐. 𝐃𝐨𝐜𝐮𝐦𝐞𝐧𝐭 𝐐&𝐀 𝐰𝐢𝐭𝐡 𝐂𝐢𝐭𝐚𝐭𝐢𝐨𝐧𝐬
Build a knowledge assistant that answers questions from documents and points to the exact source passages behind each answer.
Dataset: huggingface.co/datasets/BeIR…
What it demonstrates: document ingestion, chunking, retrieval, reranking and grounded responses.
𝐎𝟑. 𝐋𝐋𝐌 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐇𝐚𝐫𝐧𝐞𝐬𝐬
Build an evaluation pipeline that tests model responses against defined criteria and catches regressions before deployment.
Dataset: huggingface.co/datasets/goog…
What it demonstrates: evaluation datasets, scoring, regression testing and CI integration.
𝐎𝟒. 𝐀𝐈 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐰𝐢𝐭𝐡 𝐄𝐧𝐭𝐞𝐫𝐩𝐫𝐢𝐬𝐞 𝐒𝐒𝐎
Build an AI application with login, role-based access, rate limits and audit logs. The model should operate behind the same security boundaries as the rest of the application.
Starter: github.com/keycloak/keycloak…
What it demonstrates: authentication, authorization, tenant isolation and application security.
𝐎𝟓. 𝐂𝐮𝐬𝐭𝐨𝐦𝐞𝐫 𝐒𝐮𝐩𝐩𝐨𝐫𝐭 𝐀𝐠𝐞𝐧𝐭 𝐰𝐢𝐭𝐡 𝐓𝐨𝐨𝐥𝐬
Build an agent that can look up orders, check refund eligibility, initiate permitted actions and hand off complex cases to a human.
Dataset: huggingface.co/datasets/bite…
What it demonstrates: tool calling, workflow controls, API integration and human approval.
The real value isn't having five projects on GitHub. It's being able to explain the architecture, design decisions, failure handling, security boundaries and trade-offs behind each one.
Pick one, build it end to end, document the decisions and make it something another engineer can run.
Save this list for your next AI project, or repost it with someone building their AI engineering portfolio.




