{"id":188,"date":"2026-04-06T15:30:52","date_gmt":"2026-04-06T15:30:52","guid":{"rendered":"https:\/\/bkbc.net\/index.php\/2026\/04\/06\/a-guide-to-context-engineering-for\/"},"modified":"2026-04-06T15:30:52","modified_gmt":"2026-04-06T15:30:52","slug":"a-guide-to-context-engineering-for","status":"publish","type":"post","link":"https:\/\/bkbc.net\/index.php\/2026\/04\/06\/a-guide-to-context-engineering-for\/","title":{"rendered":"A Guide to Context Engineering for LLMs"},"content":{"rendered":"<div>\n<h2><a href=\"https:\/\/go.bytebytego.com\/Sentry_040626\">The workshop for teams drowning in observability tools (Sponsored)<\/a><\/h2>\n<div class=\"captioned-image-container\">\n<figure><a class=\"image-link image2 is-viewable-img\" target=\"_blank\" href=\"https:\/\/go.bytebytego.com\/Sentry_040626\" data-component-name=\"Image2ToDOM\"><\/p>\n<div class=\"image2-inset\"><picture><source type=\"image\/webp\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!l9qe!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!l9qe!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!l9qe!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!l9qe!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png 1456w\" sizes=\"100vw\"><img decoding=\"async\" src=\"https:\/\/bkbc.net\/wp-content\/uploads\/2026\/04\/81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png\" width=\"1200\" height=\"1200\" data-attrs='{\"src\":\"https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png\",\"srcNoWatermark\":null,\"fullscreen\":null,\"imageSize\":null,\"height\":1200,\"width\":1200,\"resizeWidth\":null,\"bytes\":3219528,\"alt\":null,\"title\":null,\"type\":\"image\/png\",\"href\":\"https:\/\/go.bytebytego.com\/Sentry_040626\",\"belowTheFold\":false,\"topImage\":true,\"internalRedirect\":\"https:\/\/blog.bytebytego.com\/i\/193030505?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png\",\"isProcessing\":false,\"align\":null,\"offset\":false}' class=\"sizing-normal\" alt=\"\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!l9qe!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!l9qe!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!l9qe!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!l9qe!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F81bba391-502a-4680-a1ac-941be8df2f5a_1200x1200.png 1456w\" sizes=\"100vw\" fetchpriority=\"high\"><\/picture>\n<div class=\"image-link-expand\">\n<div class=\"pencraft pc-display-flex pc-gap-8 pc-reset\"><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container restack-image\"><svg role=\"img\" width=\"20\" height=\"20\" viewbox=\"0 0 20 20\" fill=\"none\" stroke-width=\"1.5\" stroke=\"var(--color-fg-primary)\" stroke-linecap=\"round\" stroke-linejoin=\"round\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><g><title><\/title><path d=\"M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882\"><\/path><\/g><\/svg><\/button><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container view-image\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20\" height=\"20\" viewbox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" class=\"lucide lucide-maximize2 lucide-maximize-2\"><polyline points=\"15 3 21 3 21 9\"><\/polyline><polyline points=\"9 21 3 21 3 15\"><\/polyline><line x1=\"21\" x2=\"14\" y1=\"3\" y2=\"10\"><\/line><line x1=\"3\" x2=\"10\" y1=\"21\" y2=\"14\"><\/line><\/svg><\/button><\/div>\n<\/div>\n<\/div>\n<p><\/a><\/figure>\n<\/div>\n<p>Five vendors, rising costs, and you still can\u2019t tell why something broke.<\/p>\n<p>Sentry\u2019s Lazar Nikolov sits down with Recurly\u2019s Chris Barton to talk through what observability consolidation actually looks like in practice: how to evaluate your options, where AI fits in, and how to think about cost when you\u2019re ready to simplify.<\/p>\n<p class=\"button-wrapper\" data-attrs='{\"url\":\"https:\/\/go.bytebytego.com\/Sentry_040626\",\"text\":\"Register your spot today\",\"action\":null,\"class\":null}' data-component-name=\"ButtonCreateButton\"><a class=\"button primary\" href=\"https:\/\/go.bytebytego.com\/Sentry_040626\"><span>Register your spot today<\/span><\/a><\/p>\n<div>\n<hr>\n<\/div>\n<p style=\"text-align: justify;\">Giving an LLM more information can make it dumber. A<a href=\"https:\/\/research.trychroma.com\/context-rot\"> 2025 research study by Chroma<\/a> tested 18 of the most powerful language models available, including GPT-4.1, Claude, and Gemini, and found that every single one performed worse as the amount of input grew.<\/p>\n<p style=\"text-align: justify;\">The degradation wasn\u2019t minor, either. Some models held steady at 95% accuracy and then nosedived to 60% once the input crossed a certain length.<\/p>\n<p style=\"text-align: justify;\">This finding busts one of the most common myths about working with LLMs that more context is always better. The reality is that LLMs have architectural blind spots that make what you put in front of them, and how you structure it, far more important than how much you include.<\/p>\n<p style=\"text-align: justify;\">The discipline of getting this right is called context engineering.<\/p>\n<p style=\"text-align: justify;\">In this article, we\u2019ll look at how LLMs actually process the information you give them, what context engineering is, and the strategies that can help with it.<\/p>\n<h2 style=\"text-align: justify;\">Key Terminologies<\/h2>\n<p style=\"text-align: justify;\">Before we go further, there are three terms that come up constantly when talking about LLMs. Getting clear on these first will make everything that follows much easier to reason about.<\/p>\n<ul>\n<li>\n<p style=\"text-align: justify;\"><strong>Tokens: <\/strong>They are the units LLMs think in. They aren\u2019t full words, but rather chunks of text that average roughly three-quarters of a word each. The word \u201ccontext\u201d is one token, while the word \u201cengineering\u201d gets split into two. Every piece of text the model processes, from your question to its instructions to any documents you\u2019ve included, is measured in tokens.<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\"><strong>Context Window: <\/strong>It is the total number of tokens the model can see at once during a single interaction. Everything has to fit inside this window: the system instructions that define the model\u2019s behavior, the conversation history, any external documents or data you\u2019ve injected, and your actual question. Modern models advertise context windows ranging from 128,000 to over 2 million tokens. That sounds enormous, but as we\u2019ll see, bigger isn\u2019t straightforwardly better.<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\"><strong>Attention: <\/strong>This is the mechanism the model uses to figure out which tokens matter to which other tokens. Before generating each new token of its response, the model compares it against every other token currently in the context window. This gives LLMs their ability to connect ideas across long stretches of text, but it\u2019s also the source of their most important limitations.<\/p>\n<\/li>\n<\/ul>\n<h2 style=\"text-align: justify;\">How LLMs Process Context<\/h2>\n<p style=\"text-align: justify;\">When we send text to an LLM, it doesn\u2019t read from top to bottom the way a human would. The attention mechanism compares every token against every other token to compute relationships, which means the model can, in principle, connect an idea from the first sentence of the input to one in the last sentence. However, this power comes with two critical costs.<\/p>\n<ul>\n<li>\n<p style=\"text-align: justify;\">The first is computational. Doubling the number of tokens in the context window roughly quadruples the computation required. Longer contexts are disproportionally slower and more expensive.<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\">The second cost is more consequential. Attention isn\u2019t distributed evenly across the context window. Research has consistently shown that LLMs pay the most attention to tokens at the beginning and end of the input, with a significant drop-off in the middle. This is known as the \u201clost in the middle\u201d problem, and research has found that accuracy can drop by over 30% when relevant information is placed in the middle of the input compared to the beginning or end.<\/p>\n<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">See the diagram below that shows the attention curve:<\/p>\n<div class=\"captioned-image-container\">\n<figure><a class=\"image-link image2 is-viewable-img\" target=\"_blank\" href=\"https:\/\/substackcdn.com\/image\/fetch\/%24s_!NrJV!,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png\" data-component-name=\"Image2ToDOM\"><\/p>\n<div class=\"image2-inset\"><picture><source type=\"image\/webp\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!NrJV!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!NrJV!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!NrJV!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!NrJV!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png 1456w\" sizes=\"100vw\"><img decoding=\"async\" src=\"https:\/\/bkbc.net\/wp-content\/uploads\/2026\/04\/ad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png\" width=\"1456\" height=\"873\" data-attrs='{\"src\":\"https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/ad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png\",\"srcNoWatermark\":null,\"fullscreen\":null,\"imageSize\":null,\"height\":873,\"width\":1456,\"resizeWidth\":null,\"bytes\":164017,\"alt\":null,\"title\":null,\"type\":\"image\/png\",\"href\":null,\"belowTheFold\":true,\"topImage\":false,\"internalRedirect\":\"https:\/\/blog.bytebytego.com\/i\/193030505?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png\",\"isProcessing\":false,\"align\":null,\"offset\":false}' class=\"sizing-normal\" alt=\"\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!NrJV!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!NrJV!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!NrJV!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!NrJV!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fad9e966a-1edf-4ddc-bf4b-14ab36e23b70_2910x1744.png 1456w\" sizes=\"auto, 100vw\" loading=\"lazy\"><\/picture>\n<div class=\"image-link-expand\">\n<div class=\"pencraft pc-display-flex pc-gap-8 pc-reset\"><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container restack-image\"><svg role=\"img\" width=\"20\" height=\"20\" viewbox=\"0 0 20 20\" fill=\"none\" stroke-width=\"1.5\" stroke=\"var(--color-fg-primary)\" stroke-linecap=\"round\" stroke-linejoin=\"round\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><g><title><\/title><path d=\"M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882\"><\/path><\/g><\/svg><\/button><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container view-image\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20\" height=\"20\" viewbox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" class=\"lucide lucide-maximize2 lucide-maximize-2\"><polyline points=\"15 3 21 3 21 9\"><\/polyline><polyline points=\"9 21 3 21 3 15\"><\/polyline><line x1=\"21\" x2=\"14\" y1=\"3\" y2=\"10\"><\/line><line x1=\"3\" x2=\"10\" y1=\"21\" y2=\"14\"><\/line><\/svg><\/button><\/div>\n<\/div>\n<\/div>\n<p><\/a><\/figure>\n<\/div>\n<p style=\"text-align: justify;\">This isn\u2019t a bug in any particular model, but rather a structural property of how transformers (the neural network architecture that powers virtually all modern LLMs) encode the position of tokens.<\/p>\n<p style=\"text-align: justify;\">The positional encoding method used in most modern LLMs (called Rotary Position Embedding, or RoPE) introduces a decay effect that makes tokens far from both the start and end of the sequence land in a low-attention zone. Newer models have reduced the severity, but no production model has fully eliminated it.<\/p>\n<p style=\"text-align: justify;\">The practical implication is that the position of information in the input matters as much as the information itself. If we paste a long document into an LLM, the model is most likely to miss information buried in the middle pages.<\/p>\n<h2 style=\"text-align: justify;\">Why More Context Can Hurt<\/h2>\n<p style=\"text-align: justify;\">The uneven attention distribution is one problem, but there\u2019s a broader pattern that compounds it, known as context rot.<\/p>\n<p style=\"text-align: justify;\">Context rot is the degradation of LLM performance as input length increases, even on simple tasks. The<a href=\"https:\/\/research.trychroma.com\/context-rot\"> Chroma research team\u2019s 2025 study<\/a> tested 18 frontier models and found that this degradation isn\u2019t gradual. Models can maintain near-perfect accuracy up to a certain context length, and then performance drops off a cliff unpredictably, varying by model and by task in ways that make it impossible to reliably predict when you\u2019ll hit a breaking point.<\/p>\n<p style=\"text-align: justify;\">Why does this happen?<\/p>\n<p style=\"text-align: justify;\">Every token you add to the context window draws from a finite attention budget. Irrelevant information buries important information in low-attention zones, and content that sounds related but isn\u2019t actually useful confuses the model\u2019s ability to identify what\u2019s relevant. The model doesn\u2019t get smarter with more input, but kind of gets distracted.<\/p>\n<p style=\"text-align: justify;\">On top of this, LLMs are stateless. They have zero memory between calls, and each interaction starts completely fresh. When there is a multi-turn conversation with an LLM like ChatGPT, and it seems to \u201cremember\u201d what we said earlier, that\u2019s because the system is re-injecting the conversation history into the context window each time. The model itself remembers nothing, which means someone, or some system, has to decide for every single call what information to include, what to leave out, and how to structure it.<\/p>\n<p style=\"text-align: justify;\">There\u2019s also a meaningful gap between marketing and reality. Models advertise million-token context windows, and they pass simple benchmarks at those lengths. However, the effective context length, where the model actually uses information reliably, is often much smaller. Passing a \u201cneedle in a haystack\u201d test (finding one planted sentence in a long document) is very different from reliably synthesizing information scattered across hundreds of pages<\/p>\n<h2 style=\"text-align: justify;\">Defining Context Engineering<\/h2>\n<p style=\"text-align: justify;\">Context engineering is the practice of designing, assembling, and managing the entire information environment an LLM sees before it generates a response. It goes beyond writing a single good instruction to orchestrating everything that fills the context window, so the model has exactly what it needs for the task at hand and nothing more.<\/p>\n<p style=\"text-align: justify;\">To understand what this involves, it helps to see what actually competes for space inside a context window. There are six types of context in a typical LLM call:<\/p>\n<ul>\n<li>\n<p style=\"text-align: justify;\">System instructions (the behavioral rules, persona, and guidelines the model follows)<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\">User input (your actual question or command)<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\">Conversation history (the short-term memory of the current session)<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\">Retrieved knowledge (documents, database results, or API responses pulled in from external sources)<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\">Tool descriptions (definitions of tools the model can call and how to use them)<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\">Tool outputs (results returned from previous tool calls)<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\">The user\u2019s actual question is often a tiny fraction of the total token count.<\/p>\n<\/li>\n<\/ul>\n<div class=\"captioned-image-container\">\n<figure><a class=\"image-link image2 is-viewable-img\" target=\"_blank\" href=\"https:\/\/substackcdn.com\/image\/fetch\/%24s_!GjNA!,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png\" data-component-name=\"Image2ToDOM\"><\/p>\n<div class=\"image2-inset\"><picture><source type=\"image\/webp\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!GjNA!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!GjNA!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!GjNA!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!GjNA!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png 1456w\" sizes=\"100vw\"><img decoding=\"async\" src=\"https:\/\/bkbc.net\/wp-content\/uploads\/2026\/04\/9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png\" width=\"1456\" height=\"875\" data-attrs='{\"src\":\"https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png\",\"srcNoWatermark\":null,\"fullscreen\":null,\"imageSize\":null,\"height\":875,\"width\":1456,\"resizeWidth\":null,\"bytes\":96412,\"alt\":null,\"title\":null,\"type\":\"image\/png\",\"href\":null,\"belowTheFold\":true,\"topImage\":false,\"internalRedirect\":\"https:\/\/blog.bytebytego.com\/i\/193030505?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png\",\"isProcessing\":false,\"align\":null,\"offset\":false}' class=\"sizing-normal\" alt=\"\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!GjNA!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!GjNA!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!GjNA!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!GjNA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9534a00d-5563-42ba-82e9-6c1a2aa1f04f_2526x1518.png 1456w\" sizes=\"auto, 100vw\" loading=\"lazy\"><\/picture>\n<div class=\"image-link-expand\">\n<div class=\"pencraft pc-display-flex pc-gap-8 pc-reset\"><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container restack-image\"><svg role=\"img\" width=\"20\" height=\"20\" viewbox=\"0 0 20 20\" fill=\"none\" stroke-width=\"1.5\" stroke=\"var(--color-fg-primary)\" stroke-linecap=\"round\" stroke-linejoin=\"round\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><g><title><\/title><path d=\"M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882\"><\/path><\/g><\/svg><\/button><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container view-image\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20\" height=\"20\" viewbox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" class=\"lucide lucide-maximize2 lucide-maximize-2\"><polyline points=\"15 3 21 3 21 9\"><\/polyline><polyline points=\"9 21 3 21 3 15\"><\/polyline><line x1=\"21\" x2=\"14\" y1=\"3\" y2=\"10\"><\/line><line x1=\"3\" x2=\"10\" y1=\"21\" y2=\"14\"><\/line><\/svg><\/button><\/div>\n<\/div>\n<\/div>\n<p><\/a><\/figure>\n<\/div>\n<p style=\"text-align: justify;\">The rest is infrastructure, and that infrastructure is what context engineering designs.<\/p>\n<p style=\"text-align: justify;\">This also clarifies how context engineering differs from prompt engineering. Prompt engineering asks, \u201cHow do I phrase my instruction to get the best result?\u201d On the other hand, Context engineering asks, \u201cWhat does the model need to see right now, and how do I assemble all of it dynamically?\u201d<\/p>\n<p style=\"text-align: justify;\">Prompt engineering is one component within context engineering, focused on the instruction layer, while context engineering encompasses the full information system around the model. As Andrej Karpathy put it in a<a href=\"https:\/\/x.com\/karpathy\/status\/1937902205765607626\"> widely referenced post<\/a>, context engineering is the \u201cdelicate art and science of filling the context window with just the right information for the next step.\u201d<\/p>\n<p style=\"text-align: justify;\">Two people using the same model can get wildly different results. The model is the same, but the context is different, and context engineering is the factor that determines things.<\/p>\n<h2 style=\"text-align: justify;\">Core Strategies<\/h2>\n<p style=\"text-align: justify;\">Developers have converged on four broad strategies for managing context,<a href=\"https:\/\/blog.langchain.com\/context-engineering-for-agents\/\"> categorized <\/a>as write, select, compress, and isolate. Each one is a direct response to a specific constraint we\u2019ve already covered.<\/p>\n<h3 style=\"text-align: justify;\">Write: Save Context Externally<\/h3>\n<p style=\"text-align: justify;\">The constraint it addresses is that the context window is finite, and statelessness means information is lost between calls.<\/p>\n<p style=\"text-align: justify;\">Instead of trying to keep everything inside the context window, save important information to external storage and bring it back when needed. This takes two main forms.<\/p>\n<ul>\n<li>\n<p style=\"text-align: justify;\">The first is scratchpads, where an agent saves intermediate plans, notes, or reasoning steps to external storage during a long-running task.<a href=\"https:\/\/www.anthropic.com\/engineering\/building-effective-agents\"> Anthropic\u2019s multi-agent research system<\/a> does exactly this. The lead researcher agent writes its plan to external memory at the start of a task, because if the context window exceeds 200,000 tokens, it gets truncated and the plan would be lost.<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\">The second form is long-term memory, which involves persisting information across sessions. ChatGPT auto-generates user preferences from conversations, Cursor and Windsurf learn coding patterns and project context, and Claude Code uses CLAUDE.md files as persistent instruction memory. All of these systems treat external storage as the real memory layer, with the context window serving as a temporary workspace.<\/p>\n<\/li>\n<\/ul>\n<h3 style=\"text-align: justify;\">Select: Pull In Only What\u2019s Relevant<\/h3>\n<p style=\"text-align: justify;\">The constraint it addresses is that more context isn\u2019t better, and the model needs the right information rather than all available information.<\/p>\n<p style=\"text-align: justify;\">The most important technique here is Retrieval-Augmented Generation, or RAG. Instead of stuffing all your knowledge into the context window, we store it externally in a searchable database. At query time, retrieve only the chunks most relevant to the current question and inject those into the context, giving the model targeted knowledge without the noise of everything else.<\/p>\n<div class=\"captioned-image-container\">\n<figure><a class=\"image-link image2 is-viewable-img\" target=\"_blank\" href=\"https:\/\/substackcdn.com\/image\/fetch\/%24s_!Bdgk!,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png\" data-component-name=\"Image2ToDOM\"><\/p>\n<div class=\"image2-inset\"><picture><source type=\"image\/webp\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!Bdgk!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!Bdgk!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!Bdgk!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!Bdgk!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png 1456w\" sizes=\"100vw\"><img decoding=\"async\" src=\"https:\/\/bkbc.net\/wp-content\/uploads\/2026\/04\/eadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png\" width=\"1456\" height=\"875\" data-attrs='{\"src\":\"https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/eadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png\",\"srcNoWatermark\":null,\"fullscreen\":null,\"imageSize\":null,\"height\":875,\"width\":1456,\"resizeWidth\":null,\"bytes\":250907,\"alt\":null,\"title\":null,\"type\":\"image\/png\",\"href\":null,\"belowTheFold\":true,\"topImage\":false,\"internalRedirect\":\"https:\/\/blog.bytebytego.com\/i\/193030505?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png\",\"isProcessing\":false,\"align\":null,\"offset\":false}' class=\"sizing-normal\" alt=\"\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!Bdgk!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!Bdgk!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!Bdgk!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!Bdgk!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feadec10f-1ecc-4644-b35c-8627ddc69acb_2526x1518.png 1456w\" sizes=\"auto, 100vw\" loading=\"lazy\"><\/picture>\n<div class=\"image-link-expand\">\n<div class=\"pencraft pc-display-flex pc-gap-8 pc-reset\"><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container restack-image\"><svg role=\"img\" width=\"20\" height=\"20\" viewbox=\"0 0 20 20\" fill=\"none\" stroke-width=\"1.5\" stroke=\"var(--color-fg-primary)\" stroke-linecap=\"round\" stroke-linejoin=\"round\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><g><title><\/title><path d=\"M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882\"><\/path><\/g><\/svg><\/button><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container view-image\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20\" height=\"20\" viewbox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" class=\"lucide lucide-maximize2 lucide-maximize-2\"><polyline points=\"15 3 21 3 21 9\"><\/polyline><polyline points=\"9 21 3 21 3 15\"><\/polyline><line x1=\"21\" x2=\"14\" y1=\"3\" y2=\"10\"><\/line><line x1=\"3\" x2=\"10\" y1=\"21\" y2=\"14\"><\/line><\/svg><\/button><\/div>\n<\/div>\n<\/div>\n<p><\/a><\/figure>\n<\/div>\n<p style=\"text-align: justify;\">Selection also applies to tools. When an agent has dozens of available tools, listing every tool description in every prompt wastes tokens and confuses the model. A better approach is to retrieve only the tool descriptions relevant to the current task.<\/p>\n<p style=\"text-align: justify;\">The critical tradeoff with selection is precision. If the retrieval pulls in documents that are almost relevant but not quite, they become distractors that add tokens and push important context into low-attention zones. The retrieval step itself has to be good, or the whole strategy backfires.<\/p>\n<h3 style=\"text-align: justify;\">Compress: Keep Only What You Need<\/h3>\n<p style=\"text-align: justify;\">The constraint it addresses is the context rot and the escalating cost of attention across more tokens.<\/p>\n<p style=\"text-align: justify;\">As agent workflows span dozens or hundreds of steps, the context window fills up with accumulated conversation history and tool outputs. Compression strategies reduce this bulk while trying to preserve the essential information.<\/p>\n<p style=\"text-align: justify;\">Conversation summarization is the most common approach. Claude Code, for instance, triggers an \u201cauto-compact\u201d process when the context hits 95% capacity, summarizing the entire interaction history into a shorter form. Cognition, the company behind the Devin coding agent,<a href=\"https:\/\/blog.langchain.com\/context-engineering-for-agents\/\"> trained a separate, dedicated model specifically for summarization<\/a> at agent-to-agent boundaries. The fact that they built a separate model just for this step tells us how consequential bad compression can be, since a specific decision or detail that gets summarized away is gone permanently.<\/p>\n<p style=\"text-align: justify;\">Simpler forms of compression include trimming (removing older messages from the history) and tool output compression (reducing verbose search results or code outputs to their essentials before they enter the context).<\/p>\n<h3 style=\"text-align: justify;\">Isolate: Split Context Across Agents<\/h3>\n<p style=\"text-align: justify;\">The constraint it addresses is that of attention dilution and context poisoning when too many types of information compete in one window.<\/p>\n<p style=\"text-align: justify;\">Instead of one agent trying to handle everything in a single bloated context window, this strategy splits the work across multiple specialized agents, each with its own clean, focused context. A \u201cresearcher\u201d agent gets a context loaded with search tools and retrieved documents, while a \u201cwriter\u201d agent gets a context loaded with style guides and formatting rules, so neither is distracted by the other\u2019s information.<\/p>\n<p style=\"text-align: justify;\"><a href=\"https:\/\/www.anthropic.com\/engineering\/building-effective-agents\">Anthropic demonstrated this with their multi-agent research system<\/a>, where a lead Opus 4 agent delegated sub-tasks to Sonnet 4 sub-agents. The system achieved a 90.2% improvement over a single Opus 4 agent on research tasks, despite using the same underlying model family. The entire performance gain came from how context was managed, not from a more powerful model.<\/p>\n<p style=\"text-align: justify;\">See the diagram below:<\/p>\n<div class=\"captioned-image-container\">\n<figure><a class=\"image-link image2 is-viewable-img\" target=\"_blank\" href=\"https:\/\/substackcdn.com\/image\/fetch\/%24s_!hrla!,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png\" data-component-name=\"Image2ToDOM\"><\/p>\n<div class=\"image2-inset\"><picture><source type=\"image\/webp\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!hrla!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!hrla!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!hrla!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!hrla!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png 1456w\" sizes=\"100vw\"><img decoding=\"async\" src=\"https:\/\/bkbc.net\/wp-content\/uploads\/2026\/04\/4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png\" width=\"1456\" height=\"862\" data-attrs='{\"src\":\"https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png\",\"srcNoWatermark\":null,\"fullscreen\":null,\"imageSize\":null,\"height\":862,\"width\":1456,\"resizeWidth\":null,\"bytes\":326859,\"alt\":null,\"title\":null,\"type\":\"image\/png\",\"href\":null,\"belowTheFold\":true,\"topImage\":false,\"internalRedirect\":\"https:\/\/blog.bytebytego.com\/i\/193030505?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png\",\"isProcessing\":false,\"align\":null,\"offset\":false}' class=\"sizing-normal\" alt=\"\" srcset=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!hrla!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png 424w, https:\/\/substackcdn.com\/image\/fetch\/$s_!hrla!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png 848w, https:\/\/substackcdn.com\/image\/fetch\/$s_!hrla!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png 1272w, https:\/\/substackcdn.com\/image\/fetch\/$s_!hrla!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ce09eed-d88b-456b-a7c3-3ee5929a6067_4002x2368.png 1456w\" sizes=\"auto, 100vw\" loading=\"lazy\"><\/picture>\n<div class=\"image-link-expand\">\n<div class=\"pencraft pc-display-flex pc-gap-8 pc-reset\"><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container restack-image\"><svg role=\"img\" width=\"20\" height=\"20\" viewbox=\"0 0 20 20\" fill=\"none\" stroke-width=\"1.5\" stroke=\"var(--color-fg-primary)\" stroke-linecap=\"round\" stroke-linejoin=\"round\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><g><title><\/title><path d=\"M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882\"><\/path><\/g><\/svg><\/button><button tabindex=\"0\" type=\"button\" class=\"pencraft pc-reset pencraft icon-container view-image\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20\" height=\"20\" viewbox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" class=\"lucide lucide-maximize2 lucide-maximize-2\"><polyline points=\"15 3 21 3 21 9\"><\/polyline><polyline points=\"9 21 3 21 3 15\"><\/polyline><line x1=\"21\" x2=\"14\" y1=\"3\" y2=\"10\"><\/line><line x1=\"3\" x2=\"10\" y1=\"21\" y2=\"14\"><\/line><\/svg><\/button><\/div>\n<\/div>\n<\/div>\n<p><\/a><\/figure>\n<\/div>\n<h2 style=\"text-align: justify;\">Tradeoffs<\/h2>\n<p style=\"text-align: justify;\">These strategies are powerful, but they involve trade-offs with no universal right answers:<\/p>\n<ul>\n<li>\n<p style=\"text-align: justify;\"><strong>Compression versus information loss:<\/strong> Every time you summarize, you risk losing a detail that turns out to matter later. The more aggressively you compress, the more you save on tokens, but the higher the chance of permanently destroying something important.<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\"><strong>Single agent versus multi-agent:<\/strong> Anthropic\u2019s multi-agent results are impressive, but others,<a href=\"https:\/\/blog.langchain.com\/context-engineering-for-agents\/\"> notably Cognition<\/a>, have argued that a single agent with good compression delivers more stability and lower cost. Both sides are debating the same core question of how to manage context effectively, and the answer depends on task complexity, cost tolerance, and reliability requirements.<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\"><strong>Retrieval precision versus noise:<\/strong> RAG adds knowledge, but imprecise retrieval adds distractors. If the documents you retrieve aren\u2019t genuinely relevant, they consume tokens and push important content into low-attention positions, so the retrieval system itself has to be well-engineered, or RAG makes things worse.<\/p>\n<\/li>\n<li>\n<p style=\"text-align: justify;\"><strong>Cost versus richness:<\/strong> Every token costs money and processing time. The disproportionate scaling of attention means longer contexts get expensive fast, and context engineering is partly an economics problem of figuring out where the return on additional tokens stops being worth the cost.<\/p>\n<\/li>\n<\/ul>\n<h2 style=\"text-align: justify;\">Conclusion<\/h2>\n<p style=\"text-align: justify;\">The core takeaway is that the model is only as good as the context it receives. Working with LLMs effectively requires thinking about the entire system around the model, not just the model itself.<\/p>\n<p style=\"text-align: justify;\">As models get more powerful, context engineering becomes more important. When the model is capable enough, most failures stop being intelligence failures and start being context failures, where the model could have gotten it right but didn\u2019t have what it needed or had too much of what it didn\u2019t need.<\/p>\n<p style=\"text-align: justify;\">The strategies are evolving, and best practices are being revised as new models ship. However, the underlying constraints of finite attention, positional bias, and statelessness are architectural.<\/p>\n<p><strong>References<\/strong><\/p>\n<ul>\n<li>\n<p><a href=\"https:\/\/arxiv.org\/abs\/1706.03762\">Attention Is All You Need<\/a><\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/cs.stanford.edu\/~nfliu\/papers\/lost-in-the-middle.arxiv2023.pdf\">Lost in the Middle: How Language Models Use Long Contexts<\/a><\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/research.trychroma.com\/context-rot\">Context Rot: How Increasing Input Tokens Impacts LLM Performance<\/a><\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/arxiv.org\/abs\/2005.11401\">Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks<\/a><\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/blog.langchain.com\/context-engineering-for-agents\/\">Context Engineering for Agents<\/a><\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/www.anthropic.com\/engineering\/building-effective-agents\">\u201cBuilding Effective Agents<\/a><\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/x.com\/karpathy\/status\/1937902205765607626\">Post on context engineering<\/a> by Andrej Karpathy<\/p>\n<\/li>\n<\/ul>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<div>In this article, we\u2019ll look at how LLMs actually process the information you give them, what context engineering is, and the strategies that can help with it.<\/div>\n","protected":false},"author":1,"featured_media":189,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30,6],"tags":[113,115,112,114],"class_list":["post-188","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-to-head-2-head-comparison","category-technology","tag-architecture","tag-diagrams","tag-software","tag-system-design"],"_links":{"self":[{"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/posts\/188","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/comments?post=188"}],"version-history":[{"count":0,"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/posts\/188\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/media\/189"}],"wp:attachment":[{"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/media?parent=188"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/categories?post=188"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bkbc.net\/index.php\/wp-json\/wp\/v2\/tags?post=188"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}