Close Menu
    Facebook X (Twitter) Instagram
    • Privacy Policy
    • Terms Of Service
    • Social Media Disclaimer
    • DMCA Compliance
    • Anti-Spam Policy
    Facebook X (Twitter) Instagram
    Fintech Fetch
    • Home
    • Crypto News
      • Bitcoin
      • Ethereum
      • Altcoins
      • Blockchain
      • DeFi
    • AI News
    • Stock News
    • Learn
      • AI for Beginners
      • AI Tips
      • Make Money with AI
    • Reviews
    • Tools
      • Best AI Tools
      • Crypto Market Cap List
      • Stock Market Overview
      • Market Heatmap
    • Contact
    Fintech Fetch
    Home»AI News»Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang
    ▶
    AI News

    Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang

    October 28, 20262 Mins Read
    Share
    Facebook Twitter LinkedIn Pinterest Email
    murf
    Tsinghua University · arXiv 2610.28577

    AKV: KV Cache Engineering for LLM Agents

    Agents drop and reposition KV cache instead of rewriting their text history.

    −27.7% avg cost · +1.7 pts avg score




    Context engineering (summarize text)

    Re-prefill work: 0 blocks

    Cache engineering (AKV drop)

    Re-prefill work: 0 blocks

    frase

    Both agents hold a user prompt, assistant turns and tool results. Press the button to remove old tool result A.

    UserAssistantTool resultRecomputedNew turn




    Active KV cache (number = position encoding index)

    Start state: tokens A, B, C sit at positions 0, 1, 2. Step through the 3 operations.

    {"messages": [...], "drop_message": {}, "reposition": []}

    Model
    Benchmark
    Pass rate (%) · higher is better
    Cost (US cents per task) · lower is better

    Concurrency: 14 · GPT-OSS-120B · 80 long-context SWE-bench Verified tasks · 8 A800 GPUs

    P95 request latency (s) · lower is better
    Output throughput (tokens/s) · higher is better
    1.18x

    Output throughput, peak vs peak

    1.53x

    Input throughput, peak vs peak

    1.51x

    Faster P95 latency, peak vs peak

    2.849ms

    Full attention per decode step (was 7.082)

    • Few supported model familiesDrop and Repos are enabled for Qwen, GPT-OSS and MiniMax-M2 text architectures. Linear attention models are future work.
    • Self-hosted onlyYou run the AKV server yourself. It extends SGLang with cache-operation support.
    • Self-reported, single hardware setupAll results come from the authors, on 8 A800-80GB GPUs in BF16.
    • Different measurement windowsHigh-concurrency AKV runs were measured through the first 30 task completions. Standard used a 1-hour window.
    • Not every benchmark improvesMiniMax-M2.7 on BrowseComp-Plus scores 58.9 with AKV, below Text Engineering at 62.3.
    • Dropping reasoning cache hurtsOn SWE-bench Verified, dropping reasoning cache fell to 36.5, and dropping both to 34.5, versus 41.0 for tool drop.

    Tap a row to expand it.

    ledger
    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email
    Fintech Fetch Editorial Team
    • Website

    Related Posts

    DailyAI Briefing: Fed survey names AI top risk, plus 4 more

    DailyAI Briefing: Fed survey names AI top risk, plus 4 more

    November 16, 2026
    Helping robots fit into the human workplace | MIT News

    Helping robots fit into the human workplace | MIT News

    November 15, 2026
    CopilotKit Releases OpenBot 1.0: An MIT-Licensed, Self-Hostable AI Coworker With Built-In Governance

    CopilotKit Releases OpenBot 1.0: An MIT-Licensed, Self-Hostable AI Coworker With Built-In Governance

    November 14, 2026
    DailyAI Briefing: Lawsuit aims to halt new AI models, plus 4 more

    DailyAI Briefing: Lawsuit aims to halt new AI models, plus 4 more

    November 13, 2026
    Add A Comment

    Comments are closed.

    Join our email newsletter and get news & updates into your inbox for free.


    Privacy Policy

    Thanks! We sent confirmation message to your inbox.

    kraken
    Latest Posts
    blockstream bitcoin custody

    Blockstream and Thales Introduce Bitcoin Custody Solution

    November 16, 2026
    I Let AI Hack My TV… It Didn’t Stop at the Ads

    I Let AI Hack My TV… It Didn’t Stop at the Ads

    November 16, 2026
    Cointelegraph

    Bitcoin Drops to Four-Month Low Around $62K as Cryptocurrency Market Declines

    November 15, 2026
    Cointelegraph

    RWA Protocol stUSDT Shuts Down After $2.3B TVL Peak

    November 15, 2026
    Solana Staking Holds Near 69% Even As SOL Price Crashes To $67 On Breakpoint Day

    Solana Staking Remains Around 69% Despite SOL Price Plummeting to $67 on Breakpoint Day

    November 15, 2026
    aistudios
    LEGAL INFORMATION
    • Privacy Policy
    • Terms Of Service
    • Social Media Disclaimer
    • DMCA Compliance
    • Anti-Spam Policy
    Top Insights
    Orange mining cable connects to an OCEAN port beside a Bitcoin stencil and the confirmed block-reward receipt.

    AI-Related Security Breach Hits Bitcoin Mining Sector, Redirecting 13 EH/s of Computing Power to Hackers

    November 16, 2026

    Tesla (TSLA) Stock Forecast: Bulls Must Maintain Momentum Beyond Current Resistance Levels

    November 16, 2026
    aistudios
    Facebook X (Twitter) Instagram Pinterest
    © 2026 FintechFetch.com - All rights reserved.

    Type above and press Enter to search. Press Esc to cancel.