AI & ML · Nov 14
Apple’s Weighted State Souping says cache states, not just stretch context
A new Apple Machine Learning Research paper points builders toward reusable SSM states for retrieval, tools, and memory.
- Cache reusable state encodings for documents, tools, and memory instead of relying only on larger context windows.
- Use query-dependent composition as the key idea, since equal blending can bury the relevant context.
- Watch for independent tests of the sparse variant before betting production systems on the efficiency claims.