Hacker News new | past | comments | ask | show | jobs | submit login
Working Set Size Estimation (2018) (brendangregg.com)
30 points by pncnmnp 3 months ago | hide | past | favorite | 2 comments



If anyone is interested in space & time efficient caching and working set tracking check out some of my favorites, gil einziger https://scholar.google.co.il/citations?user=kWivlnsAAAAJ and the ex coho data guys https://www.usenix.net/system/files/conference/osdi14/osdi14....

Ben Manes[1] worked with Gil Einziger on an implementation of TinyLFU and actually contributed to an updated version of the TinyLFU paper, IIRC. Gil has a ton of very relevant work[2][3] around cache admission, control, and information density.

Coho data was working very large storage arrays, specifically optimizing data placement across different media/hosts/etc. lots of similar problems where even the metadata of the working set was prohibitively large. So they had space & time efficient functions to track access patterns in realtime. Their business didnt work out (unfortunately) and most of them ended up at AWS, working on storage like S3 and Glacier.

[1] https://github.com/ben-manes/caffeine/wiki/Efficiency#window... [2] https://scholar.google.com/citations?user=kWivlnsAAAAJ&hl=en [3] https://github.com/gilga1983


The original idea is purposed by Peter Denning[1], but the theoretical WorkingSet reclaiming algorithm is hard to implement in real hardware.

[1]: https://denninginstitute.com/pjd/PUBS/WSModel_1968.pdf




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: