That Safari Website Data bloat issue
One morning I sat down at the computer and — to my surprise — Docker stopped, Mail said it couldn't fetch, and a dozen other errors caused by not enough disk space. I was nonplussed since I knew for sure there had to be hundreds of Gigabytes available.
I set out to investigate and met with a strange anomaly. While df reported only about 2%, du said that about 140GB is used on the 500GB SSD. How was that possible?
There was no single large file that would stand out, so there was no easy solution.
My first idea was Docker, since I didn't limit the disk usage of the containers and readsb + ADSB feeders create copious amounts of logs. But still not hundreds of Gigabytes of them, and also the logging is limited in their configuration. Docker Desktop showed normal disk usage less than 2GB.
Only when recursively trying to list the directories or files that are the culprit did I notice that a lot of directories are skipped. And then it dawned on me: I'm using iTerm2, and even with sudo, it does not have full disk access, as per MacOS policies. So one needs to explicitly go to MacOS settings and grant iTerm2 Full Disk Access in order to be able to scan everything.
After that it was easy to find that one of the directories in ~/Library/Containers/com.apple.Safari/Data/Library/WebKit/WebsiteData/Default was a whopping three-hundred-something GB. Nice for a website to use my SSD as a NAS, but how about no. The follow-up issue here is that for security reasons it isn't easy to tell which website it is, because the directory is just a hash, and the real URL is matched in Cache.db, an SQLite database. But I was curious.
You'll never guess which site it was: Microsoft Teams on the web. Apparently when you use Teams on the web, it basically downloads the whole software to cache, and then leaves it there (?). But it's never re-used, so if you join a different meeting, it does it again, and this goes on and on. I have been on a couple of interview calls lately, many in Teams, and apparently I inadvertently started to fill up my free space with copies of Teams…
After a quick purge, which can even be done in the GUI at Safari → Settings → Privacy → Manage Website Data (or you just delete the directory) I reclaimed my SSD again.
A little side-note as to why I don't use Teams Desktop. I used to, seemed like a more convenient choice, but one day I started playing World of Warcraft … only it had no sound. Whatever I did, I could't fix it, so I started investigating, and apparently the audio driver that Teams installs for whatever reason messes with WoW's sound output and without any workaround one of them had to go. (Take a wild guess.) This is how Teams Desktop and I parted ways.
Anyway, then I thought, if this happened once, it can happen again, I need to monitor this. So with heavy agentic help Miss Agent and I came up with the following bash script to be placed in ~/.zshrc which gives you a command, safari-data, that lists the top fifteen website data directories by size and matches them to their respective URL:
# --------------------------------------------------
# SAFARI DATA INSPECTOR
# --------------------------------------------------
safari-data() {
local base_dir="$HOME/Library/Containers/com.apple.Safari/Data/Library/WebKit/WebsiteData/Default"
local size path hash_name domain mb_size
/usr/bin/du -sk "$base_dir"/* 2>/dev/null | /usr/bin/sort -rn | /usr/bin/head -n 15 | while read -r size path; do
hash_name="${path##*/}"
domain=""
# Fast path: Search shallow SQLite databases and origin descriptors first
domain=$(/usr/bin/find "$path" -maxdepth 3 -type f \( -name "*.sqlite" -o -name "*.db" -o -name "Origin*" \) -exec /usr/bin/strings {} + 2>/dev/null | /usr/bin/grep -E -o 'https?://[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}' | /usr/bin/sed -E 's#https?://##' | /usr/bin/head -n 1)
# Fallback path: Full scan if shallow search misses
if [[ -z "$domain" ]]; then
domain=$(/usr/bin/find "$path" -type f -exec /usr/bin/strings {} + 2>/dev/null | /usr/bin/grep -E -o '([a-zA-Z0-9-]+\.)+(com|org|se|net|io|app|gov|edu|co|uk|de)' | /usr/bin/grep -v -E '(apple|webkit|sqlite|schema|system|localhost|bplist|SQLite)' | /usr/bin/head -n 1)
fi
mb_size=$((size / 1024))
printf "%-8s %-35s %s\n" "${mb_size}MB" "${domain:-Unknown Domain}" "($hash_name)"
done
}
This is the first line of an example output from today when I noticed that the Unifi Console decided it needed nearly 6GB in cache, because reasons:
❯ safari-data
5839MB unifi.ui.com (zerE8ilEaEBfw61aLNeQ0pQtQn1C4PrfjyDriHGiLVc)
Did any of you run into something similar? Apparently Safari's caching has caused issues before.