# Activity Watch

[All work](https://sulayman-bowles.dev/work)

I built Activity Watch to answer where my day actually went. It runs entirely on my Mac and labels its own record overnight with on-device models.

ProblemCapturePrivacyModelsJudgment

## Screen Time counts minutes but can’t say what they were for

Screen Time can say how long a browser was open. It can’t say whether that time was research, email, or drifting. Activity Watch joins the signals a Mac and an iPhone already produce into one timeline: the window in front, screenshots of changed screens read by Apple’s on-device text recognition, the iPhone app intervals Apple syncs to the Mac, and when files and commits change in my projects.

I started in September 2026 on the open-source ActivityWatch and Pensieve, then replaced their servers with my own collectors. Nothing is uploaded automatically: the collectors open no network port, and the models run locally with no cloud service.

## Each source records the coarsest level that still answers the question

The keyboard is the clearest case: the record can say how long I typed in an app, never what I typed.

**What Activity Watch records and what it leaves out**

| Source | How often | Limit |
| --- | --- | --- |
| App and window in front | Every 5 seconds | Locked, idle, and unidentified time is labeled, not guessed |
| Screenshots, read with Apple Vision text recognition | Right after a window change, then at most one changed screen every 20 seconds plus one every 2 minutes | Sensitive windows and locked sessions are skipped |
| Keyboard and pointer | Per minute, per app | Counts, timing, and shortcut names; which letter or number I pressed is never stored |
| iPhone app use | Hourly | Only the intervals Apple already syncs to the Mac |
| Projects | Each file change; commits daily | File paths and times, commit IDs and times; no commit messages or diffs |

## The collectors can’t reach the network

The archive is the most sensitive thing on the machine: screenshots, recognized text, app names, and phone use. So capture, sampling, text recognition, and the phone reader run under a macOS sandbox profile that denies every network operation. None of the collectors opens an HTTP port, and every database and image file is readable only by my user account.

The models get the same treatment. They run with network access denied, keep only the final caption instead of their reasoning, and read screenshot text as untrusted input with no tools to call. Text on a web page can still influence a caption, but the models have no tools to act on it.

## Only 5% of frames had a caption, so I rebuilt the overnight run

By November 18, only 1,804 of 33,112 screenshots with recognized text (5%) had a caption from the 9-billion-parameter model. Each frame started a new process and reloaded the model, and its step-by-step reasoning spent up to 2,048 tokens, roughly 1,500 words, on thinking it then threw away: about 94 seconds for a one-sentence caption.

Version 2, shipped November 18, loads Qwen3.5-9B once a night behind a local server, starts with four requests at a time (the first night’s calibration can change that), turns reasoning off and asks for a short JSON answer of about 40 tokens. It labels the first frame of every app in every 15-minute slot before it labels a second one, so a short night still covers the whole day. A frame that matches the last labeled frame of the same window inherits its label.

Two shortcuts stay off until they prove themselves: labeling text-heavy frames from the recognized text alone, and a 4B model that takes the easy frames before the 9B. Each turns on only if, on a calibration set of real frames, it agrees with the 9B and runs faster.

## A gap is unknown, not idle

The dashboard draws a gap as a gap. A window in front isn’t proof of attention, files and commits can come from automation, and phone and Mac totals overlap, so they’re never added together. Model labels sit beside what was observed and count as outcomes only after I confirm them.

Method & limits: this is a private application for Apple silicon Macs, and it covers digital activity only. On my Mac it’s called Your Time. The recorder’s source and data aren’t public; the review app, which lists a day in 15-minute blocks and takes my corrections, is public with a synthetic demo. Version 2’s overnight speed isn’t measured yet; its first runs record a calibration against the old path.

- [YourTime Review on GitHub](https://github.com/SulaymanB2024/yourtime-review)

[All work](https://sulayman-bowles.dev/work)[Continue with Chegg](https://sulayman-bowles.dev/work/chegg)

---
Source: https://sulayman-bowles.dev/work/activity-watch · Sulayman Bowles
