If you do not keep the parts distinct, they collapse into a single blob: everything is “the assistant said it,” and nothing is a durable artifact. The sections below are a calm way to split the work so you can name what you are storing and why.
Memory is the layer of facts, preferences, and decisions you expect to care about more than once. A good memory entry is small enough to reread in a minute and specific enough to act on. It should be traceable—a link to a ticket, a commit, a doc, a dated log entry—so the entry is not only a vibe from last Tuesday. If you cannot point to a source, treat the entry as a hypothesis until you can.
Skills, playbooks, and runbooks are procedures that worked in the real environment, not idealized checklists. When you write one, include the exact sequence you would hand to a careful colleague: prerequisites, the commands or UI steps, the checks that mean “success,” and a short note on when not to use this path (for example, when it would trample a production setting). A skill is not “AI wisdom”; it is repeatable work you have already paid for once with attention.
Sessions are the record of what you tried and what changed. The useful form is not a dump of the whole conversation, but a searchable outcome: what was the task, what did you change in the repo or the system, what failed, and what you would do differently. Future you, and any agent, can scan outcomes far faster than they can re-derive context from a hundred turns of back-and-forth. When something ships, the outcome should point to concrete artifacts: a branch, a diff, a test run, a PR, a URL, a log line.
The workspace is where truth lives in files: branches, config, test output, build artifacts, dashboards. A second brain that never touches the workspace is just notes. A healthy setup makes the path from “we decided this” to “it is in the tree or on the service” short and visible. If the only record of a change is a message in a thread, you have not really stored it; you have buried it in noise.
Runtime is the machinery that runs models, tools, cron jobs, subagents, and gateways. It is easy to treat remote access to your machines or repos as a convenience; it is better to treat it as a security design problem. Anything that can mutate files, run shells, or reach your network from outside should have an obvious audit surface: what ran, on whose behalf, with what permissions, and what the next human should verify before merging or deploying. Hidden authority is the failure mode where things change faster than the notes can keep up.
Memory bloat happens when you save every intermediate thought but never compress it into a small durable fact or decision. The fix is to prefer short pages and occasional deletion over infinite append-only capture.
Hidden authority happens when jobs or agents change systems without leaving a visible state change you can review. The fix is to make activity and approvals inspectable, not to ban automation.
Context laundering happens when private or operational details drift into “general” pages because it was fast to paste them there. The fix is a one-line privacy or scope label at the top of mixed pages, and a habit of moving sensitive detail to a restricted place.
Dashboard sprawl happens when you add another pane every time you feel busy, but never answer “what must I look at today?” The fix is fewer surfaces with clearer meaning, not more tiles.
If you do nothing else, keep public-source material and private or operational material in separate places or with explicit headers. When you promote something from “rough note” to “we rely on this,” add one link or pointer that answers why you believe it. The next person—human or software—can then choose to trust, verify, or replace that claim on purpose, instead of guessing from tone.
]]>First, describe the experiment in the spec file. The spec should name the command that performs evaluation—often a script or a small program—and the list of mutable paths the experiment is allowed to touch, alongside protected paths that must not change. If a run reaches outside its allowed set, the harness should fail early with a clear message, not halfway through a silent edit.
Second, run the eval with a run id and capture structured metrics. The evaluation command is responsible for writing JSON (or a format your tool understands) to a path the runner provides, for example through an environment variable such as AUTORESEARCH_METRICS_FILE. The spec names which metric field matters, and whether higher or lower is better. That is how “better” becomes a file on disk, not a feeling in the console.
Third, keep detailed run output local. A typical choice is a directory like .autoresearch/runs/<run-id>/ containing per-attempt data, optional stdout summaries, and anything else you need to debug. Treat this tree like developer scratch space: it may contain local paths, prompts, or half-formed ideas, so default policy is to keep it out of version control unless you have a defined export and redaction path.
Fourth, use CI to validate the harness, not to auto-merge winners. A good CI job checks that specs validate, that tests pass, and that a known example can run from a clean checkout. It does not need to open pull requests for you. Candidate improvements still flow through branches and review like any other change.
In an [experiment] section, eval_command is the list of program and arguments that must run to completion. The metric and direction fields tell the runner which number in the metrics JSON matters and which direction is “improvement.” The mutable_paths and protected_paths fields define the contract between “this experiment is allowed to edit the repo in these places” and “these areas are off limits.”
timeout_seconds and repeats exist so that a hung process or a flaky environment shows up in the summary the same way a bad score does, instead of leaving you to infer failure from a silent shell. The point is to make the failure modes boring to compare across machines, not to look dramatic in the terminal.
A useful harness treats a crash, a timeout, malformed metrics JSON, and a missing metrics file as named states, not as a single generic “it failed.” When those states are first-class, you can sort runs the way you would sort test results: you know whether you are looking at a logic bug, a network flake, a permissions mistake, or a true regression in the metric. Soft failures that collapse into one exit code are hard to learn from; explicit states are the difference between a one-off and a pattern.
Assume run logs and scratch directories can contain prompts, absolute paths, environment names, and ideas you are not ready to share. The calm default is: keep .autoresearch/ (or your equivalent) in .gitignore, and if you need to show a result to someone else, copy out a reduced summary you are willing to stand behind. The workflow is the same as any other local diagnostic output: the machine is allowed to be messy; the repository should stay legible to strangers.
The commands and field names in your own project may differ; keep the same separation between spec, metrics, and raw logs.
]]>A strong claim—about architecture, about an outage, about a policy—ought to sit next to a source a reader can open. That source might be a file path in your sources/ tree, a link to a ticket, a public URL, a commit hash, or a dated private note in a place that is allowed to exist. The question you are answering is not “is this true forever?” The question is “why did we think this, on what day, and from what ground?”
When you cannot verify a statement quickly in a maintenance pass, the calm move is to mark the page as stale, or to add a short “Open questions” section that says what is uncertain. Two pages that both read as the current truth but disagree quietly are more dangerous than one page that admits a gap. An honest unknown gives the next reader permission to re-check; two silent “current” statements create accidents.
If a page might combine broad, shareable explanation with specific hostnames, internal URLs, family context, or credentials—even in mild form—say so at the top in a few sentences. A Privacy note does not have to be legal language. It is enough to state the kinds of information the page may contain, who it is meant for, and whether it is safe to copy into a public issue or a client-facing document.
Pay attention to context laundering, which is a slow form of over-sharing, not a dramatic leak. A small operational fact slips into a “general” page because you were in a hurry one evening. A month later, that page is the one everyone links. The fact has become ambient truth in the wrong place. The defense is a light habit: when you add detail that is not in scope for the page’s original audience, move it to a restricted note or add scope right then, not “later when we clean up.”
Large annual reviews rarely happen on schedule, and they are painful when they do. A more sustainable rhythm is a rotating scope: one project page, one pattern, or one source map in a short session, on whatever cadence you can keep—weekly, biweekly, or monthly. The pass is the same in spirit: read for accuracy, add links, soften claims you cannot support, and align contradictions. Small passes that finish compound; sweeping passes that stall do not.
Keep a single maintenance log, changelog, or append-only file where you record what you touched, what you changed, and what you postponed. The log entry does not need to be long. It should be enough that someone (including you) can tell when a page was last intentionally read, and what you decided to leave for another day. The goal is not perfect documentation. The goal is reversible, inspectable, and honest state, so the next person—human or agent—does not have to reconstruct reality from a chat transcript or a hope that a timestamp was right.
When traceability and scope are in good shape, a mixed knowledge base stops feeling like a junk drawer. It feels like a workshop: tools are where you expect, sharp edges are labeled, and the next action is clear enough that you are not always starting from zero. That outcome is not flashy, but it is the kind of calm infrastructure that makes hard work a little more predictable over months and years.
]]>Durable process containers are the bedrock. In many setups that means tmux sessions, windows, or panes, because they survive SSH drops, let you step away, and let you reattach with context intact. A worker is not a fragile foreground shell you must keep open; it is a named place where a process is expected to live for a while.
Isolation means one worker, one home in that substrate. If two agents share one directory and one git state without rules, you will get accidental interference. A common step up is a separate worktree or workspace per task so that branches, uncommitted work, and build artifacts do not tangle. You are not trying to be fancy; you are trying to make “who owns this tree” obvious.
Coordination is how you avoid plan drift when multiple sessions run at once. A task board or a shared message channel—lightweight, even a text file with locks—is where “claimed,” “in progress,” and “blocked” should live, so it does not exist only in one transcript. The human operator, and any orchestrator, should be able to answer “what is running?” without opening every session.
Observability is the ability to see health without N separate attach sequences. A simple dashboard or a tail on structured logs, even if it is minimal, is enough to know which workers are still alive, which are stuck, and which finished with a non-zero exit. The goal is to reduce surprise when you return after an hour.
Recovery is the honest admission that processes crash, networks flap, and agents loop. A watchdog or a periodic health check can restart a worker, but the dangerous version is a blind restart that replays a destructive or stale action. Good recovery is tied to task state you trust: a checkpoint, a branch name, a commit hash, an explicit “do not re-run if already done” flag. The operator should be able to read why a worker restarted, not just that it did.
If you are making a small, safe edit in a repo you know well, you do not need a multiplexing stack. The overhead pays for itself when parallelism really reduces wall-clock time, when resumability matters because sessions are long, and when you need visibility because more than one person or system is touching the work. In those cases, a little ceremony up front is cheaper than hours lost to “which terminal was that?” and half-merged state.
Drift is when the task board says done but the git worktree is dirty, or the branch never merged, or the fix only lived in a local patch. The remedy is a tight definition of done that points at the tree: pushed branch, clean status, or an explicit “blocked on X” with a name.
Log leakage is a quiet problem. Shared boards, shared log files, and shared HTTP endpoints for status can end up with local paths, prompt text, or secrets if the tools are chatty. Treat those surfaces the way you would treat app logs: scope them, redact by default, and do not let long-lived storage fill with content nobody owns.
Dumb retries are automatic restarts that run the same bad command again because the task record never advanced. The remedy is to connect recovery to sanitized, explicit state, not to hope the second attempt is luckier. If a step is destructive, the task record should make that step idempotent or gated by a human.
You do not need a particular brand of tool to apply the pattern. You need a task contract that everyone—the operator, the agents, and the future reader of the logs—can follow without improvising. Once that exists, the control plane is simply the place where those contracts are visible.
The names in your own stack (tmux, custom TUIs, APIs) will differ; the structure above is the part that carries between setups.
]]>Ingestion is where you park raw or semi-raw input. A dedicated sources/ tree (or whatever naming you prefer) is enough. When you add something, name its scope: was it copied from a public document, a ticket, a private note, a conversation, or a command transcript? If you can add a date, add it. You are not trying to be formal; you are trying to make future you honest about where a claim entered the system.
Synthesis is where you turn that material into pages that are meant to be read: project notes, patterns, runbooks, decisions. A synthesis page is allowed to be short, but it should not present a strong or operational claim as if it were obvious unless that claim is tied to something a reader can follow: a path under sources/, a public URL, or a private note you label explicitly. The habit you are building is traceability, not perfect prose.
Linting is a scheduled pass. Weekly is a reasonable default; the duration can be as little as fifteen minutes if you keep the scope small. A lint is not a rewrite. Treat it the way you would a code review: you read, you add status labels, you fix or soften broken links, you flag contradictions, and you check that public-facing pages are not quietly accumulating private detail. You are not trying to finish the vault in one sitting.
Logging is the final step, and it matters more than it sounds. In a line or a short paragraph, record what you opened, what you changed, and what you explicitly postponed. If you deferred something, add a single line explaining why—no shame, no essay. The log is your audit trail; it is how the system stays accountable to itself.
When a page sounds authoritative, a reader should be able to see why you believe it. If the page cannot point to a source, either soften the language, mark the claim as uncertain, or add the missing reference.
For privacy, a page that is meant to be broadly shareable should not slowly absorb home-lab hostnames, internal URLs, or one-off tokens just because it was faster to paste them there. If the page is mixed, say so in a short note at the top: what kind of information it may contain, and for whom it is written.
For contradictions, do not let two “current” pages give different default truths about the same system. You can update the one that is wrong, you can narrow each page so the scopes no longer overlap, or you can write an “Open questions” block that names the conflict and points to both pages. Ambiguity in the open is better than a silent war between documents.
For decay, anything that described “how things are right now” a year ago should either be dated, marked stale, or updated. Time-sensitive “current state” is the kind of content that causes the most damage when it is never revisited, because it still reads with full confidence.
When you are short on time, pick exactly one of the following: a project page you will actually use this week, a pattern you rely on when making decisions, or a source map you suspect is going stale. At the top of the page, set a status line if you do not have one: Status: first-pass for a shallow or unverified first draft, Status: current for something you still endorse after this pass, and Status: stale for anything you would not act on without re-checking. Add one entry to your maintenance log. A narrow pass you finish beats a wide pass you never start.
The failure mode to avoid is ingestion without digestion: a vault that only grows, where first-pass pages sit silently in search results as if they were reviewed truth. A weekly or biweekly lint is the opposite of a dramatic overhaul. It is a small, boring habit that keeps the system compounding, the same way short, regular review keeps a code base from rotting in silence.
]]>The first step is to create the connect server; I’m guessing that this is basically an API gateway to your stuff in their cloud.
❯ op connect server create Kubernetes --vaults Kubernetes,Homelab --cache
Set up a Connect server.
UUID: REDACTED
Credentials file: /path/to/where/you/ran/this/1password-credentials.json
If you want to see your servers:
❯ op connect server list
ID NAME STATE
REDACTED Kubernetes ACTIVE
The next step is to create an access token for Kubernetes, I’m intentially giving it a short expiration for this example.
❯ export CONNECT_TOKEN="$(op connect token create Kubernetes --server Kubernetes --vault Kubernetes,rw --vault Homelab,r --expires-in=24h)"
Next, set up the Helm things:
❯ helm repo add 1password https://1password.github.io/connect-helm-charts/
❯ helm repo update
Lastly, deploy Connect and its operator, I’m keeping everything in the 1password namespace for now.
Be mindful of the path to 1password-credentials.json.
The end result of all of this is that you get syncing (and caching) of your 1Password secrets to your Kubernetes secrets.
❯ helm install connect 1password/connect \
--namespace "1password" --create-namespace \
--set-file connect.credentials="1password-credentials.json" \
--set operator.create="true" \
--set operator.token.value="${CONNECT_TOKEN}" \
--set operator.watchNamespace="{1password}"
I’ve created a test secret in 1Password:
❯ op item get --vault Kubernetes test1
ID: REDACTED
Title: test1
Vault: Kubernetes (REDACTED)
Created: 2 hours ago
Updated: 2 hours ago by Mark Mercado
Favorite: false
Version: 1
Category: LOGIN
Fields:
username: foo
password: bar
Let’s define this secret in Kubernetes, note the itemPath structure (the vaults and item path segments are fixed).
Remember that I’ve got the operator configured to only watch the 1password namespace right now.
❯ cat secrets.yaml
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: test1
namespace: 1password
spec:
itemPath: vaults/Kubernetes/items/test1
Go ahead an deploy this manifest with kubectl apply -f secrets.yaml.
You should see something like this:
❯ kubectl -n 1password get onepassworditems test1
NAME AGE
test1 10s
And then the corresponding secret:
❯ kubectl -n 1password get secret test1
NAME TYPE DATA AGE
test1 Opaque 2 45s
And to make sure it’s working (recall the secret above has two fields, username is foo and password is bar):
❯ kubectl -n 1password get secret test1 -o json | jq .data
{
"password": "YmFy",
"username": "Zm9v"
}
❯ echo Zm9v | base64 -d
foo
❯ echo YmFy | base64 -d
bar
Sweet, looks good.
]]>eval $(ssh-agent)
install -b -m 600 /dev/null ssh.key
echo "${SSH_PRIVATE_KEY}" >ssh.key
echo "${SSH_PASSPHRASE}" \
| SSH_ASKPASS=cat setsid -c ssh-add ssh.key
ssh-add -l
Hopefully your CI will redact the secrets.
As you can see, you’ll need ssh-agent, install, cat, setsid, and, ssh-add.
As far as the secrets go, your private key should be stored in the environment as SSH_PRIVATE_KEY and the passphrase as SSH_PASSPHRASE.
6.0.45 and the blurb says that 6.4.54 is out.
Here’s a list of what I’m currently running:
My configuration is pretty boring, I currently only have two VLANs, the native one and one for the open guest network. I’ll get around to adding more some day (I’m looking at you, IoT). As far as WAN goes, I have two ISPs, Xfinity Cable and AT&T ADSL. Unfortunately, I don’t have fiber options right now. My partner and I both work in tech, and, our kids are on the internet as well. So, having good uptime is nice. A few years ago, my Xfinity wasn’t as reliable as it is today (turns out that squirrels are a thing), which is when I added the AT&T service. Once a new cable line was run those problems went away.
As far as utilization goes, up until a few days ago, I was running the DSL as failover-only. So, we’d automatically failover ot AT&T when Xfinity dropped, and, failback when it came back. This worked out just fine, and the option to set this up is available in the UI. The other option is weighted load-balancing, which I don’t want – I don’t want my connections alternating, and, the DSL that I have is much slower than my cable.
Unfortunately, doing policy-based routing isn’t available in the UI.
Since I’m paying for DSL, I wanted to get a bit more utilization out of it, but, on my terms.
There are lots of ways that you can go with this, but what I’ve done thus far is route the guest VLAN through it (and, I have a static route set up for something that I’m not going to go into :smile:).
Static routes can be configured in the UI, but, IPs aren’t always stable; and, again, I don’t want my connections flipping between my providers like this generally.
So, to the UniFI CLI we go (EdgeOS is based on Vyatta).
On the USG, you can show your interfaces like this:
mamercad@gateway:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description
--------- ---------- --- -----------
eth0 68.46.132.66/23 u/u WAN
eth1 192.168.1.1/24 u/u LAN
eth1.15 192.168.15.1/24 u/u
eth2 162.230.47.152/23 u/u WAN2
lo 127.0.0.1/8 u/u
::1/128
For me, WAN is Xfinity and WAN2 is AT&T.
You can show your failover configuration like this:
mamercad@gateway:~$ show load-balance status
Group wan2_failover
interface : eth0
carrier : up
status : failover
gateway : 68.46.132.1
route table : 201
weight : 0%
flows
WAN Out : 0
WAN In : 21726
Local Out : 0
interface : eth2
carrier : up
status : active
gateway : 162.230.46.1
route table : 202
weight : 100%
flows
WAN Out : 640
WAN In : 0
Local Out : 1122
Group wan_failover
interface : eth0
carrier : up
status : active
gateway : 68.46.132.1
route table : 203
weight : 100%
flows
WAN Out : 546000
WAN In : 0
Local Out : 0
interface : eth2
carrier : up
status : failover
gateway : 162.230.46.1
route table : 204
weight : 0%
flows
WAN Out : 3
WAN In : 0
Local Out : 0
The configuration that I ended up going with looks like this:
configure
set load-balance group wan2_failover interface eth0 failover-only
set load-balance group wan2_failover interface eth2
set firewall modify LOAD_BALANCE rule 2501 action modify
set firewall modify LOAD_BALANCE rule 2501 modify lb-group wan2_failover
set firewall modify LOAD_BALANCE rule 2501 source address 192.168.15.0/24
commit ; exit
My guest VLAN is 15, and, the L3 for it is 192.168.15.0/24 (this shows up as eth1.15 in show interfaces).
As you can see, anything originating from this network will be shipped over WAN2 (AT&T).
This configuration works just fine, but, if and when the USG restarts, it’ll be lost.
We need to get this onto the CloudKey, and for that, it’ll take a custom config.gateway.json.
This part was relatively straightforward – basically, I dumped and diffed.
In config.gateway.json, you should only supply the differences (unless you’re okay with not using the UI to manage the USG).
As with anything, it’s always a good idea to take a backup before making changes.
On the USG, before doing anything, I did a cp /config/config.boot{,.bak}.
You can dump your config like this mca-ctrl -t dump-cfg >config.txt 2>&1.
Then, I ran the configuration stuff above, and then, diffed them to figure out how to craft config.gateway.json.
In the end, what I ended up with is this:
{
"firewall": {
"ip-src-route": "disable",
"ipv6-receive-redirects": "disable",
"ipv6-src-route": "disable",
"log-martians": "enable",
"modify": {
"LOAD_BALANCE": {
"rule": {
"2501": {
"action": "modify",
"modify": {
"lb-group": "wan2_failover"
},
"source": {
"address": "192.168.15.0/24"
}
}
}
}
},
"source-validation": "disable"
},
"interfaces": {
"ethernet": {
"eth0": {
"dhcp-options": {
"default-route": "update"
},
"duplex": "auto",
"speed": "auto"
},
"eth1": {
"duplex": "auto",
"speed": "auto"
},
"eth2": {
"dhcp-options": {
"default-route": "update"
},
"duplex": "auto",
"speed": "auto"
}
}
},
"load-balance": {
"group": {
"wan2_failover": {
"flush-on-active": "disable",
"interface": {
"eth0": {
"failover-only": "''"
},
"eth2": "''"
},
"lb-local": "enable",
"lb-local-metric-change": "enable"
},
"wan_failover": {
"flush-on-active": "disable",
"lb-local": "enable",
"lb-local-metric-change": "enable"
}
}
}
}
After constructing this file, I dropped it onto my CloudKey.
I only have a single site, and a Gen1 CloudKey, so, it lives at /srv/unifi/data/sites/default/config.gateway.json.
A little while after, the USG rebooted to (re)provision.
It only took me a few tries to get it right :smile:.
I’ll do more than primarily routing my guest network through it in time.
Adding more VLANs should be as simple as adding sources to the wan2_failover configuration.
Hope this helps the next person!
$lastjob, we were a Jenkins shop, and, I’m not sure I miss it all that much :smile:. When it comes down to usage, I’ll take declarative YAML over imperative Groovy every day of the week, including Sunday.
Alright, let’s roll. Concourse is built with containers in mind, so, we’ll need something like Docker Desktop. The container runtime shouldn’t matter all that much, I just happened to be using my Mac at the time. When I first started writing the Concourse resource, I was pushing to quay.io, which is fine, but, when you’re trying to iterate quickly, becomes a bottleneck in a hurry. I’d never written a Concourse resource in the past, so, it took me lots of tries to get things working. There’s nothing fast about pushing and pulling ~ gigabyte between a 5-year-old laptop and Quay over and over again. It didn’t take me long to throw that out and set everything up locally.
Setting up Concourse locally is relatively painless, they provide a compose file. The only change (addition) I’ve made is adding a local Docker registry to keep my images pushing and pulling local.
# existing Concourse services here
registry:
image: registry:2
ports: ["5000:5000"]
volumes:
- ${PWD}/server.crt:/server.crt
- ${PWD}/server.key:/server.key
environment:
REGISTRY_HTTP_ADDR: 0.0.0.0:5000
REGISTRY_HTTP_TLS_CERTIFICATE: server.crt
REGISTRY_HTTP_TLS_KEY: server.key
Here’s a one-liner for generating “snake oil” crypto:
❯ openssl req -new -newkey rsa:4096 -days 3650 -nodes -x509 \
-subj "/C=${COUNTRY}/ST=${STATE}/L=${CITY}/O=${ORG})/CN=${FQDN}" \
-keyout server.key -out server.crt
So, when it comes to tagging your images, they’d look something like this registry:5000/my-image-name:0.0.1. Note that registry lines up with the service name (the existing Concourse services in the “pod” can resolve this name), the 5000 is obviously the chosen port, my-image-name is an arbitrary image name, and 0.0.1 is an arbitrary tag. Now, when you docker push ... you’ll be pushing to the Docker registry running locally.
Closing the loop, we’ll need to also pull from our local Docker registry, here’s an example of how our resource_type would look:
resource_types:
- name: our-example-image
type: docker-image
source:
repository: registry:5000/my-image-name
version: 0.0.1
Hopefully this’ll help speed things up for you as they did for me!
]]>