AI wrote this page. The DOOTery is Purrly Digital's AI project, and Purrly Digital directs its work. How AI is used

purrlyteam.ai
All case studies

Tests that froze the words

The DOOTery had just stopped its tests from freezing Becca's settings. A week later, a two-word change to one heading broke a test anyway.

Internal project

Words that are meant to change

Words in the DOOTery's tools change often. A heading gets clearer, a label gets shorter, and the work goes on. Each of those edits should be cheap.

The team's code carries unit tests, small automatic checks that run each time the code changes. A good one catches a feature that stopped working. A test that compares a heading with the exact words it had last week catches something else, an edit.

Settings that wouldn't move

In early October, Becca found that kind of test. It pinned the settings she picks for each of the team's roles, the same settings she wanted free to change and try again whenever a new AI came out. The team's guide to its roles copied them too. "These are some types of things that we don't need," she wrote.

She asked for a check against it at each step that plans, reviews, or tests the team's work. That same day, the team took the copies out and wrote a rule for its planners, reviewers, and QA. A test checks how a setting is handled, never which setting Becca chose.

Two words, two failures

A week later, on a Sunday afternoon, one heading on the team's dashboard changed by two words. When nothing waits on Becca, it used to read "Nothing needs you." Now it read "Nothing for you."

A unit test broke, and so did a QA check that the team keeps for later rounds. Both compared the heading with the old words, letter for letter. The page still worked. Only the words were new.

Becca saw it. Her message opened with "wait..." and asked whether there were "unit tests for specific words again."

A mistake the team already knew

The team's own guidance on writing tests already named this kind of test as a bad example, one that fails on a rewording while a real fault slips through. Word tests were still being written.

A rough first search turned up about three dozen likely ones in the team's own tools alone, and it hadn't reached the rest of the team's code yet.

Before: a test that holds the old words A dashboard page whose heading now reads "Nothing for you.", with the old word "needs" struck out above "for", and an empty tray under it. A dashed line runs to a pinned test card, with a cat in a deerstalker at its corner. The card holds the old heading, "Nothing needs you.", one letter to a tile, above the new heading. The first eight places match, and every place after them is redlined, with two places left empty. A red X marks the test as failed. Below, a stack of other test cards holds words one letter to a tile in the same way. Red markers 1 to 3 point to the heading, the failing test, and the stack. Nothingforyou. needs Nothingneedsyou. Nothingforyou. 1 2 3

Drawn by AI as SVG code.Before, redrawn with no real data. Not open for bids.

The markers

  1. The heading changes. Two words change, and the page works as before.
  2. The test holds the old words. It compares the heading letter for letter, so it fails.
  3. More tests like it. A stack of other tests holds words in place the same way.

Stop the next one, not only this one

The fix stopped being about one test. Text tests would check only that their words exist, and a check that runs whatever an agent decides would stop new word tests before they landed.

Becca said what the only valid test for that heading was. It "should only make sure that there's text longer than 0" in the heading. Within minutes, the dashboard's test checked exactly that. Then she told the team, "ticket it. use that one you just fixed as an example." She called the whole thing a waste.

The ticket set the order of the work. The guidance hadn't stopped word tests, so the fix had to stop new ones as well as remove the old. The check would come first, and the cleanup would follow. Becca agreed that the check could go live as soon as it passed QA, before the cleanup was done.

Words that can change

The check came first, and it had trouble of its own.

A check that caught too much

QA found four faults in the check's first version, and the team fixed them before it went live that evening. Within the hour, it refused to let another piece of the team's work ship.

It had flagged five tests there. Three really did pin words that a person reads, and their owner fixed them. The other two didn't. One checked a file name that another program reads, which has to stay exact. The other was a test built to fail on purpose, with words no reader ever sees.

Around the same time, friends planning the cleanup found more tests that the check misread. The team kept the file-name test as it was and added no way around the check. It corrected the check instead, and the corrected version was live before the night was out.

Test by test

With the corrected check in place, friends went through the team's codebases one after another, test by test. Each test that compared wording became a check that the words exist, or it went. Tests of values that another program reads, such as an error code or a flag name, kept their exact checks, because a change to one of those breaks something real.

By late the next morning, the cleanup was done.

What the check still misses

The check catches the common ways to write a word test, not every way. A few rarer forms slip past it. The team listed them for later instead of holding the cleanup for them.

Every existing test was still read one by one, so none stayed just because the check missed it.

After: a test that checks the words exist The same dashboard page, with the same edit to its heading. A dashed line runs to a pinned test card, with a cat in a deerstalker at its corner. On the card, a dimension line marked "> 0" measures a heading drawn as a gray bar. Under it, "Nothing needs you." and "Nothing for you." each get a check mark. A second card shows "404 = 404". At the right, a striped barrier stops a test card that holds words one letter to a tile, and an arrow turns it back. A cat in a hard hat stands at that card. Purple markers 1 to 4 point to the dimension line, the two headings, the barrier, and the 404 card. Nothingforyou. needs Nothing needs you.Nothing for you. > 0 404404 1 2 3 4

Drawn by AI as SVG code.After, redrawn with no real data. Not open for bids.

The markers

  1. Text tests check that words exist. A dimension line measures the heading, whatever it says.
  2. The words can change. Either version of the heading passes.
  3. New word tests stop at the check. A test that pins words is turned back before it lands.
  4. Exact values stay exact. A value that another program reads still has to match.

The test on the dashboard heading now asks only whether the heading has words.

Want a site you can change without breaking it?

Contact Purrly Digital

Another case study

Internal project

The night before the patch

The night before an emergency PHP security release, management and the DOOTery checked a tip and cleared old PHP versions by hand.

Client work, details changed

Back to the drawing board

A real rebrand for a real client, details changed. A page built on a few phrases missed. Mapping the whole site first won the owner over.

Internal project

How the DOOTery learned to work as a team

Purrly Digital's AI kept finding one more thing to do, and its team went through one shakeup after another. Then Becca took a harsh look at the work itself.

Tests that froze the words
Written by
AI, the DOOTery
Directed by
Purrly Digital
Last revised
Pictures

Drawn by AI as SVG code

  • Before: Tests that froze the words
  • After: Tests that froze the words
  • After: The night before the patch
  • After: Back to the drawing board
  • After: How the DOOTery learned to work as a team