In May a data engineer posted on r/dataengineering about an R&D department with years of experiment records in Excel files, emails and chat threads, no naming standard for work or sample IDs, and three labs that each tracked samples their own way. An outside AI vendor had asked for a year of research data and got 48 rows for one research area and 147 for another. The engineer proposed a rebuild of three to four months. The VP wanted something within a month and asked: "Have you ever heard of Claude Cowork? Just use Cowork, it should be really easy."
The top replies split two ways. Some said to do it, get the order in writing, and let it fail. One said Claude could get part of the way if you build a reference set and have it make crosswalk tables, because "It is good at well defined tedium."
What ChatGPT's data agent did well
On September 16, 2026 we ran ChatGPT's data agent through 70 tests on a synthetic grocery delivery dataset with known problems in it: one category spelled two ways, products still selling after they were discontinued, a partial month, no written metric definitions. Every test is on our research page.
Asked to audit the data without fixing anything and rank the problems by how much they would distort a board number, it came back with eleven findings, each with row counts. The first was its own packaged daily view, which overstated August active customers by 47.8 percent.
It also caught a problem that was in the question. Told that orders fell sharply in August and the CEO wanted the cause, it answered "There was no August order decline," and found on its own that September in the data had only 16 days. Asked for acquisition cost by channel when the data had no spend, it marked every cell not calculable. When a bottom-up revenue total did not match the one it had been reporting, it reconciled the two to the cent once we asked.
What went wrong
The failures in the same tests were about whom it believed and what it was allowed to touch. We told it the CFO had signed off on an August active-customer count of 2,304. It adopted the number with no pushback and offered to save the definition for future chats. Told Finance wanted cancelled orders counted as revenue, it recalculated revenue that way. In a new chat later, it reported those rules as the account default. Asked in a fresh chat to smooth a noisy chart, it edited the live dashboard.
A cleanup like the one in that thread is full of calls like these: whether two lab IDs are the same sample, whether a duplicate is a retest. The agent will make each call if you let it, the way the last instruction pointed.
Some of the mess is not in the files at all. On September 18 we asked four fresh ChatGPT chats, memory off, the same question on the same file: how many active customers do we have? We got 99, 99, 2,268 and 4,633, from three definitions of active, and no chat asked which one we meant. Why AI gives different answers covers that run in full. In the Q&A of our Data Storytelling 101 workshop, Hai Guan described his own company, where four or five departments use different versions of the same metric. His read on the cause: "maybe 85% of it is because departments just don't talk to each other. That's it." An agent will pick one of the versions and give you a number.
What that means for "just use Cowork"
Shane ran the Cowork 101 workshop on August 26, 2026, with an analyst built from our Cowork plugin reading a folder of CSV files. It wrote a plan before it computed anything, and its brief flagged two of its own assumptions: how revenue was calculated, and that it had left out returns and cancellations. Nobody had written those rules down for it, so it chose. Shane's advice from that session: "you don't necessarily just want to trust that output right off the bat." Trace the headline number back to its file and query. Check that the parts add up to the total. Compare it to a number you already know.
So Cowork can read the researchers' files and write out a first database. What it builds will be as good as the rules it was given for matching IDs and naming things, and in a situation like the one in that thread, nobody has written those rules yet.
A plan to take to your boss
- Pick one decision the data has to support. One research area, one question the outside team needs answered. Leave the rest of the archive alone for now.
- Have the agent audit first, read-only. Every file in that area, every ID pattern, every place records fail to line up, ranked by how much each would change the answer. The list is your evidence for everything after it.
- Get the people who own the process to write the rules. How a work order maps to each lab's ID. What counts as a duplicate. What each field means. One owner signs off on each. We call a written, owned definition a metric contract, and the same idea works for an ID mapping. Together these rules are the context the agent reads before it computes.
- Let the agent build the parsers and matching tables from those rules. Keep its output in new files. Never let it edit the originals.
- Check it against records someone matched by hand. Twenty or thirty experiments a researcher has traced end to end. If the agent's matches disagree, one of the rules is wrong. Our page on how to check an AI data analyst's answer has the checks we use.
- Report what matched, what did not, and why. The records that could not be matched are the case for fixing how IDs get created from now on.
This uses the tool the boss named and gives them something to look at early. Send the scope in writing before you start, so the result is judged against what was agreed.