What Breaks When an Agent Skill Changes Platforms

Every data pipeline goes through the same steps: find the source data, design the metric, define the schema, write the code, backfill and test, deploy, open a pull request. Those steps, said Shishir Tewari on stage at GitHub Universe, “were never the problem.”
The hard part is the questions that pop up along the way. What is the grain, meaning what does one row represent? Do deleted records belong in the pipeline? How should the columns be named? How should the joins be written? Only the most experienced engineers know the answers. They “know it by their heart because they have applied it again and again.”
His co-speaker, GitHub’s Andrea Griffiths, was a construction manager before becoming a developer. On every job site, Griffiths said, there’s one foreman who knows everything that’s going on. Engineering teams have one too, and in the talk, that person was Bob. You probably have a “Bob” in your team – the first person everybody is going to call.
Everyone else waits for Bob or makes assumptions. But assumptions have a price:
Whenever we make these assumptions, we get late-night calls. We turn on our computers and start fixing the pipelines midnight.
Tewari is Head of Enterprise Data Engineering at Procore Technologies, which makes construction management software. In the talk “Build once, run on any agent: a practical guide to agent skills,” he and Griffiths showed how to write that judgment into agent skills, so every engineer and every agent gets the same answers. Afterwards, he explained to me what broke when those skills moved to other platforms.
One file instead of everyone’s prompts
A skill, as Griffiths explained on stage, and as you probably already know, is a folder with a SKILL.md file: a name, a description and instructions. An agent loads it when a request matches the description.
Before skills, Tewari said, teams kept that knowledge in prompts. He calls skills a “very well-defined, polished version of the prompts.” The problem with prompts is that you have to repeat them again and again. And everyone keeps their own – two engineers doing the same work won’t use each other’s prompts, he said, because each thinks their own are better.
A SKILL.md file gives the team one shared copy. It’s version controlled and reviewed, he said, so engineers reach “one point where we both agree on a set of things.”
For Procore, that copy has to work everywhere, since they use different AI tools. Tewari defines a portable skill this way:
A set of instructions that can be executed at any platform, at any solution provider, anywhere, without any issues.
The instructions travel, but the connections might not
In practice, Tewari separates the skill from what surrounds it. The structure and instructions stay identical across platforms and the rest changes:
But when you port it, the tools – MCP, CLI, or other way of connecting these skills to the framework or to the platform – would be different. The models, the outcomes, the permissions and the credentials might be different as well.
The first thing to break was the connection layer. Procore uses Databricks, Snowflake and other providers, and his skill connected to Databricks through the CLI. When he ran the same skill on Snowflake, the CLI wasn’t supported. “So it literally stopped,” Tewari said.
He had to create an MCP server for Snowflake before the skill would connect and run. That was three months ago, he added, and things may have changed since.
Models were a different problem, because the team couldn’t control them. A model available in one tool wasn’t available in another. The team fell back to auto mode, where the tool picks the model, and the output changed. When they reviewed it, some column names were different. The fix was to run the skill again on a different tool.
Tewari doesn’t expect the differences between tools to last. New features sometimes land first in Cursor or Copilot, he said, and the others catch up in a day or two instead of months. Model support, though, he called “a debatable question.”
One manifest for business context
Tewari’s open-source project, dataforge, is his answer to Bob. The README describes it as “Four agent skills that build data pipelines, with no knowledge of your data in any of them.”
- forge-init connects to the database, reads what’s already there and writes a manifest with the zones, grains, domains, schemas and naming conventions.
- forge-dbx is the only way the skills talk to the data platform: authentication, queries, jobs.
- forge-lint holds the team’s standards for naming, SQL formatting and joins.
- forge-builder uses the other three to take a GitHub issue to a pull request.
The skills are written once as SKILL.md files and packaged for Claude Code, Cursor and Copilot, and CI fails if a package no longer matches the source. The project is MIT licensed and in beta. According to the README, the DuckDB adapter and GitHub Issues intake, both new in version 0.1.0, “are exercised by the example project rather than by production use.”
All the business context lives in the manifest, and the other skills read it. The design rule, as he put it on stage:
Each skill should not invent their own way to work in a database.
His reason is that business-critical information shouldn’t be repeated. In his demo, the domain was a service desk, and the schema covered its entities, such as incidents. The zone, grain, domain and schema are fixed, he said. If they aren’t written in one place, they get repeated in many, and if one copy is wrong, “then the outcome would be incorrect.”
When the business asks for another metric, he said, the team changes the metric name instead of rewriting a skill. The live demo ran on DuckDB, and pointing it at Databricks took a config change.
A checkpoint at every phase

The mentioned demo started from a GitHub issue asking how many service desk tickets were opened and how many were resolved. forge-builder asked about the grain and whether to use daily or cumulative snapshots, ran its discovery queries and wrote the findings into the ticket. Then it stopped:
So it is like a human-in-the-loop process. It is not going to write anything.
It stops at every phase. It proposed counting resolved tickets by the closed_at column instead of the status column and asked for approval. It saves its progress to a config file, so if the run stops halfway, it doesn’t start over and burn the same tokens again. It showed the DDL and the test SQL for review, and “it is not going to deploy until and unless you literally review everything and you just say, yes, done, approved.” While running, it pointed out that it counted distinct incident IDs, not all rows. At the end, it opened the pull request and updated the ticket.
The lint skill hit a case its standards didn’t cover. The team had defined a naming convention for metrics, roughly “total,” then the schema and the entity. When the skill ran against a legacy database built before those standards existed, lint stopped the run and asked for approval. Rewriting the existing database would have been too much work, so the team made an exception to the standard for that database.
Testing is part of the run
Most of Tewari’s career has been in finance data, at J.P. Morgan, Morgan Stanley and Google Finance. At Google, he said, he led platform engineering on pipelines whose data was reported to Wall Street. That’s why he knows the value of QA.
For him, testing isn’t a separate skill. The skill that builds the pipeline also writes the tests, runs them and only then moves on, and there’s a separate QA pass on top. It’s how engineers work, he said: they write the code, test it, then hand it to QA.
His advice for teams starting out is to build testing into development from the start. A defect you don’t catch early costs extra money, and fixing it later means a lot of back and forth. Or it gets escalated to senior leadership:
They will not like the numbers. And then they will reach out to you and then you will fix it.
The money isn’t the only cost. You also lose the confidence of the people who rely on your data, he said, whether that’s senior leadership or an outside customer. That’s true in any company, and just as true if you run your own business.


