The developer already tested it, so why do you need to test it too?

If you work in tech, you’ve probably heard this sentence a thousand times.

A developer writes the code, runs the checks, all the tests pass, and the feature works.

So why do companies still hire Quality Assurance (QA) engineers? Are we just here to double-check developers’ work, or is there a deeper purpose?

The short answer is: no.

The longer answer is that there’s a fundamental difference between writing code and creating a great customer experience. The value of QA lies in perspective, risk awareness, and understanding how people actually use a product.

The builder

To build a truly great product, you need two very different mindsets: the builder and the chaos maker.

A developer’s brain is wired for creation, logic, and structure. While QA starts the day with a finished product, a developer starts with a blank screen. Their job is to create something out of nothing and assemble the many moving parts of a complex system into something that works exactly as intended.

Developers are responsible for creating the magic. To build even a single feature, they have to master a labyrinth of tools, frameworks, databases, and configurations. They carry the responsibility of making sure the architecture is secure, the system can handle heavy load, and the new code integrates cleanly with millions of lines of existing logic. Their work is highly complex and demands deep understanding, precision, and sustained focus. They are the architects of the digital world.

Because their primary mission is building and stabilizing the system, their testing mindset naturally reflects that. When developers test their code, they are checking whether their blueprint holds up. They focus on the happy path, along with a few common scenarios, in a controlled environment, using standard data to confirm that the core functionality works as intended. Their attention is on making sure the foundation is solid.

Think of a developer as an architect and engineer combined. They design a beautiful, highly complex door, check that the handle turns smoothly, inspect the hinges, and confirm that it opens perfectly.

Now what? Aren’t developers already responsible for the whole thing?

The chaos maker

While developers are incredibly skilled at what they do, they cannot be expected to carry the entire burden of the user experience alone. That is where QA steps in – to relieve some of that pressure and help refine the product for the real world.

If a developer’s brain is wired for creation, a QA’s brain is wired for exploration and chaos. The goal is not to prove that the software works, but to discover what happens when the scenario is not perfect. We test unusual cases, unpredictable behavior, and the edge cases that real users may actually trigger. We test for the messy real world.

Because of this, QA goes far beyond engineering. The work calls for empathy, human psychology, and an eye for the emotional, reliability, and invisible factors that make customers want to use a product.

As a QA, I’m constantly stepping into the user’s shoes and asking myself a very different set of questions:

• Is this accessible and intuitive to use?
• What emotion does this product create for me – frustration or delight?
• Would I still enjoy using it tomorrow, or would it already be starting to annoy me?

What happens in the real world

To make this point come to life, let me share a personal experience.

I’ve seen that even a large team of skilled developers means every single scenario is not automatically covered. Tight deadlines, heavy workloads, and the sheer mental focus required to build complex architecture make it nearly impossible for any single perspective to catch everything. Once a feature is done with implementation, it’s QA’s turn to step in, look beyond the happy path, and hunt down every possible edge case.

The “blank space” API trap

For instance, when new feature was implemented, it was time to test it. My team and I ran automated tests for our Voice APIs.

During testing, I decided to try something slightly unusual: I sent a call using a sender name that contained a space. Immediately, the calls started failing. This turned out to be a critical issue. If a real client wanted to do something as simple and common as putting a space in their sender name (e.g., “Global Bank” instead of “GlobalBank”), their entire voice campaign would fail. We only discovered this flaw because we had an automated QA project specifically designed to stress-test the exact parameters our clients use daily, which even shows the importance of regression tests and their benefits before the release.

The edge case that shouldn’t break the system

Another instance happened while I was testing the delivery reports for our system. The documentation stated that the record limit field was limited to 1 to 1000. My QA brain asked, “What happens if a user accidentally types 0?” When I entered 0, the system crashed and returned a 500 Internal Server Error.

Technically, it was just a wrong number. But from a customer experience standpoint, this is a big deal. A 500 error tells the client, “Your servers are broken,” which causes panic, especially during real-time actions. It should have gracefully returned a 400 Bad Request, which politely tells the user, “Oops, your input was invalid, please try again.”

Logically, it makes no sense for a user to request 0 records. But it is our duty as QA to make sure that even when a user does something illogical, the system handles it elegantly instead of catching on fire.

The trapped user

The chaos isn’t always in the backend, either. Recently, while performing manual UI testing, I found a scenario where a user could fill out an entire complex form, only to find that the “Save” button remained permanently disabled because of a tiny edge-case conflict in the UI logic. Imagine the frustration of spending ten minutes typing out intricate details, only to be trapped at the finish line and unable to save the work.

All of these instances prove why additional testing is necessary. It is incredibly easy for crucial edge cases to slip through when a team’s primary focus is building complex core functionality. No one should set a limit to 0. A sender name shouldn’t break an API. But humans aren’t strictly logical, and neither is the real world. That is why QA exists.

QA and devs are two halves of the same brain

Ultimately, this story comes back to the same point: success. Despite our different approaches, both teams are chasing the same finish line. We come together, we collaborate, and we bring an idea to life in the best possible way.

Both developers and QA want the same thing: to deliver a product that is simple, easy to use, unique, and highly functional. We want to build something so reliable and intuitive that the end user does not even have to think about how it works under the hood – it just does.

Because my job involves finding edge cases, logging bugs, and occasionally breaking a developer’s hard work, it is easy for outsiders to assume that developers and QA are at war. But that is far from the truth. We are not enemies; we are an essential partnership. We are a continuous feedback loop where logic meets empathy.

Just like a human being cannot function with only half a brain, a truly great product cannot survive in the wild with only one mindset. It needs the builder’s solid foundation, and it needs the chaos maker’s reality check.

The developer brain ensures the software is built right. The QA brain ensures we’re building the right software for the customer. Together we create the product that customers love.

Before you go…

Next time someone asks, “The developer already tested it, why do you need to test it too?” you’ll know exactly what to say:

Code lives on servers, but customer experience lives in the real world, and QA is the bridge between the two.

The post QA Is What Happens When Software Meets Reality appeared first on ShiftMag.

We help companies put AI to work in real environments.

We spoke with him about what it takes to move AI agents into production, how developers can make them reliable and secure, and which skills will matter most as AI changes software development.

Production starts with evals

As Luis hinted in the intro, Forward Deploy Engineers (FDE) help companies turn AI demos into real products. So we asked him what developers most often underestimate when they make that jump.

According to him, the real challenge starts when a demo reaches production. A demo can work well in controlled conditions, but production forces teams to understand exactly what works, what does not, and why failures happen.

That is where eval-driven (Evaluation-driven) development becomes important. He says OpenAI uses it extensively in its engagements to track and measure different workflows and tasks:

We have a scientific approach to measure the performance of the models doing a task. When, basically, you clear those benchmarks, those evals, you are good to go to production in a really good and safe way.

The deployment gap is the real challenge

We wanted to understand what makes an AI agent reliable enough for production, rather than something that only works well in a controlled environment.

Luis said it all comes down to “cracking the evals.” As he explained, today’s models already have impressive intelligence, but putting them to work in a company’s own environment requires giving them access to the company’s data, procedures, and tools:

You know, those three things, we kind of call the deployment gap, right? And this is what we’re trying to close as FD, basically.

Agents went from six-minute to 16-hour tasks

But what happens when an agent works on a task for hours rather than minutes? How should developers think about errors, memory, context, and human approval as those workflows become longer?

Luis answered with an example from his own experience. While checking the latest data from METR’s long time horizon benchmark, he was surprised by how quickly agents had improved. In just over three years, they went from handling tasks that would take a human around six minutes to tasks that would take around 16 hours.

That represents a massive jump, he said:

On one hand, you have the models getting better at long-term coherency and persistence, but on the other hand, you also have the harnesses improving a lot. Back in the days, a developer needed to care about context management, memory creation, and all of that. Right now, as I say, with harnesses such as Codex, all that is gone.

For Luis, that progress comes from combining more capable models with more powerful harnesses. Together, they allow engineers and developers to run increasingly complex workflows over much longer periods of time.

Agents should follow the same access rules as employees

When it comes to security, we wanted to understand the safest way to connect AI agents to company data without creating new permission and access risks.

Luis explained it through a familiar workplace example. When a company onboards someone who works in finance, it gives that person access to financial data, but not to HR data they do not need.

He argues that companies can apply the same principle to AI agents, using the same access controls and permission rules they already use for employees:

In the same way that when you onboard an employee into your company, and let’s say he or she is working in finance, you only give access to the finance data. You don’t give access to the HR data, right? So, in the same way, those same principles, those same ACLs, can also apply to AI agents, right?

For him, the principle comes down to giving agents access only to what they require to perform their tasks:

So, in my view, it’s about having the roles, the permissions, and the setup for the agents to only see what they need to see for completing their workflow, basically.

Evals are fundamental to AI development

Finally, we asked Luis what developers should test and measure before releasing an AI agent to real users.

Once again, he came back to evals. He said he keeps returning to the concept because it remains critical and fundamental to AI development:

I think we go back to evals once again, right? And I keep coming back to that concept because it’s absolutely critical and fundamental for AI development, right? If you’re able to put together that representative eval set containing easy cases, medium cases, and also edge cases, and you’re able to highly lean on your skills, on your guardrails, on the prompts you send to the model, that will give you the guarantees that when you deploy to production, nothing will break.

He added that evals also give developers a way to test for regressions when they upgrade to a new model and make sure the change does not break existing workflows:

Furthermore, when you upgrade your model to a new one, you have a way to test regressions and make sure that that upgrade won’t break anything.

System design taste will never go away

Before joining OpenAI, Luis also worked at Google, so over the years he has seen plenty of developers, roles, and career dilemmas up close. That is why, to close the interview, we asked him which skills developers should focus on today to stay relevant in the age of AI agents.

He believes that, in this new era, engineers will no longer create code as their primary output, because code has become very cheap to produce.

Instead, he sees much more value in everything developers build around the model:

I think now, in this new era we’re entering, the primary thing that engineers are creating is not code anymore. Code is very cheap to produce. It’s all about building the system around it, so the system of intent, the guardrails, the constraints for the agent to work reliably, right? So I think creating those harnesses around the model will be absolutely critical.

Luis ended our interview with a simple but sharp statement:

Plus, I always say, like, system design taste will never go away.

The post OpenAI’s Luis Velasco: Code is very cheap to produce, but the system around it isn’t appeared first on ShiftMag.

I have never particularly liked being the code reviewer.

It feels like getting all the responsibility but zero fun of actually writing the code. Even before most of the code became AI generated, the paradox was familiar to many: the more lines of code a PR has, the less likely it is that the changes will get a very detailed review. Why do we expect standard code reviews will still work when AI generates much more code volume?

For a workplace where agents are doing the heavy lifting, benefits of code review through classic PRs are gone, and there are a few reasons why:

1. Engineers do more prompting and decision making than coding. But most people aren’t reviewing the former.
2. The reviewer might be the first one to actually have to read all the code. Since the implementer gets familiar with the implementation through the prompting process, the reviewer is in a worse position. If they go through everything line by line, it probably takes them longer than it took the original developer. If the real time cost of review isn’t reflected in sprint capacity planning, reviewers are probably not given enough time to finish the task.
3. AI code review agents are a logical next step. To some extent, they are already utilized by AI-transformed companies. However, just throwing an agent at a PR without any structural and conceptual changes around code reviews, such as new types of safeguards and a robust production setup with automated rollbacks, misses the point of code reviews entirely. The human is no longer in the loop.
4. Context switching. It was always an unwanted companion of software engineers, but now it’s like a shadow we can’t escape. While our agents are thinking and combobulating, we do something else. We spin up a few agents for a different task, and then another for the next one. What happens to the PRs then? They pile up. The pressure to downsize that pile can be real, so the review quality becomes questionable.

Teams have adapted to AI quickly in many parts of the workflow, but code review is still lagging behind. Instead of following the same workflow as before, we should rethink what code review is and how we practice it. While this might not suit best for every organization, for me and my team, the most useful model right now is a pair-programming spinoff: pair planning and pair validation.

Pair planning

Pair planning is the first step. It starts with prompting the agent. And it matters how you do it.

The easy option is to drop the task description to an AI agent and let it do the work however it wants. Long-term, this is a good way to create a big pile of slop.

A better way is to ask the agent to propose multiple options and document them, without any code changes. While the LLM is doing its magic, this is your time to take a moment and think about the task in front of you. Ideally, the thinking happens out loud because you’re pairing with another colleague.

Why is the part when you’re thinking without any AI assistance important? Once you start reading the AI output, it’s highly likely your brain will lock in. Alternatives which the agent didn’t propose are more difficult to think of. It’s also possible that the task description wasn’t detailed enough and the agent didn’t take an important detail into account. Any caveats, edge cases, deployment scenarios, or other details that come to mind while you’re brainstorming about the possible task implementation details are good to note down before you read the agent output.

The agent is done planning. Output is ready. Now what?

Both you and your pair should get to reading. Flag any issues you see and discuss them. Refine the plan. Squeeze the implementation details from the agent. Be aware of all components that will change and how. This is the juicy part of the task.

With AI added to the traditional code review flow, we often plan the implementation twice. The first time is by the original engineer implementer. This person will make all the decisions individually. They will guide the implementation. Then the second time is by the reviewer, possibly in even more detail. If the reviewer doesn’t agree with the way task was implemented and proposes a different approach, the author and agent will go back to the beginning. All tokens used on the wrong path were wasted.

These kind of “disposable code” scenarios are happening more and more. When pairing with another colleague during the planning phase, it’s more likely you will choose the best implementation option and reach your goal in minimal time and tokens spent.

Structural prompt committing

Most of the engineering work gets done in the planning phase. Decisions are made and propagated into code. From commits only, we can only see what was decided. Why it was decided and what the alternatives were is valuable information that tends to get lost.

Prompting practices can be refined to avoid losing decision data. I practice structural prompt committing. In the codebase, each task is documented with my prompts, distilled versions of agent responses, and a few more useful files. The structure can look something like:

• task folder
  • prompt.md
    • chronological log of your prompts and distilled info about agent responses, append only
  • context.md
    • current context, changes can be traced through git versioning
    • either for human reference or for loading agent context
  • plan.md
    • after all the decisions are made via prompting, full plan is documented to a separate file
  • summary.md
    • summary of implementation changes

This structure is useful while working on the task, especially if multiple persons are involved. Afterwards it can be deleted. For important decisions, architectural decision records should be made.

Implementation ideas

How you decide to do the implementation after the plan is ready depends on many factors. Some ideas can be applied generally:

• If unsure between options proposed by an agent, ask it to prototype them on multiple branches. See how they act in action and validate through tests, metrics, etc.
• Small commits are desirable. It is easier to follow the changes and revert precisely if needed.
• For bigger tasks, create checkpoints. Don’t let the agents create a massive change which will have to be fully reverted if it proves wrong.

You might be wondering, what should you do while Claude is claudeing? Should the pair of colleagues just… wait and watch?

Well, maybe. If the checkpoints are small enough it might make sense to wait and discuss the direction the agent is taking. If not, it could make sense to split up and meet again when the project is ready for validation and review. It’s up to you to weigh if the context switch is worth it.

Pair validation

Whatever road you choose, you should arrive at the last part: pair validation. Why didn’t I call it pair review? Because review sounds like you’re just reading the code. Instead, at this stage you should validate, i.e. prove, that code works.

If the implementer and reviewer validate alone, the job is doubled and less efficient. Since AI is generating most code, we are all validators more than implementers. Let’s not validate twice. Two colleagues should go through the code changes in a structured way, making sure the requirements and acceptance criteria are met. Ideally, start from the tests. Are they proving the new code is working? Are they covering all use cases? Are they guarding the future implementation from possible breaking changes?

Through conversation, concerns can be raised and resolved immediately. Learning and knowledge sharing are instantaneous. Don’t lose this value by spending time navigating in heaps of generated code and writing async comments someone should check between three different context-switching sessions.

If you work in an environment that allows it, I encourage you to start thinking about the process of implementing and reviewing as one unit, not two separate jobs for two separate people. A good way to avoid getting in the “just review this quickly please” trap is to plan ahead. During planning of what each teammate will do in a given period, allocate the same amount of time and resources for both the implementer and reviewer. Teamwork begins before the task is started.

Not all jobs can follow this framework. Examples are open-source project PRs, distributed teams across multiple time zones, and contractor engagements. In these kinds of environments, reimagining how we do reviews gets even more interesting. I expect new and completely different frameworks to emerge. Until then, my team and I are quite confident shipping pair-planned and pair-validated code.

Until then, my team and I are quite confident shipping pair-planned and pair-validated code.

The post Review fatigue is real. Here’s what my team did about it appeared first on ShiftMag.

Small teams are taking over the whole build 

The structure of software teams is one area where Dana already sees a noticeable shift. Larger teams traditionally split these roles among several people, but much smaller groups can now handle them together:

The biggest difference is the empowerment that teams have. Before, you would have teams of 5, 6, 8, 10 people that all had very distinct roles in the software development life cycle, and I think the biggest shift is the enablement that people that are enabled solo, or two or three, can do everything end-to-end.

That shift has also influenced how Netlify organizes product development. Dana says the company no longer relies on sequential handoffs between product managers, designers, and engineers.

Instead, Netlify gives teams more autonomy through a narrative-driven product roadmap. The company sets the broader direction, while individual teams decide how to shape what they build next. 

To stay aligned without introducing additional layers of coordination, Netlify combines familiar tools and routines with AI agents:

We really let our teams have some personal take on what they’re going to build next, but we utilize systems like Linear and stand-ups, and honestly agents to keep us all in line so that we can move at the speed of machines. 

Low-ego leadership 

More autonomous teams also change what Netlify CTO expects from the people leading them. 

When asked which leadership lesson has stayed with her through roles at GitHub, Heptio, New Relic, and Netlify, Dana returned to two ideas, trust people and recognize expertise regardless of title: 

Low ego. You are going to have people smarter than you in the room just because you have the amazing title and you are marked the leader. Your job is to be humble, build connections, and really learn, and be a part of your team and not just mining your team, and I think the biggest lesson is to trust people and allow them to be adults.

That principle becomes even more relevant as the work itself changes and developers increasingly operate alongside AI agents. 

Developer experience is now an agent experience 

When we asked Dana what a great developer experience looks like today, she framed the question around a different relationship between engineers and code. 

For her, the developer’s job is increasingly about enabling agents to act safely on their behalf: 

A great developer experience is an agent experience. No longer should my developers be writing code. They should be enabling agents to do stuff safely and on their behalf. 

That kind of environment also depends on developers being able to understand what those agents are doing. 

Dana points specifically to clarity, signals, and telemetry as the mechanisms engineers need both to help agents complete their work and to understand what they have done once changes reach production: 

A good engineering experience is ensuring that we have clarity, signals, and telemetry so that their agents can get the job done, and so that they can know what the agents did in production. 

From writing code to system design 

If agents take on more of the direct implementation work, Dana sees engineers spending more of their attention at the system level. 
 
She describes that change as a move away from creating code itself and toward designing the environment in which software is built and operated: 

We’ve moved it from creating code to really architecting systems and allowing engineers to do what they do best, which is build trust within the systems.

In that framing, the impact of AI is not limited to making individual development tasks faster. It also changes where engineering judgment is applied. 

Don’t use AI agents to automate bad processes 

Dana is also cautious about one of the most straightforward ways companies can approach AI adoption, taking an existing process and simply inserting an agent into it. 

Instead, she argues that teams should reconsider the workflow itself: 

Agents can’t do everything. They can do everything, but they shouldn’t do everything, and it’s a trap. Don’t start building things on existing workflows. Change your workflows.

That becomes even more relevant as Dana expects engineering roles, tools, and workflows to change significantly over the next few years:

Enable an agent experience, because in two years, our jobs will significantly change, and some of the tools that we’ve been building and some of the workflows will be commoditized. 

That doesn’t mean replacing every existing tool 

At the same time, Dana is not arguing that teams should replace technology simply because something newer exists. 

She specifically points out that tools such as Bash can still have a place when they remain appropriate for the problem. 

Her advice is: “Keep it simple.” 

The post Netlify CTO: “No longer should my developers be writing code” appeared first on ShiftMag.

When I first encountered MongoDB, I tried to understand it through the database I knew best: PostgreSQL. My journey was a baptism by fire. I was managing a sharded cluster with 20 TB of data and as many as ~55k operations per second, including ~29k writes. The cluster had six shards, each with its own replica set, a config server replica set, and six routers.

To operate that cluster safely, I had to stop treating MongoDB like PostgreSQL and learn how it worked internally.

At one point, while tackling a flow control problem on the stated cluster, an AI model recommended increasing the portion of memory assigned to the WiredTiger cache. At the moment, the recommendation sounded genuine and correct, but I did not know what the model had based its conclusions on. To be completely honest, I didn’t even know how RAM is used by MongoDB. The nail in the coffin was the official docs saying don’t change WiredTiger cache to RAM ratio.

That experience clarified why I wanted to learn the internals. I want/need enough understanding to connect operational advice to a mechanism, identify the evidence it depends on, and decide. This applies whether the advice comes from AI, a colleague, documentation, or my own assumptions.

What happens after a MongoDB client sends a write?

How does that write pass through MongoDB and WiredTiger, and when does it become durable on disk?

And when someone recommends changing a WiredTiger setting, what would have to be true for that recommendation to make sense?

Following one write

When a client sends a write operation to a sharded cluster, the request first reaches a router called mongos. Routers cache data from the config server about chunk – shard placements. Using this data mongos determine which shard owns the relevant data and forward the operation to that shard’s primary node.

However, this post does not compare replica sets and sharded clusters. Once routing is complete, the same question applies inside every shard.

How does the primary mongod process request, persist it, and replicate it to secondary nodes?

Separation of concern

To answer my question(s), I firstly had to understand where mongod ends and the storage engine WiredTiger begins.

mongod handles client commands, authorization, query execution, transaction coordination, and replication. WiredTiger is the embedded storage engine that stores collection and index data. It provides local transactions, MVCC, caching, compression, journaling, checkpoints, and crash recovery.

MongoDB client
      │ write operation
      ▼
mongos query router
      │ routes to the owning shard
      ▼
shard primary (mongod)
      ├── authorization and command handling
      ├── query parsing and execution
      ├── transaction and session coordination
      ├── replication and oplog coordination
      └── storage-engine API
                  │
                  ▼
             WiredTiger
                  ├── local transactions and MVCC
                  ├── internal cache
                  ├── journal
                  ├── checkpoints
                  └── collection and index data in dbPath

Ok, but how does this come into play? Boundaries are not absolute.

The oplog belongs to MongoDB’s (logical) replication model, but WiredTiger stores it alongside other collection data.

One write, two logs

At this point, I had another question. If MongoDB already has a journal, why does it also need an oplog?

The WiredTiger journal is the closest equivalent to PostgreSQL’s WAL (Write Ahead Log). It records the changes needed to recover one MongoDB member (keep this thought) after a crash. Unlike PostgreSQL’s WAL, MongoDB does not use the journal as its replication stream.

The oplog has a different purpose. It is a capped collection stored in the local database as local.oplog.rs. Secondary members copy its entries and apply them to their own data.

This raises an important consistency question. What prevents MongoDB from committing a document change without its corresponding oplog entry?

Data folder

Let’s examine what happens on one node and then additionally complicate the story by adding secondaries.

To understand local durability, I first looked inside the dbPath of a MongoDB member. A simplified listing:

$ ls /data/mongodb/
collection-<ident>.wt
index-<ident>.wt
journal/
WiredTiger
WiredTigerHS.wt
WiredTiger.lock
WiredTiger.turtle
WiredTiger.wt

The exact files and their names depend on the MongoDB version and configuration, but these are the main WiredTiger files relevant to this discussion.

The collection-<ident>.wt and index-<ident>.wt files are WiredTiger tables that store collection records and index entries.

MongoDB’s durable catalog maps logical objects, such as a collection namespace, to these storage-engine identifiers. WiredTiger maintains its own metadata in WiredTiger.wt, including its tables, their configuration, and their latest checkpoints.

The small WiredTiger text file identifies the WiredTiger version used to create the database:

WiredTiger
WiredTiger 10.0.2: (November 30, 2021)

WiredTigerHS.wt is the history store. It keeps older committed versions of records needed by MVCC readers, snapshots, etc. WiredTiger removes versions once no active reader or snapshot can require them.

WiredTiger.turtle contains enough metadata to locate and open the latest checkpoint of WiredTiger.wt. It bootstraps the metadata table during startup and recovery.

WiredTiger version string
WiredTiger 10.0.2: (November 30, 2021)
WiredTiger version
major=10,minor=0,patch=2
file:WiredTiger.wt
...
checkpoint_lsn=(2248,16291200)
...

The checkpoint_lsn identifies a position in the WiredTiger journal using a log file number and byte offset. During recovery, WiredTiger opens the last consistent checkpoint and replays later journal records.

The journal/ directory contains those write-ahead log files. They protect changes made after the last checkpoint from being lost during an unexpected shutdown.

Finally, WiredTiger.lock is the file on which WiredTiger acquires a lock to prevent two processes from opening the same database directory simultaneously. The lock state matters, not merely the existence of the file.

The WiredTiger cache

WiredTiger does not modify the collection-*.wt and index-*.wt files directly for every client operation. It first reads the required pages into its internal cache and applies the change there as part of a storage transaction. Modified pages become dirty until WiredTiger reconciles and writes them to disk.

This internal cache keeps uncompressed collection data and is separate from the operating system’s filesystem cache where compressed data is stored.

MongoDB therefore benefits from both caches, WiredTiger and the OS cache.

Giving all available memory to the WiredTiger cache would leave too little memory for the filesystem cache and the rest of mongod and respecting the default, max((availRAM - 1024) x 0.5, 256 MB), is strongly advised by the official docs.

This is already enough to show why “increase the WiredTiger cache” is incomplete advice. A larger cache does not create memory. It transfers memory away from the filesystem cache and other allocations. Whether that trade is useful depends on the workload and the source of the observed pressure.

When the cache approaches its limits, WiredTiger evicts pages to make room. Clean pages can be discarded and read again later. Dirty pages must first be reconciled into an on-disk representation.

A committed change does not have to wait for its dirty page to reach the collection file. The journal supplies durability between checkpoints. This separation is the reason a write can be durable even though its final data page is still dirty in memory.

Recovering after a crash

In case of a crash, WiredTiger starts from the last complete checkpoint. The checkpoint metadata identifies a consistent view of the WiredTiger tables and the journal position associated with it.

WiredTiger then replays the journal records created after that checkpoint. At a high level, recovery combines two inputs:

 last complete checkpoint + later durable journal records = recovered local state  

Checkpoints and data files

WiredTiger normally creates a checkpoint every (configurable default) 60 seconds. A checkpoint can take longer when there is more dirty data or the storage device is under pressure.

During a checkpoint, WiredTiger writes a point-in-time view of dirty pages to the collection, index, history-store, and metadata files.

WiredTiger transaction
        │
        ├── dirty pages in cache ──checkpoint──▶ .wt data files
        │
        └── journal records ───────────────────▶ journal files

                         crash
                           │
                           ▼
       latest checkpoint + subsequent journal records
                           │
                           ▼
                    recovered state

How the primary records an operation

For an incoming query, mongod will, among other things, determine collection and index changes and construct the oplog logical entry.

It uses WiredTiger to start/use a transaction and either commits both parts or commits neither. This prevents a successful document change from existing on the primary without an operation (oplog entry) that secondaries can replicate.

The oplog entry contains an operation type, namespace, timestamp, and the data required to reproduce the change. Its timestamp provides an ordering position in the replica-set history. The entry describes a logical insert, update, delete, command, or transaction operation rather than the low-level page modifications stored in the WiredTiger journal.

rs_a [direct: primary] local> db.oplog.rs.findOne()
{
  ts: Timestamp(...),
  t: NumberLong(...),
  op: "u",
  ns: "foo.bar",
  o: { $v: 2, diff: { ... } },
  o2: { _id: 123 }
}

This is not a complete definition of how oplog and collection changes work, but the important invariant is: the primary commits the replicated data changes together with the oplog records which describe them.

Only the local commit is atomic. Sending or copying the oplog entry across the network is not part of that storage transaction.

How secondaries copy and apply oplog entries

Each secondary continuously selects a sync source and streams newer oplog entries from it. The sync source is often the primary.

The secondary writes fetched entries to its own oplog and then applies the represented operations to its local collections and indexes.

The secondary uses its own WiredTiger instance to apply changes. They modify its cache, produce local journal records, and later become part of its checkpoints.

This is the important distinction and the answer why we can’t just ship primaries *.wt binaries to the secondary.

On two MongoDB servers, these are different files.

Oplog versus journal

Takeaways

This gives a practical sequence for evaluating operational changes:

1. Identify which component owns the behavior
2. Describe the mechanism that is a cause of the problem
3. Find evidence supporting that mechanism
4. Clarify what the improvements should be
5. Measure
6. Iterate

The technical facts in this article support that way of reasoning:

• The WiredTiger journal and the oplog solve different problems. The journal recovers one member; the oplog replicates logical operations between members.
• MongoDB commits a replicated data change and its oplog entry in the same local storage transaction on the primary.
• A durable change can still exist as a dirty cache page because the journal protects it between checkpoints.
• Primaries and secondaries have separate WiredTiger databases, caches, journals, and data files.
• Memory assigned to WiredTiger is part of a larger allocation decision that includes the filesystem cache and other process memory.

Why I wanted to understand this

Learning how WiredTiger works does not mean that I can derive every production decision from first principles. It gives me a way to examine a recommendation instead of accepting it because the source sounds confident, and we all know AI can sound like that.

The quality of an (AI) recommendation depends on the question and the evidence supplied to it.

Further reading

• MongoDB manual: WiredTiger storage engine
• MongoDB manual: Journaling
• MongoDB manual: Replica set oplog
• MongoDB manual: Replica set synchronization
• MongoDB source documentation: Storage engine

A note on scope – This article records my current understanding of MongoDB and WiredTiger, built through hands-on operation and continued study. I have tried to keep the technical details accurate, but any errors or oversimplifications are mine. MongoDB official docs and WiredTiger documentation should always be consulted for correctness and clarity.

The post MongoDB: One write, two logs appeared first on ShiftMag.

The best runbook I ever read was also the most dangerous one I ever followed.

It was about database failover procedure. 14 steps. Step 3 said “verify the replica is caught up” but didn’t say how. Step 7 said “if the promotion fails, escalate to the DBA team” like the DBA team was sitting around at 3 AM. Step 11 was just the word “validate” followed by a period.

The whole document was a confession. It said: “We know this could go wrong, someone wrote down what they remembered, and we hope you figure out the rest”.

This is the standard state of operational knowledge at most companies and in the AI era it’s an architectural dead end.

The runbook is the institutional debt

Every runbook starts with good intentions: something breaks, someone fixes it and they write down the steps so the next person doesn’t suffer. Six months later, half the commands are deprecated, the dashboard URLs point to a monitoring tool that was replaced, and the person who wrote it left the company.

Think about what a runbook really asks of the person on call. At 3 AM, with adrenaline pumping and Slack pinging, you’re supposed to read a wall of text, mentally parse conditional branches (“if the primary is unreachable but the replica is healthy, proceed to step 4, otherwise skip to step 8”), manually copy-paste commands into a terminal, and hope you don’t typo a hostname.

Let’s be real: nobody practices runbooks and nobody rehearses them. Your first time through the procedure is the incident itself. This looks like a runbook, but it behaves like a liability with good formatting.

Knowledge should be executable

Here’s the shift: stop asking where to document the procedure, and start asking how to build it into the platform.

An executable runbook behaves like a service: you send it inputs, it runs the steps, and it returns a result. The real logic lives in tested, versioned code.

The progression usually goes like this:

• Stage 1: The prose runbook. A page someone wrote after the last outage. Accurate for about two weeks, then slowly rotting.
• Stage 2: The personal script. Someone on the team writes a bash script that automates the worst parts. It lives in their home directory on a bastion host. Nobody else knows it exists. When they leave, it’s gone.
• Stage 3: The shared tool. The script gets checked into a repo. It has a README. Maybe a Makefile. People use it, but they have to know where the repo is, clone it, install dependencies, and run it from the right machine.
• Stage 4: The config-driven service. The tool becomes a proper internal service with an API. It reads configuration from a central source of truth. It validates inputs before acting. It logs what it did. It can be called by a Slack bot, a CLI, or an AI agent without any of them knowing how it works internally.
• Stage 4 is where executable runbooks unlock the AI era. Here’s the hard part: AI agents can’t always follow a runbook. They can’t read a Grafana screenshot, SSH into a bastion host, or run a script from someone’s home directory. What they can do is call APIs, invoke tools, and pass structured inputs to services that return structured results.

During incidents, the system should do the procedure for you

Let me make this concrete.

Today. Today you get paged: the primary database is down. You open the internal wiki and start following the steps. Check replication lag. Copy the command. Swap in the hostname. Run it. Lag is zero. Good.

Then you try to promote the replica, but it fails because you’re on the wrong jump host. So you find the right one, SSH in, run it again, and it works. You update the application connection string, forget the read-replica config, and suddenly half the services start failing because they’re still pointed at the old primary. You fix that too.

Total time: 25 minutes. Steps you had to remember: 10. Things that went wrong because the runbook was out of date: 3.

After. Same scenario. You get paged. You type /db-failover in Slack. The bot asks which cluster. You say orders-db-prod. It checks replication status, promotes the replica, updates the connection strings in the service mesh config, verifies all services can reach the new primary, and posts the result back to the incident channel.

Total time: 90 seconds. Steps you had to remember: 1. Things that went wrong: 0.

The executable version isn’t magic. Under the hood it’s calling the same commands. The difference is that the knowledge of how to do those steps, in what order, with what validation between each one, is encoded in a service rather than a document that someone reads for the first time during an outage.

What makes it a tool instead of a script

The real difference is whether the thing can be safely used at 3 AM by someone seeing it for the first time.

• Every step validates before it acts. The tool checks replication health before touching anything. If active writes are still in flight, it aborts instead of corrupting data. The human doesn’t have to remember the preconditions. The tool refuses to proceed without them.
• Every step has a failure path. Promotion fails? Roll back. Connectivity check fails? Report which services are broken. The tool handles the branches. The person on call only needs to know one command.
• It’s idempotent. Running it twice on an already-promoted replica is a no-op, not a disaster. That matters at 3 AM when you’re not sure if the first attempt went through.
• One endpoint, every consumer. POST /tools/db-failover is what the Slack bot calls, what the CLI calls, and what the MCP server exposes. One backend. One source of truth. Every surface gets the same capability.

Why this matters for AI agents

This is the part I care about most. The current excitement around AI operations assistants, AI SRE copilots, AI incident responders, all of it hits the same wall: these systems can only operate on structured, machine-accessible interfaces.

A document can be read and summarized, but it still gives an AI agent nothing to call.

When you turn a runbook into a tool, you’re helping more than the person on call. You’re building something your AI agents can use too. Today it might run from a Slack command. Tomorrow it might be called by an MCP server or an agent. Either way, it just does the job.

This is the bridge between the other articles in this series. MCP servers need tools to expose. Slack bots need tools to call (article two). Executable runbooks are those tools. They’re the operational layer that both the human interface and the AI interface sit on top of.

This has never been easier

Five years ago, converting a runbook into an API-backed service meant writing a backend, wiring up auth, deploying it somewhere, and maintaining it forever. That’s why most teams never did it, the friction was too high.

That equation has changed. Low-code and no-code automation tools like n8n have made it trivial to wrap existing APIs, databases, and CLI commands into callable workflows. You can build a db-failover tool in an afternoon by chaining HTTP requests, database queries, and conditional logic in a visual editor. Expose it as a webhook, and suddenly your Slack bot, your MCP server, and your AI agent all have something to call.

The barrier isn’t technical anymore. We’re still defaulting to static documentation because that’s what we’ve always done. But when an n8n workflow can be built, tested, and deployed faster than you can format a runbook, “we don’t have time to automate it” stops being a valid excuse.

Start with the procedure that scares you the most

You don’t need to convert every runbook overnight. Start with the scariest runbook. Build the tool, test it in staging, use it in a maintenance window, then move on to the next one.

Each tool you build reduces operational risk by a measurable amount. More importantly, each tool becomes a building block. Once db-failover exists as a tool, the full-site-recovery runbook doesn’t need to include database steps. It just calls db-failover and moves on.

The end state is a platform where operational knowledge doesn’t live in documents but in services. Not because documents are bad, but because services are callable, testable, composable, and reachable by both humans and machines.

Humans still belong in operations; the machine should handle the repeatable work. Write the runbook once as code, and let the platform carry it.

The post Runbooks Are Dead; Make Operations Executable appeared first on ShiftMag.

For developers, that means model quality still matters, but benchmark scores no longer tell the whole story. As Igor Dmochowski (Developer Relations Manager, Nvidia) said at Infobip Shift 2026:

An agent is far more complex than a 2022-style chatbot: it can use tools, handle multimodal context, retain memory, and spawn subagents.

Hybrid architectures preserve useful state without making every turn more expensive

Longer context windows matter because agents need to remember more: past messages, documents, tool outputs, code, plans, and earlier steps. But as the context gets longer, self-attention becomes more expensive and harder to scale.

Dmochowski’s point was that context length is a real systems constraint. A model may accept a huge prompt, but still be too slow or too memory-hungry for a real application. The question now is what they cost to process:

The compute required to process your context was growing quadratically with regards to the context. This is not scalable.

The approach discussed on stage was a hybrid model that combines Transformer attention with Mamba-style state-space layers. Instead of using full attention for every token, the model uses more efficient layers for most of the work and keeps attention where it matters most.

That matters for agents because long context often contains information with very different levels of importance. “A production system needs enough capacity to preserve useful state without making every turn proportionally more expensive. Hybrid architectures are one attempt to move that curve in a more manageable direction,” Dmochowski said.

Models like MoE, active compute and throughput matter more than total parameter count

The second shift is sparsity.

In a conventional dense model, every token passes through the same large set of parameters. Igor explained that Mixture of Experts, or MoE, adds a routing mechanism that activates only a subset of specialized expert blocks for each token. The total model can contain far more parameters than are actually used during a single forward pass.

According to Dmochowski, this distinction between total parameters and active parameters is becoming much more useful when comparing models. A large parameter count can suggest capacity, but it says much less about serving cost once sparse architectures enter the picture:

We don’t need to put all of the tokens into all of the parameters of the model, we route them to the experts. For deployment, active compute, memory footprint, context length, KV cache behavior and tokens per second can be more informative than the headline parameter number alone.

Dmochowski connected that efficiency directly to capability. His point was that more throughput gives a system more room to reason within the same compute budget.

Agents can do more intermediate work and finish long tasks with lower latency. When reasoning uses thousands of tokens, throughput becomes part of the capability envelope, which is why multi-token prediction matters.

Lower-precision formats can reduce memory and compute requirements as well. Both techniques point toward the same goal: spend less hardware effort per useful token without destroying model quality:

The faster model is going to be the smarter model. There is obviously a trade-off, but producing two or four times more tokens with the same compute gives you room to make that trade-off.

For developers, efficiency is becoming an architectural feature

A model’s behavior depends on where you run it – the same model can act differently through a basic API than it does inside an agent harness with tools, memory, system instructions, and a loop that shows it the results of its actions.

A leaderboard or prompt benchmark only tells part of the story. An agentic app is a full system, not just a model. “Tool schemas, retry logic, context construction, memory strategy and the execution loop all influence the final result. Evaluation should therefore resemble the environment in which the model will actually run,” Dmochowski said:

The environment in which the models are set matters quite a lot. The behavior you get through an API can be completely different from the one you get within a harness.

The same idea applies during training and post-training: models should be trained in environments that simulate agent harnesses, and model choice and system design should be tested together.

Open models can be the better choice when private data has to stay in-house

Dmochowski also made a case for open models in domains where the most valuable data cannot be handed to a frontier model provider. Internal code, proprietary documents, regulated records and specialized workflows may require organizations to adapt models within their own environment:

Open weights, training recipes and reproducible tooling can matter most in exactly those cases, because the competitive advantage is often the private data and workflow rather than the base model itself.

The useful takeaway from Dmochowski’s talk is that the bottlenecks are moving. The early LLM conversation focused on scale and benchmark scores, but agentic systems force developers to think about long context, memory, generation speed, tool use, and the cost of every extra reasoning step. That changes how models should be evaluated: a smaller or sparser model can be the better choice if it keeps quality while lowering latency, and a huge context window only matters if the system can afford to use it.

The post LLMs Have a New Limit – It Costs More to Think Longer appeared first on ShiftMag.

Yesterday at Infobip Shift 2026, I listened to Apple Principal Engineer and CNCF Technical Oversight Committee member Katie Gamanji talk about the state of cloud native and its slow turn towards AI.

She said cloud native spent years becoming stable and unexciting, and AI teams now need that stability more than new tools. The big question is whether the people around that infrastructure can move as fast as the platform itself.

For teams already using Kubernetes in production, the next change is very concrete, Katie said:

Dynamic Resource Allocation, or DRA, reached general availability in Kubernetes 1.34, adding a native way to request GPU and TPU capacity for AI workloads.

Kubernetes is now officially boring

Katie started with a number that captures how far Kubernetes has come: “98% of organizations in the CNCF’s annual report said they had adopted cloud native technologies.” After a decade of development, Kubernetes has become mainstream:

Kubernetes is now described as being boring, and I think this is a wonderful achievement.

The problem appears when AI workloads enter that mature infrastructure. According to the figures Katie presented, 66% of organizations are already deploying AI workloads on Kubernetes. But only 7% are doing it daily, while 47% deploy them occasionally. Of the organizations using Kubernetes for AI, 23% have fully adopted the Kubernetes stack and 43% have partially adopted it.

That 67 to 7 gap points to one likeliest cause. The talk did not name the cause directly, but the pattern Katie described points to operational immaturity as the likeliest one:

Teams fine-tune an existing model rather than build one, deploy it once to prove it works, and lack the automated retraining pipelines that would treat the model as a dynamic component.

The 7% deploying daily, she noted, “most likely implemented an automated retraining pipeline that treats the model as a dynamic component rather than a static element.” Running an AI workload is not as simple as putting another application into a container. Teams need model artifacts, rollback strategies, and training pipelines. Kubernetes is being positioned as the shared platform, though whether teams adopt it that way is another question.

DRA and AI Conformance are making Kubernetes the AI platform

Katie compared this to DevOps: Kubernetes once brought developers and operations together, and now it could do the same for infrastructure teams and data scientists. But the analogy has limits – DevOps took years to work, and it is still not fully settled everywhere.

Data scientists, for their part, rarely want to operate Kubernetes; they want a model served. Still, a platform that can run both traditional workloads and training or inference jobs reduces the need for separate AI infrastructure.

Kubernetes is also adapting for this: DRA in 1.34 gives teams a more reliable way to allocate accelerators, which matters most for teams deploying daily.

The AI Conformance Working Group defines what a platform must support to run AI workloads reliably, with portability as the goal as Katie said:

If you have two conformant platforms, it’s going to be easy to lift and shift one product to the next platform.

Other initiatives target batch workloads, inference performance, and AI integration, and an Agent Sandbox effort is designing stateful, isolated runtimes for AI agents. The Serving Working Group completed its milestones and archived itself in February, continuing as a SIG on inference performance.

The list is still changing quickly. Katie expects much of this landscape to look different within six months to a year.

The cloud native playbook AI can borrow

Katie divided the emerging open source AI ecosystem into three areas: training, inference, and agents. Training turns data into a model. Inference serves it. Agents connect it to the outside world.

For platform leads, maturity matters. Training is led by the PyTorch Foundation, inference has many options, and agents are still young under the Agentic AI Foundation.

Cloud native experience can help AI tools mature faster. Security, observability, and identity were already solved in cloud native, and the CNCF project pipeline shows that kind of progress at scale.e.

It also prunes: 28 archived projects. For anyone choosing a stack, the archive list is a practical filter, and Katie argued archival is a healthy sign, letting maintainers redirect energy toward projects that earn adoption.

Teams that engage now will shape the patterns everyone else follows

For Katie, the next phase of cloud native is about whether the people building and maintaining the ecosystem can keep pace. She pointed out that contributing also means production feedback, feature requests, documentation, and white papers all shape projects.

But in the end Katie’s talk left the hard questions unanswered, but the data shows why:

98% of organizations trust Kubernetes with their infrastructure, two thirds have tried running AI on it, and almost nobody operates it continuously.

The specific bottlenecks, daily operational maturity, DRA adoption, and the conformance baseline, are still being defined, which means teams deploying AI on Kubernetes today are writing the patterns everyone else will copy:

All of the working groups’ meeting notes, invites, and repositories are public on GitHub. The teams that engage now, before the patterns harden, will not have to retrofit someone else’s choices in two years.

The post Only 7% of Organizations Using Kubernetes for AI Do So Daily appeared first on ShiftMag.

The easier it is to make something look finished, the more important it is to understand what the team actually decided.

That was the starting point for Hugo Raymond, Designer Advocate at Figma, in his talk at Infobip Shift 2026.

His main point was that AI becomes a problem when it fills in the gaps before decisions are made, because it can hide unresolved decisions.

A prototype shouldn’t try to look like a finished product

Hugo reminded us that the purpose of a prototype is to help the team test an idea. He compared it to a model airplane:

A model doesn’t try to be a real airplane. We use it to study specific characteristics of the airplane.

The problem, he argued, is that AI can now create a very convincing prototype even when the original idea is still incomplete:

In the past, gaps were easier to see. If a screen did not exist, a flow ended in a dead end, or the team had not defined a certain state, you could clearly see that nobody had made that decision yet. AI can now connect the flow, generate data, create loading and error states, and add behavior that nobody explicitly defined.

In the end, everything looks like one coherent product, even though the team designed some parts and the model simply filled in the rest.

Visual polish no longer proves that the team solved the problem

According to Hugo, this creates an especially important problem for developers: the more finished a prototype looks, the more authority we give it and the more likely we are to treat it as a specification that developers simply need to implement.

Hugo warned that a polished prototype no longer means that someone has thought through every edge case, business rule, or technical constraint.

One of his key points was that polish used to be expensive. If something looked highly refined, you could at least assume that someone had spent time thinking through the problem. Now, the tables had turned:

AI has broken that connection. The first draft can now look like the final iteration. Teams therefore need to make a much clearer distinction between what they have actually decided and what is still only an assumption.

Code is not the opposite of design

Hugo doesn’t think the solution is to move design back into static mockups, quite the opposite: code is not the opposite of design but one of the materials we use to design:

A code prototype can use real data, react to unexpected input, and show states that a designer did not draw in advance. AI has made this kind of experimentation much more accessible.

Still, code and a design canvas don’t show the same things.

Code is very good at showing what happens, while the canvas can do a better job of preserving why the team made a certain decision, which alternatives the team considered, and what the team still needs to solve.

Because of that, he sees a future where teams combine different materials depending on what they’re trying to learn.

AI speeds up execution, but not decision-making

The final and most important point of his talk focused on the limits of automation.

AI can take over a lot of repetitive work, speed up prototyping, and help teams test ideas more cheaply, but making something faster does not mean we understand what to make any better.

Hugo therefore makes a distinction between friction worth removing and friction that serves a purpose:

We can automate copying, waiting, and repetition. We should not automate comparing options, testing assumptions, discussing trade-offs, and questioning decisions.

As he pointed out, these moments help teams develop the judgment that AI cannot simply replace. “When almost every prototype can look finished, how convincing it looks is no longer the most important thing,” he said. What matters is whether the team can still clearly see what it has decided, what it has assumed, and what it still needs to solve.

The post AI speeds up prototyping, but teams still need to distinguish decisions from assumptions appeared first on ShiftMag.

Google DeepMind’s VP of Technology argued that one of the most important developments to watch is the rise of systems that can learn beyond the limits of human-generated training data.

He said AI has already gone through most publicly available human knowledge and is ready for the next step:

What it really needs now is to start generating its own experience and its own data. In the next year or two, that’s exactly where AI is heading.

That’s where recursive self-improvement comes in: AI could start learning from its own experience by trying problems, checking its answers, finding mistakes, and testing different approaches instead of depending only on human-generated data. Humans do this naturally, but AI still struggles with it for now.

AI can learn from experience, but monitoring has to improve too

Schillings pointed to AlphaGo as an early example.

AlphaGo is DeepMind’s system that beat one of the world’s best Go players, and Go is an ancient Chinese board game where the goal is to control territory on a grid. What made AlphaGo important was how it learned: by playing against itself and finding strategies that didn’t come from human games.

Humans spent centuries developing Go, and now they study strategies discovered by AI. So: who is teaching whom? AI and humans learn from each other.

There is, however, a major problem with the idea of AI learning from itself. What happens when the AI is wrong and who determines whether the data it produces is useful? This is particularly important because, as Schillings noted during his talk, models can sometimes find ways to appear successful without actually solving the problem they were given.

Cheating is a real problem. You ask a model to solve a problem, especially in code, and it will tell you, “I’ve done it, it’s there, it’s beautiful.” Then you check and find that it actually did not solve the problem. It may have stolen the result from somewhere else, or, even more amusingly, you ask it to write a tool for Linux and it says, “I wrote the tool,” when it’s really just invoking the existing tool and hiding the trace.

Sophisticated cheeting is becoming more and more an issue even for model to self-verify. So, we need to get the monitor to become more sophisticated also. That’s a part of the recursive self-improvement, he said.

AI needs step-by-step feedback

Another part of Schillings’ point is how AI systems are trained. Some problems are easy to check: a proof works or it doesn’t, code passes tests or it doesn’t. But real-world problems are usually messier than that.

Schillings compared it to teaching someone to climb a mountain with only one bit of feedback: you fell or you didn’t. That would be a terrible way to learn. A better teacher would give step-by-step feedback:

This part was good. Try this section differently. You’re getting closer. Here’s where you can improve.

AI systems are being trained to give this kind of step-by-step reward. That matters more than it first seems: if models can learn not just from success and failure, but from how close they were to a better solution, they can handle much more complex problems.

New systems may reveal things we haven’t noticed yet

What happens when AI starts exploring the unknown? This is where the implications become interesting.

Code is a relatively convenient environment for AI because there is usually a way to test whether something works. Schillings expects the coming wave of AI development to produce an increasing number of breakthroughs in areas such as physical sciences, biology, mathematics and engineering. The reason is straightforward.

If AI can generate hypotheses, test them, reject unsuccessful approaches and build on successful ones, it can start exploring the space of possibilities itself. That is a very different proposition from using AI as a better search engine. The ultimate value of these systems may be that they help us discover things nobody knows yet.

And that could turn out to be the most important AI story of the next few years.

The post Google DeepMind: AI’s Next Step Is Learning From Its Own Experience appeared first on ShiftMag.