#067

AI Product Engineer Interview: Bring Your Agent, Not GitHub

· by Marian Kamenistak

AI Product Engineer Interview: Bring Your Agent, Not GitHub

“I know that I need LeetCode, system design.” That’s how a senior developer I mentor answered on 11 November, when we talked about his next interview. Let’s call him Pavel.

An AI product engineer interview checks if you can direct coding agents and prove that what they ship is right. LeetCode is a small part of that hour now.

Read the Product Engineer (AI Native) ad at FTMO in Prague: “Coding agents write most of the code, and we spend our time on specifying, verifying, and owning what ships.” And further down: “Your harness, shown. Agent configs, CLAUDE.md / AGENTS.md files, evals, test loops, MCP integrations or repos. We’ll ask to see it.”

When the ad reads like that, bring your agent setup, not your GitHub. The ad lists repos too, as one place a harness can live, but a repo of code shows what the agent can write. The setup shows what you stop the agent from doing, and that’s what they pay a senior for.

There’s a catch the ad doesn’t mention. A setup is easy to copy. AGENTS.md is used by over 60k open-source projects, and a curated list of Claude Code configs and skills has 57,674 GitHub stars as of this week. If I sat across the table from you, I’d skip the file and ask why each rule is there.

Your config is copyable. Your scars aren’t.

A year ago Pavel failed a big-tech interview on system design, and his plan was more of the same preparation. I told him about a Prague company that just created product builder roles. In the interview they ask you to show your CLAUDE.md.

His first answer was “I would ask for skills.” It’s the right instinct, with no history behind it yet. Next session he sits a mock interview with me, and the ten questions below are the ones I’ll ask him.

I grew the product engineering teams at Mews from 8 to 80, so I’ve sat on the other side of this table. Today I run my business through my own setup: a 206-line instruction file and 16 hooks on top of it, as of November 2026.

My best interview answer about it is a failure. The alerting I built to catch silent failures failed silently, for five days, and an agent made it worse before another agent found it. That’s question 5.

The job ad already tells you what the interview checks

I read a job ad as the interview script. Each line of FTMO’s “What you’ll do” and “What you bring” turns into a question in the room.

SignalThe ad saysWhat I’d expect them to askWhat to bring
Agent setup”Your harness, shown.”Walk me through itThe files, and why each rule exists
Verification”especially where money moves”How do you check agent code on a payment path?A test the agent can’t edit
Inherited code”Learn it fast with your agents, then make it yours.”What does the agent do in week one?A read-first plan
Production”monitoring, incidents, and fixes”Walk me through an incidentOne real incident, step by step
Vendors”payment and KYC providers”A vendor says success and the data is wrongThe check that caught it
Team tooling”Your agent setup, test loops, and conventions become the team’s.”What would you change in our setup?One rule you’d add, and the reason

Source: FTMO, Product Engineer (AI Native), read 10 November 2026.

None of the six rows asks you to reverse a linked list. The stack (TypeScript, NestJS, React, Google Cloud, Kafka) gets one line in the ad. Mews calls the same seat a product builder; the job boards call it product engineer.

FTMO asks to see the setup. Elicit and Canva watch you use it

FTMO is the one company I found that writes it into the ad. Others check the same skill live. At Elicit candidates work on their own machine, “using whatever tools you’re comfortable with”, and the interviewers score “Steering, not just prompting” and “Spotting risk”: “When is something safe to assume? When do you need to stop and check?”

Canva lets candidates use AI and checks one thing hard: “Can they identify and fix issues in AI-generated code?”

The strongest counter-voice comes from the company that makes Claude Code. Anthropic tells candidates for live interviews: “This is all you–no AI assistance unless we indicate otherwise.” It’s a fair position. If you can’t reason without the agent, you can’t judge the agent.

You’ll probably meet both kinds of round in one process, and the setup helps you in the no-AI one too. Pick one rule from your setup and explain, without the agent, what breaks if you delete it. If you can do that for five rules, the no-AI round is the same conversation with a whiteboard.

Your setup belongs to your employer. Show the rules, not the repo

Most of a year of agent work happens inside an employer’s code, and you can’t open that repo in another company’s interview. You don’t need to.

Marian Kamenistak smiling at the camera while typing on an open laptop at a wooden café table in a sunny courtyard, a cup of coffee beside the laptop.
The setup is yours. The code it worked on usually isn’t.

A personal mirror of the rules

Copy the instruction file and the hook list into a private repo with the client details taken out. The rules and their dates are yours; the code they protect isn’t. A team lead I know built a loop that writes new rules into his team’s instruction files, all inside his employer’s repos. My advice to him was to publish the loop as a public skill that points at none of them.

A change log of the rules

One line per rule change: the date, what broke, what you added. It’s the most convincing page you can bring, and it holds no employer code at all.

One rule, rebuilt live

Offer to rebuild one hook or one test from scratch in the room. It takes ten minutes and proves the rule is in your head, not only in a file.

The 10 AI product engineer interview questions, and what a strong answer shows

Each question maps to a line in the ad. For each one: why they ask, what a strong answer sounds like, and the red flag that costs you the offer.

1. Walk me through your agent setup

They wrote “Your harness, shown”, so open the files instead of describing them. Show the instruction file, what’s in it and what you deleted from it. A long file the agent half-follows is worse than a short one it obeys.

Then show the history, because that part can’t be copied. Most rules in my file carry a date and the incident behind them, from the Python certificates in August to the headless Chrome rule in October.

In a dry-run interview in September, the question a candidate I mentor couldn’t answer was the AI one: which models his team used, and how they tracked usage and traffic. Let’s call him Marek. His own notes from that session say “failed to answer”. My advice was to name the tools, the telemetry and what he checks himself.

Red flag: your file looks like a template, and you can’t say which rule came from which mistake.

2. Which rule did you move from the instruction file into a hook?

There’s a line in my file for this: “CLAUDE.md lines are requests, hooks are rules.” The agent reads the file and mostly follows it. A hook runs before the tool call and blocks it, whatever the agent decided.

Mine moved in two steps. In August 2026 I moved my business files out of Google Drive onto local disk, and Drive’s trash went with them, so a hook now asks before any rm on a glob or a variable. On 17 September 2026 the Gmail connector turned out to rewrite every link in my drafts into a google.com tracking redirect. The same day it became a deny hook on the Gmail write tools.

Red flag: every rule lives in prose and you trust the agent to read it.

3. What does your agent never do without you?

Elicit describes the opposite: candidates who “yolo claude --dangerously-skip-permissions” and accept every suggestion without reading the diff. A strong answer is a short list with a reason per item.

Mine: no deletes outside scratch folders without asking me, no deploys of client folders, no email through the connector, no merge of a blog post. A pull request is where my pipeline stops, and I merge it myself. In a product repo the same list usually reads: database migrations, payment config, secrets and production deploys.

Red flag: you click “yes” on every permission prompt without reading it.

4. How do you verify agent-written code where money moves?

Stripe says it best in a post on AI-built payment integrations: “Implementing the PaymentIntent is getting cheap. Being right about the business intent behind it is now the whole job.”

I haven’t shipped a payment path with agents, so I won’t pretend I have a scar here. The shape of a strong answer is clear anyway: it separates the checker from the writer. You write the business cases yourself (a refund issued twice, a currency mismatch, a webhook arriving out of order), and the agent can’t edit those tests to make them pass.

Red flag: “The agent wrote the tests too, and they’re green.”

5. Tell me about a time your checks passed and the system was broken

Mine: for five days in August 2026, my failure alerts were dead and I didn’t know.

It started with the opposite bug. Until 16 August all nine of my scheduled jobs reported success even when they failed, because the last line of each script was a log line, and that line always succeeded.

So an agent and I added Slack alerts to all nine. The agent wrote the helper, and so that a broken alert couldn’t kill the job, it swallowed every error. I approved it. It looked careful.

The Python on my Mac had never been given its certificate bundle, so every https call it made died, quietly. curl uses the system certificates, so everything else worked and nothing looked broken.

On 21 August one agent session hit that certificate error and routed around it with a different library, then moved on. A second session, building a new health check the same afternoon, went after the root cause and found that the alert helper had been dead since the day it was written. My note from that day: the mechanism built to end silent failures was itself failing silently.

I never found out whether a job failed in those five days. The fix is the answer they’re listening for. Preflight, don’t fall back: a job that needs the network checks for the certificate bundle first and exits non-zero if it’s missing. And fix the machine, not the one script, because the workaround leaves the trap armed for every other script.

Red flag: you’ve never had one. After a year of directing agents, “never” usually means you weren’t looking.

6. You inherit a live codebase an external partner built. What does the agent do in week one?

This is FTMO’s real situation: “The first version is live with real users, built together with an external partner.” A hiring guide from jobsbyculture names the skill: “they can hold a foreign codebase in their head long enough to make a useful contribution. That’s most of the job.”

In week one the agent reads and you write notes. Map the money paths, the vendor calls and the places without tests, then write the instruction file for this codebase from what you found. Your first commit is a test.

Red flag: “I’d ask the agent to refactor it.”

7. Write the spec before the agent writes a line

Specifying is the first verb in FTMO’s sentence. Elicit’s interviewers noticed how rare it is: “Plan-first thinking is rarer than we expected.”

Marek described his team’s path to me in October. First the agents wrote the code. Then they added AI code review, with a human after it. Then testing became the bottleneck.

“Then we look all the way back to generating of the specification … we have to have the good spec.” They ended where FTMO’s sentence starts.

A strong answer is a half-page in the room: who the user is, what changes for them, what must not happen (the money cases), and how you’ll know it worked.

Red flag: you open the agent in the first minute.

8. It broke in production. Walk me through the next hour

The ad answers who’s responsible before you do: “You own your features in production: monitoring, incidents, and fixes, not a hand-off to someone else.”

Mine is from 25 October 2026. I deployed three new pages to this site, checked them, all 200. About an hour later all three returned 404. The deploy list showed two deploys I didn’t make: another agent session on the same repo had deployed an older copy of the site over mine.

My own push hadn’t reached the server either, and my check said it had, because both copies it compared were old. The fix is a rule now: push first, confirm the commit is on the main branch, deploy from there, and check the new pages again 30 to 60 minutes later when other sessions are active.

Red flag: “The agent wrote that part.”

9. A vendor call fails and everything still says OK. How do you find out?

FTMO’s list is a trading engine, market data, payments and KYC, so a quiet vendor failure costs money. A strong answer is a story where the vendor said no and your system said yes.

Mine is from 30 August 2026. Attio, my CRM, rejects the whole record when one field name is unknown. A rename and four new fields shipped, writes from my mentoring pipeline started coming back 400, and the code caught only network errors, so it logged nothing and told the booking tool “ok”. The list of new inquiries sat at zero entries while the offer emails went out perfectly.

Now every write goes through one helper that checks the status code and logs an error, and the daily health check pages me on that error.

Red flag: “We trust their SLA.”

10. How do you know your agent setup is getting better?

The ad asks for evals and test loops. The best answer I’ve heard came from the team lead with the rules loop. Let’s call him Ondřej.

His loop reads his team’s agent sessions and pull-request reviews. When a reviewer leaves the same comment on five pull requests in a row, it proposes a rule and opens a pull request against the instruction file. He ran it over six months of review history.

A human still reviews every one of those pull requests. And once every month or two they read the file “with fresh eyes” and cut, because it can’t grow forever. That’s a setup getting better, with the evidence in the git history.

Red flag: “It feels faster.”

Build the 15-minute walkthrough before the interview

Plan 15 minutes, three per part. Fill in the right column with your own setup, then say it out loud once without the agent.

Part (3 min each)What to showMine, as an example
The filesInstruction file, skills, configsOne global file of 206 lines, plus one per project
The stop listWhich rules are hooks, and whyThe rm guard and the Gmail deny hook
The loopHow a change gets checked before it shipsThe build fails on a missing image, a post without FAQs or the wrong analytics key
One failureCaught or missed, with a dateThe five days of dead alerts, August 2026
What’s nextOne gap you know aboutMy jobs still run on one Mac, so a dead laptop is a dead pipeline

If your setup is thin, you have seven days

Use dates from mistakes you really made. The interviewer will ask what happened that day.

DaysDo this
1-2Write the instruction file from your last three mistakes with the agent, one dated rule each
3Turn the most expensive one into a hook
4-5Write one test the agent can’t edit, on the path where money or users would get hurt
6Write up one failure with the date and what you changed
7Walk through it out loud, no agent, against a timer
Wide shot of a 1:1 community mentoring session in an atrium, two people in armchairs at a round table.
A 1:1 at a community mentoring zone.

It works the other way too. Another mentee, let’s call him Radek, screens employers before he applies, with one question in his notes: “Will I be able to use AI as an everyday work lever?” Ask them to show you their setup as well.

Pavel has his week. If you want to rehearse that hour before it counts, that’s what we do in staff engineer mentoring: we open your setup together and I ask you the ten questions.

So, if you had to open your setup in front of an interviewer tomorrow, which file would you close first?

Frequently asked

What interview questions should I prepare for an AI-native product engineer role?

+

Expect questions about how you work with agents more than about algorithms: a walkthrough of your agent setup, which rules you enforce with hooks, how you verify agent-written code on payment paths, how you take over a codebase you didn't write, how you run an incident, and how you know your setup improves over time. FTMO's Prague ad for a Product Engineer (AI Native) lists all of these and says it will ask to see your harness.

How do I show my coding agent setup in a job interview?

+

Plan 15 minutes in five parts: the instruction files and configs, the stop list you enforce with hooks, the loop that checks a change before it ships, one failure the setup caught or missed, and what you'd change next. Templates for agent configs are public, so the part that convinces an interviewer is the history: the date and the incident behind each rule. If your setup lives in an employer's repo, bring a private mirror of the rules or a dated change log instead of the code.

What do companies ask engineers about verifying AI-generated code?

+

Canva asks whether candidates can identify and fix issues in AI-generated code. Elicit scores whether you spot risk and know when to stop and check. In fintech the question gets sharper: FTMO wants tests that catch what the agent got wrong, especially where money moves, and Stripe writes that being right about the business intent is now the whole job.

Who is responsible when AI-generated code breaks production?

+

The engineer who shipped it. AI-native teams put it in the job ad: FTMO's product engineers own their features in production, including monitoring, incidents and fixes, with no hand-off. Saying the agent wrote that part is one of the fastest ways to fail the interview.

What does a product engineer do?

+

A product engineer carries a feature from the problem to production and answers for what it does for users. In AI-native teams the coding agents write most of the code, so the work moves to specifying, verifying and owning what ships. Mews calls the same role a product builder.

New posts, straight to your inbox.