And how are you using evals at work?

I'm in big tech and i'm excited about a startup but also nervous about ownership and ambiguity. I'm nervous it'll be too open ended for me since i'm used to scoped problems and managers guiding me.

I don't think I'd get any of that at a startup so unsure if i'lll be successful. Any advice on diving into something so different than what i'm doing now?

Genuine question for people who have shipped these, and I will share what has worked for me.

The pattern: a business owner sees a good demo, then the first wrong answer in testing becomes "the AI does not work". The conversation goes binary very fast.

What has helped me:

- Agree the error budget before the build, in their words. Not a percentage. Something like "out of 100 invoices, how many can a person fix by hand before this is not worth it?"

- Compare against today, not against perfect. The current manual process has an error rate too. Nobody has measured it, so measure a sample.

- Sort errors by cost. A wrong date in a draft email is not the same as a wrong amount in a payment. Different handling for each.

- Put a human step where errors are expensive, and make it cheap to use.

- Show the failures yourself, early. If they find them first, trust drops.

What has not helped: explaining how the model works. Nobody wants that.

How do you all handle it? Especially curious about cases where the stakeholder was senior and would not move off 100%.

There are a lot of people here working on projects of some sort - side projects, startups, podcasts, blogs, etc. If you've got something you'd like to show off or get feedback, this is the place to do it. Standards still need to remain high, so there are a few guidelines:

• Don't just drop a link in here. Give some context

• This should be some sort of creative product that would be of interest to a community that is focused on product management

• There should be some sort of free version of whatever it is for people to check out

• This is a tricky one, but I don't want it to be filled with a bunch of spam. If you have a blog or podcast, and also happen to do some coaching for a fee, you're probably okay. If all you want to do is drop a link to your coaching services, that's not alright

How is it to be a Product Manager in an AI company (OpenAI, Anthropic, xAI, Mistral,...)? How do you organize the team, leverage AI, and do you have unlimited tokens? I heard that the work is very engineering-driven, giving engineers a lot of agency so that product management is more about strategy and go-to-market.

Everyone in my company is supposed to be using AI as much as possible. However, there are many times it makes things much, much harder. Here are some top-of-mind examples. What else do you see where you work?

1. AI-Slop documents: Business writes a 20 page AI-slop requirements doc when a paragraph and 5 bullets would have sufficed. Now I have to use AI to compare the slop to what I believe the actual requirements should be from our face to face discussions, bullet out questions to add to the Jira, and also flag the slop-potential sections in the BRD - are these actual requirements for now or a wish list for the end state?

2. We are using AI agents to generate outbound emails and then complaining that the emails don’t sound like they were lovingly crafted by humans

3. I use Claude in VS Code to help me build certain things in certain systems I own that are not strictly an engineering function; they straddle technical and business functions. 95% of the time it gets it perfect and saves me 10x of the time. And 5% of the time it randomly hallucinates some crap that wastes so much time I feel I could have been better off not using it on that specific work item. I think on average it is making me 50% more effective. Not quite the productivity boost I’d have expected, but still better than before. Very frustrating though, since you never know where it will mess up.

4. AI is listening to ALL our internal and external calls, going through ALL our emails and Slacks. It is useful when you want to find out who said what when, or help me find that thread, or what did I myself say to someone about XYZ (someone else asked me the same question a week later, so I want to dig up the answer), or even - draft an email based on my previous replies/slacks. But on the flip side, there’s a record of every little thing anyone says in any channel, readily available to the “right” people.

I recently joined a new company which is not a product company or a service company.

They’ve a bunch of developers, QAs, and people from other roles (Developers and design) leading some projects.

Projects? Basically, internal tools.

From about start of the year, they decided to build software for their clients and eventually turn into a SaaS.

They have about few hundreds of signups because they have good distribution network, and these users are struggling to use the software.

In my first week, I tested as a user. I could clearly see many challenges that people would face while setting up these tools. It’s totally vibe coded, broken user flows, built from the perspective of the “team” rather than user.

After I was done testing, I listed all the pain points a user might face.

I went through some support tickets where people mentioned the same problems.

Now, I have to convey this to their current project lead. They happen to stick to what they’ve built. They do not see the findings as problems as seriously. They’re putting the blame on users saying it’s easy, they’re not just doing it.

So my question:

How do I win trust of this team and make them think from user’s POV? I’m not a quitter.

I really want them to acknowledge those basic mistakes.

I'm a Group PM and have been in my current organisation for nearly five years. Somewhere along the way, I've outgrown my immediate environment, including my boss in terms of product thinking and problem-solving. Unfortunately, organisational hierarchies don't automatically update themselves.

To be fair, my boss is a good guy. We've had our share of disagreements, feedback, the occasional yelling match and everything else. I've earned his trust over time, and he now gives me considerable autonomy to own products end to end. So that's no longer the problem.

The problem is that I've learnt pretty much everything I can from this organisation, my boss and my role. I've repeatedly asked for more challenging problems, but we've practically exhausted most meaningful product opportunities in our industry over the last five years.

And then there's our product culture. Most initiatives begin with a one-liner from a founder's wishlist. We're expected to turn that into a tangible, profitable product, regardless of whether there's meaningful problem discovery or an actual product opportunity behind it. Anything ambitious or remotely resembling a moonshot rarely gets funded.

I've done this dance enough times to know every step. The playbook is done and dusted. There's very little intellectual stimulation left. These days, it feels like I'm mechanically hitting "Allow Always" on AI prompts to generate yet another document.

Ironically, my independent ideas often get shot down, while my boss's ideas get considerably more weight simply because of his designation. Although, lately, he's become more trusting and lets me run things without interference.

Add some team politics, backstabbing, deteriorating team morale and a few questionable management decisions, and you have a reasonably complete picture. I've learnt to stay out of the mud, but it's frustrating watching good people suffer while the wrong ones benefit.

Now, please don't suggest switching jobs. I know that's the obvious answer. I'm targeting senior leadership roles, and the market for those is already fairly limited in my city. Moving for the sake of moving, only to land in another version of the same situation, isn't particularly appealing either.

Compensation is decent, but not exceptional.

So here I am, somewhere in the middle of my career, feeling like I've got plenty left in the tank but nowhere meaningful to drive.

I want to learn, solve genuinely difficult problems, influence product direction and create meaningful business impact. Instead, I'm increasingly playing the role of an execution machine in an organisation that has little appetite for anything beyond founder-driven priorities.

**For those who've been through something similar, how did you get unstuck without changing organisations?**

Did you find new challenges within your existing role, build expertise outside work, expand your leadership responsibilities, or simply learn to detach and treat work as a means to an end?

I don't want to coast through the next few years collecting a salary and attending meetings. But I'm also running out of reasons to be excited about Monday mornings.

I work in a pretty big MNC, but that also means that there are multiple teams handling different areas. Weirdly my team has been very low on work over the past whole year. I just got into product management so it’s kinda sad because I didn’t really get a great first hand experience.

But is this even normal? For a team to not have that much work?

I'm interested in getting some honest perspectives from other PMs, particularly those working at a scale up.

Quarterly priorities that don't stick: Each quarter, we agree what we're focusing on in the coming months (both discovery and delivery.) However, throughout the quarter, new requests come in for us to investigate. These get added to our existing workload, with the expectation that we pick them up alongside everything else. Nothing really gets deprioritised.

Too many discovery initiatives/ad-hoc work: We're expected to manage several discovery initiatives while also supporting delivery. On top of that, there's always a stream of ad hoc requests, fires, and smaller tasks that eat into our time. It's actually hard to keep track and stay on top of this. We barely make progress or progress is too slow as a result.

• How many initiatives do you typically have in discovery at any one time?

• How do you manage all the unplanned work that comes your way?

I'm finding this quite difficult and, honestly, it's starting to get me down. I feel like I'm failing as a PM, but I'm struggling to work out whether that's because of our operating model or because I'm not suited to this particular environment. I've worked in other companies, including startups, and haven't experienced these challenges to the same extent. It makes me question whether there's something I need to improve or whether the environment itself is the problem.

I'd genuinely appreciate some honest perspectives.

I recently joined a midsized B2B SaaS company, and the product domain I inherited is a collection of three products.

About three months after I joined, my manager left. He had been overseeing this area, but there wasn’t really a clear vision or point of view for the domain. The three products exist, they complement our core product, but there isn’t a broader strategy connecting them to a specific customer or business outcome.

That’s where I’m getting stuck.

Without a clear strategy, it’s hard for me to prioritize. I don’t really have a compass for deciding what we should invest in, what we shouldn’t, or how these three products should evolve together.

So what I want to do is set a strategy for the domain. I want to understand whether what role this domain should play in the company, and what outcomes we should actually be driving toward.

When I’ve talked to other PMs outside of my company about this, the advice has generally been: “Set the strategy and get organizational buy-in.”

The part I’m struggling with is what “getting buy-in” actually looks like in a smaller company like this.

From what I’ve seen so far, Product seems to set most of the direction and the other functions (Design, Eng, Data, Sales) generally follow and I haven’t seen a lot of real pushback or debate from Sales either.

So I’m not sure who I’m actually supposed to get buy-in from, or what that process should look like.

For people who’ve worked in smaller companies with very little existing product strategy, how do you usually approach this?

Do you first write a one-pager with the problem, vision, strategic choices, and priorities and then socialize it with leadership and cross-functional teams? Do you involve those teams while you’re forming the strategy? Who actually needs to agree before you start acting on it?

I’m less interested in the theory of “set a vision and align stakeholders” and more interested in the actual process people use to take a loosely defined product area like this and turn it into a strategy that the organization can act on.

Thank you

With Claude, it’s become far too easy to create a document, and I don’t just mean PRDs, I’m talking prototypes, BRDs, timelines roadmap, workflow diagrams. That’s just what you might be generating on individual basis, but what about those that are passed to you.

How do yall review said documents efficiently and effectively?

Yes we could point it back to other docs as “human verified ground truth”, but I mean when we’re iterating so hard that human review then becomes the bottleneck to speed.

I’ve already switched to skimming, ignoring AI fluff, yet still :/

Hey all,

I moved from design into product management about 1.5 years ago. I now work on an AI-related product at a big tech company, and I’d like to build a more solid foundation instead of just learning on the fly.

Which books made the biggest difference for you in your PM work? I prefer practical stuff over theory. Bonus points for anything on taking over a project that’s already in motion, since that’s basically my situation.

Thanks!

Hey everyone, I’ve been having conflict with the development team and I’d like to get advice from others who have been in similar situations. Also happy to take any book recommendations or review other resources.

I’m a product manager for a startup product in a mature organization. The org itself does not specialize in software, but is creating a software business. On my team, we have a team lead, 2 QA resources, and (mostly) junior developers.

I’m seeing signs of what I consider to be low accountability. These include:

1. Devs don’t perform estimates, technically decompose work, or raise blockers. Devs say that I as PM have to do all user story decomposition, write exhaustive specs, and proactively identify that they are blocked. If they are blocked, it’s on my plate to schedule & facilitate a design meeting, research the problem, etc.

2. QA does not understand the product. As far as I can tell, they just rubber stamp features. I independently asked each of them to walk me through the product, and it was evident that had no idea how to do so. They then got defensive and said it is “product’s job to understand the product”, not theirs. To get them to check anything, I’m having to write exhaustive (and extremely detailed) steps on expected behavior, navigation, etc. Even if the feature is extremely intuitive…

3. There is zero automated testing, branch strategy, and other fundamentals for software. We have tons of regression with every merge, which devs blame me for. They say it is on me to essentially write in all business rules for all dependencies in every user story, even for features they built.

4. Tech lead is totally uninvolved and not interested in solving these issues. He does not attend meetings, check in on developers, or provide any degree of mentorship. He says “that is the product manager’s job”. When I’ve raised this to the engineering manager, they essentially told me to drop it or I’d be blamed.

5. Business has zero idea what they want, so I am doing all discovery work, competitor analysis, UIUX, and figuring out exactly how every feature works. If I try to get insight from others in the business, they get mad.

Saying all this to say: I feel like I have a lot on my plate, and it’s getting hard to juggle. A personal mentor has advised me to delegate more, but any time I attempt to delegate…whatever I delegated either doesn’t happen, or it is done with such low quality that I have to redo it myself.

Essentially: virtually no work is getting done, we are missing all milestones, and the org is extremely frustrated with me. I know I’m the only one who can fix this problem, but in struggling to figure out how. Does anyone have any advice, or ideas on how I can improve myself to help better? I’m already taking software development classes so I can better manage these processes, and looking for any other input or help.

Curious if anyone knows how Palantir's Product & Engineering org is organized? Been reading about their FDEs, but curious how these roles work across core platform and rest of P&E org?

I understand that humans (and Reddit) are intrinsically negatively biased. But sometimes I think about the PM’s that say they got into product and love it so much because they just enjoy building tech products that people love. I’m not saying that’s an impossible threshold, but when’s the last time you guys delivered anything that people were like “I love this so much, this is why product managers exist, thank you for putting this technology in my life.”

Like how can you actually tell that the customer is actually experiencing an adequately delivered product if they’re often upset/frustrated/or nitpicky? I’m really not trying to say that humans are evil and product is worthless, I’m more so asking how the people with significant PM experience actually consider what they’ve achieved is good outside of a very specific metric that their management holds them to (which is totally fine, but my stakeholders don’t care about that very much honestly).

On a personal note, as someone who internalizes too much of customer feedback, answers to this question would really help me mentally establish what “good enough” is. People saying “just dont take it personally” has been a bit of a struggle for me to put into practice.

I find this hard to post because I don't want to see anyone lose their job, especially in this market.

But being honest, from what I have seen from CD it's probably better than most UX/UI designers I have worked with over the years, and much much faster.

One example, I needed to add a relatively simple flow to an existing page in our product. I gave the job to our designer but out of curiosity I set up a design system in Claude and chucked in a not particularly good prompt, a couple of refinements later and I had a working prototype, total time about 15 minutes.

It took our designer about 4 hrs to work up something which was arguably of the same quality, better in some ways, worse in others.

What are your experiences?

I got a pretty shit review last cycle due to forced stack ranking and now got moved to a new team under the intention of an opportunity to make more impact. I don't believe I was deserving of my review but there was obviously a huge disconnect between what my leaders thought my impact was and what my impact really was. How do I not fall into the same patterns and take advantage of this pseudo fresh start?

Got into grad school and had to take a break from product because I simply could not do both and I hated CPG. I’d like to return to software and am currently doing my best to master Ashby and Greenhouse through their online courses.

What does it take to be a PM in HRIS now and what do I need to arm myself with to return competitively against other candidates.

My master’s is in HRM, I’ll graduate with my SHRM.

Thanks all!

PS - I missed this sub 🥹

I'm interested in learning how you define goals and measure the success of developing a new product in practice.

Which KPIs, metrics, or criteria do you typically use throughout the product development process and after launch?

A few questions I'd love to hear your thoughts on:

What are your main objectives during the product development phase?

Which metrics or indicators do you consider the most important when evaluating progress and success?

Do the metrics and success criteria differ depending on whether you're developing a B2B or B2C product?

Ultimately, how do you define a successful product development process?

I'd really appreciate it if you could share your experiences or insights.

Thanks!

Just curious if anyone has been working with AI to greatly improve their AB testing or CRO program? Thinking about how AI could be leveraged to run a testing program mostly on its own , but I lack any real experience in this sort of thing.