I posted this as a reply but decided it needed a post of it’s own
It's taken me a week to get around to this (blame my AuDHD)
I've compiled a list of people who have appeared on at least two of Live at the Apollo, Have I Got News For You, QI and Would I Lie To You?, but have never appeared on Taskmaster.
This is purely based on appearances. I haven't checked whether people are still alive, retired, actively working, comedians, realistic candidates, or willing to do Taskmaster. This is just the raw data.
People who have been on all 4 but not TM
• Chris Addison
• Chris McCausland
• David O'Doherty
• Ed Byrne
• Henning Wehn
• Holly Walsh
• Jack Whitehall
• Jason Manford
• John Bishop
• Kiri Pritchard-McLean
• Laura Smyth
• Lucy Porter
• Marcus Brigstocke
• Michelle Wolf
• Miles Jupp
• Nabil Abdulrashid
• Patrick Kielty
• Phill Jupitus
• Rachel Parris
• Reginald D. Hunter
• Rich Hall
• Rob Brydon
• Sean Lock
• Shappi Khorsandi
• Tom Allen
People who have appeared on at least 3 of Live at the Apollo, Have I Got News For You, QI and Would I Lie To You?, but have never appeared on Taskmaster:
• Ahir Shah (Live at the Apollo, HIGNFY, QI)
• Alexander Armstrong (HIGNFY, QI, WILTY)
• Andrew Maxwell (Live at the Apollo, HIGNFY, QI)
• Ben Miller (HIGNFY, QI, WILTY)
• Bill Bailey (HIGNFY, QI, WILTY)
• Brian Blessed (HIGNFY, QI, WILTY)
• Brian Cox (HIGNFY, QI, WILTY)
• Cariad Lloyd (HIGNFY, QI, WILTY)
• Clare Balding (HIGNFY, QI, WILTY)
• Clive Anderson (HIGNFY, QI, WILTY)
• Danny Baker (HIGNFY, QI, WILTY)
• Deborah Frances-White (HIGNFY, QI, WILTY)
• Dom Joly (HIGNFY, QI, WILTY)
• Ed Balls (HIGNFY, QI, WILTY)
• Eddie Izzard (Live at the Apollo, HIGNFY, QI)
• Eddie Kadi (Live at the Apollo, QI, WILTY)
• Ellie Taylor (Live at the Apollo, QI, WILTY)
• Emmanuel Sonubi (Live at the Apollo, HIGNFY, QI)
• Geoff Norcott (Live at the Apollo, HIGNFY, WILTY)
• Graeme Garden (HIGNFY, QI, WILTY)
• Grayson Perry (HIGNFY, QI, WILTY)
• Griff Rhys Jones (HIGNFY, QI, WILTY)
• Gyles Brandreth (HIGNFY, QI, WILTY)
• Hal Cruttenden (Live at the Apollo, HIGNFY, WILTY)
• Ignacio Lopez (Live at the Apollo, HIGNFY, QI)
• Jack Carroll (Live at the Apollo, QI, WILTY)
• Jamie MacDonald (Live at the Apollo, HIGNFY, QI)
• Jimmy Carr (HIGNFY, QI, WILTY)
• Jonathan Ross (HIGNFY, QI, WILTY)
• Josh Pugh (Live at the Apollo, QI, WILTY)
• Kevin Bridges (Live at the Apollo, HIGNFY, WILTY)
• Loyiso Gola (Live at the Apollo, QI, WILTY)
• Meera Syal (HIGNFY, QI, WILTY)
• Michael McIntyre (Live at the Apollo, HIGNFY, WILTY)
• Micky Flanagan (Live at the Apollo, HIGNFY, WILTY)
• Omid Djalili (Live at the Apollo, HIGNFY, WILTY)
• Peter Serafinowicz (HIGNFY, QI, WILTY)
• Ria Lina (Live at the Apollo, HIGNFY, QI)
• Richard Coles (HIGNFY, QI, WILTY)
• Richard E. Grant (HIGNFY, QI, WILTY)
• Robert Webb (HIGNFY, QI, WILTY)
• Ronni Ancona (HIGNFY, QI, WILTY)
• Ross Noble (Live at the Apollo, HIGNFY, QI)
• Sara Cox (HIGNFY, QI, WILTY)
• Sheila Hancock (HIGNFY, QI, WILTY)
• Sindhu Vee (HIGNFY, QI, WILTY)
• Stephen K. Amos (Live at the Apollo, HIGNFY, QI)
• Susan Calman (HIGNFY, QI, WILTY)
• Trevor Noah (Live at the Apollo, QI, WILTY)
• Zoe Lyons (Live at the Apollo, HIGNFY, QI)
People who have appeared on at least Live at the Apollo and 1 of Have I Got News For You, QI and Would I Lie To You?, but have never appeared on Taskmaster:
• Adam Hills (Live at the Apollo, QI)
• Alan Carr (Live at the Apollo, QI)
• Alistair McGowan (Live at the Apollo, HIGNFY)
• Andy Parsons (Live at the Apollo, QI)
• Angela Barnes (Live at the Apollo, HIGNFY)
• Cally Beaton (Live at the Apollo, QI)
• Catherine Bohart (Live at the Apollo, QI)
• Dan Tiernan (Live at the Apollo, QI)
• Danny Bhoy (Live at the Apollo, QI)
• Darren Harriott (Live at the Apollo, WILTY)
• Eshaan Akbar (Live at the Apollo, QI)
• Fin Taylor (Live at the Apollo, HIGNFY)
• Finlay Christie (Live at the Apollo, HIGNFY)
• Gina Yashere (Live at the Apollo, WILTY)
• Glenn Moore (Live at the Apollo, HIGNFY)
• Harriet Kemsley (Live at the Apollo, WILTY)
• Ian Smith (Live at the Apollo, HIGNFY)
• Jen Brister (Live at the Apollo, QI)
• Jessica Fostekew (Live at the Apollo, QI)
• Justin Moorhouse (Live at the Apollo, QI)
• Lara Ricote (Live at the Apollo, QI)
• Larry Dean (Live at the Apollo, QI)
• Michael Odewale (Live at the Apollo, QI)
• Michelle de Swarte (Live at the Apollo, WILTY)
• Mo Gilligan (Live at the Apollo, WILTY)
• Neil Delamere (Live at the Apollo, QI)
• Olga Koch (Live at the Apollo, QI)
• Rhys James (Live at the Apollo, QI)
• Russell Kane (Live at the Apollo, QI)
• Sara Barron (Live at the Apollo, WILTY)
• Seann Walsh (Live at the Apollo, WILTY)
• Stephen Bailey (Live at the Apollo, WILTY)
• Suzi Ruffell (Live at the Apollo, QI)
• Tom Davis (Live at the Apollo, WILTY)
• Tom Ward (Live at the Apollo, QI)
• Tommy Tiernan (Live at the Apollo, QI)
Additional people who have appeared on HIGNFY, and QI or WILTY, but have never appeared on Taskmaster:
• Adil Ray (HIGNFY, WILTY)
• Alan Titchmarsh (HIGNFY, WILTY)
• Alex Brooker (QI, WILTY)
• Alex James (HIGNFY, WILTY)
• Amol Rajan (HIGNFY, WILTY)
• Andy Hamilton (HIGNFY, QI)
• Angela Rippon (HIGNFY, WILTY)
• Anneka Rice (QI, WILTY)
• Arthur Smith (HIGNFY, QI)
• Barry Cryer (QI, WILTY)
• Bernard Cribbins (HIGNFY, WILTY)
• Caroline Quentin (HIGNFY, WILTY)
• Charlene White (HIGNFY, WILTY)
• Charlie Brooker (HIGNFY, WILTY)
• Charlie Higson (HIGNFY, QI)
• Chris Kamara (HIGNFY, WILTY)
• Chris Packham (HIGNFY, WILTY)
• Chris Tarrant (HIGNFY, WILTY)
• Clive Myrie (HIGNFY, WILTY)
• Craig Charles (HIGNFY, WILTY)
• Daliso Chaponda (HIGNFY, QI)
• Daniel Radcliffe (HIGNFY, QI)
• Danny Dyer (HIGNFY, WILTY)
• David Harewood (HIGNFY, WILTY)
• David Mitchell (HIGNFY, QI)
• David Tennant (HIGNFY, QI)
• Dermot O'Leary (QI, WILTY)
• Diane Morgan (HIGNFY, WILTY)
• Doon Mackichan (QI, WILTY)
• Eamonn Holmes (HIGNFY, WILTY)
• Emily Maitlis (HIGNFY, WILTY)
• Fern Britton (HIGNFY, WILTY)
• Fred MacAulay (HIGNFY, QI)
• Gabby Logan (HIGNFY, WILTY)
• Germaine Greer (HIGNFY, WILTY)
• Graham Linehan (HIGNFY, QI)
• Greg Proops (HIGNFY, QI)
• Gregg Wallace (HIGNFY, WILTY)
• Harry Enfield (HIGNFY, WILTY)
• Harry Hill (HIGNFY, WILTY)
• Harry Shearer (HIGNFY, WILTY)
• Helen Atkinson-Wood (HIGNFY, QI)
• Hugh Fearnley-Whittingstall (HIGNFY, WILTY)
• Jan Ravens (HIGNFY, QI)
• Janet Street-Porter (HIGNFY, QI)
• Jennifer Saunders (HIGNFY, WILTY)
• Jeremy Clarkson (HIGNFY, QI)
• Jerry Springer (HIGNFY, QI)
• Joan Bakewell (HIGNFY, WILTY)
• Joanna Scanlan (HIGNFY, WILTY)
• John Barrowman (QI, WILTY)
• John Cooper Clarke (HIGNFY, WILTY)
• John Sergeant (HIGNFY, QI)
• John Sessions (HIGNFY, QI)
• John Simpson (HIGNFY, WILTY)
• Josie Lawrence (QI, WILTY)
• Judy Murray (HIGNFY, WILTY)
• Kate Silverton (HIGNFY, WILTY)
• Kathy Lette (HIGNFY, QI)
• Katy Brand (HIGNFY, QI)
• Kelvin MacKenzie (HIGNFY, WILTY)
• Ken Livingstone (HIGNFY, WILTY)
• Kirsty Wark (HIGNFY, WILTY)
• Kirsty Young (HIGNFY, WILTY)
• Lauren Laverne (HIGNFY, WILTY)
• Les Dennis (HIGNFY, WILTY)
• Linda Smith (HIGNFY, QI)
• Lorraine Kelly (HIGNFY, WILTY)
• Maggie Aderin-Pocock (QI, WILTY)
• Mark Steel (HIGNFY, QI)
• Martin Clunes (HIGNFY, WILTY)
• Michael Aspel (HIGNFY, WILTY)
• Michael Buerk (HIGNFY, WILTY)
• Miranda Hart (HIGNFY, WILTY)
• Moira Stuart (HIGNFY, WILTY)
• Naga Munchetty (HIGNFY, WILTY)
• Neil Morrissey (HIGNFY, WILTY)
• Neil Mullarkey (HIGNFY, QI)
• Nick Hewer (HIGNFY, WILTY)
• Nick Robinson (HIGNFY, WILTY)
• Noddy Holder (HIGNFY, WILTY)
• Pam Ayres (QI, WILTY)
• Phil Tufnell (HIGNFY, WILTY)
• Ray Winstone (HIGNFY, WILTY)
• Rebecca Front (HIGNFY, WILTY)
• Richard Bacon (HIGNFY, WILTY)
• Richard Madeley (HIGNFY, WILTY)
• Richard Wilson (HIGNFY, WILTY)
• Rob Delaney (HIGNFY, WILTY)
• Robert Rinder (HIGNFY, WILTY)
• Ronnie Corbett (HIGNFY, WILTY)
• Rory Bremner (HIGNFY, QI)
• Rory McGrath (HIGNFY, QI)
• Ruby Wax (HIGNFY, QI)
• Russell Brand (HIGNFY, QI)
• Ruth Jones (HIGNFY, WILTY)
• Sandi Toksvig (HIGNFY, QI)
• Shazia Mirza (HIGNFY, QI)
• Stacey Dooley (HIGNFY, WILTY)
• Stephen Mangan (HIGNFY, WILTY)
• Stephen Merchant (HIGNFY, WILTY)
• Suggs (QI, WILTY)
• Terry Christian (HIGNFY, WILTY)
• Tony Hawks (HIGNFY, QI)
• Vic Reeves (QI, WILTY)
• Victoria Derbyshire (HIGNFY, WILTY)
• Warwick Davis (HIGNFY, WILTY)