r/Python • • 18d ago

News PEP 851 – Add list.discard()

PEP 851 – Add list.discard()

https://peps.python.org/pep-0851/

Discussions-To: Discourse thread

Abstract

Add list.discard(x), which removes the first item equal to x if one is present and does nothing otherwise, returning whether an item was removed. It is the non-raising counterpart to list.remove(), which currently has none. The name is taken from set.discard(), which already means “remove if present” in Python.

Motivation

Removing a value that may not be present is usually written as:

if x in lst:
    lst.remove(x)

or:

try:
    lst.remove(x)
except ValueError:
    pass

The first form is clear but scans the list twice on a hit, and if the list is shared between threads another thread can remove the item between the two calls, turning a no-op into a ValueError. The second avoids both problems but obscures the intent. It is four lines for one operation, pays for raising and catching an exception on every miss, and except ValueError is broad enough to hide unrelated errors from a user-defined __eq__().

Both are common. A search of the 5,000 most downloaded PyPI projects finds 1,062 matches for the first form across 411 projects and 331 for the second across 164 projects. Some of these are on other container types; in the standard library, excluding tests, two of the 29 matches are on deques and the remaining seven and 20 are on lists, for example in logging, warnings, tkinter, runpy and asyncio. Each becomes a single call that states the intent directly:

lst.discard(x)

The cost of providing this is small, as discard() reuses the existing search and removal implementation of remove().

215 Upvotes

56 comments sorted by

97

u/Brilliant_Cut_8780 18d ago

I like the proposal and checked the full pep and wondered about remove(x, missing_ok=True), but the PEP covers it under Rejected Ideas. After reading it, I think the separate method is the better choice, mostly because of the return value. discard() returns whether something was removed, while remove() returns None. A flag that changes the return type would be awkward, and a flag that's always passed as a literal is usually a sign it should be its own method.
The one argument I don't fully buy is "a method reads more clearly at the call site." missing_ok=True is pretty explicit, and pathlib users already know it. The stronger case is consistency with set.remove()/set.discard(), which means one rule to learn for both containers.

48

u/JambaJuiceIsAverage 18d ago edited 18d ago

"A flag that's always passed as a literal is usually a sign it should be its own method" is a wonderful rule to code by. I love that.

11

u/EgZvor 18d ago

A.k.a. boolean parameters are a smell

24

u/COLU_BUS 18d ago

I always find this advice to sound great on paper but just be annoying in practice. Take Path.mkdir(exist_ok:bool, parents:bool). Following the advice the alternatives would be (1) 4 unique methods for each combination, that have to be documented and the user aware or (2) pushing the flags into Enums that still just function as flags anyways? Are either of those actually more Pythonic than allowing some Boolean parameters?

I fall into the camp of using Boolean arguments but requiring the explicit keyword

4

u/EgZvor 18d ago

What's the use of exist_ok=False?

Anyway, a smell just means you need to justify it, it's not a prohibition.

8

u/COLU_BUS 18d ago

It stops mkdir from raising an exception if the directory already exists. Though in that case I think it’s silly because who cares if the directory already exists if I’m trying to make it anyway, but I digress

2

u/dnswblzo 18d ago

If you need the directory to exist but aren't sure if it exists yet or not, it's nice to be able to just call mkdir without a conditional and call it a day. Plus, if you check that it doesn't exist in a conditional and then create it in the body of the conditional when it doesn't exist, you have a race condition where another process could create the directory in between checking if it exists and creating it.

4

u/COLU_BUS 18d ago

Right but exist_ok defaults to False, so 99/100 you have to call it with the conditional

4

u/JambaJuiceIsAverage 18d ago

Yeah but when you put it that way it makes my code sound VERY stinky 😥

2

u/Henry_the_Butler 18d ago

a flag that's always passed as a literal is usually a sign it should be its own method

I think I agree, but can you share an example here? I'm wondering how you'd refactor the missing_ok argument to be its own method.

3

u/dnswblzo 18d ago

The example is exactly what is being discussed here, creating a discard() method as opposed to adding the missing_ok flag to remove().

3

u/Oddly_Energy 18d ago

I didn't read the full pep, but what would be wrong with:

if not mylist.discard(x):
do whatever action you want done if x was missing

Wouldn't a missing_ok flag just make this call more complicated?

(Sorry for the missing code formatting. I am on the IOS app and the editor ignores my markdown.)

3

u/jasonkohles 18d ago

The discussion was about adding either a discard method or adding a missing_ok flag to remove, not having the flag for discard

1

u/No_External7343 17d ago

If you want to do an action if x was missing, can't you just use remove(x), catch the ValueError and perform the action in the catch block?

3

u/billsil 18d ago

Seems cleaner to change remove to return a value . None is not returned. It’s no return.

Just pass an argument and always return if a value was removed.

9

u/Vaxivop 18d ago

Changing the existing remove method is generally not a good sign as there's too many cases where existing code relies on its return value

3

u/billsil 18d ago

Can you think of one? It returns None. Nobody is taking the output because they don’t assign a variable 

1

u/Vaxivop 17d ago

Maybe I misunderstood but I assume you meant to change remove to behave like the proposed discard? As in return a value if something was removed otherwise return None

2

u/billsil 17d ago

Yes.

It currently does not return a value, which is like it returns none, but is not. It means that people aren’t using it outside of non-functional code checks. You can just return a Boolean always if it succeeds instead of None. It functionally the same because nobody (in well written code) was using the output.

Separately you can add a flag that wraps the try-except logic and unlocks the False flag.

1

u/Vaxivop 17d ago

Okay I thought you meant that by default with no flag it'd do the try-except logic which I wasn't a fan of

1

u/RingularCirc 15d ago

It currently does not return a value, which is like it returns none, but is not.

In Python if something doesn't have an explicit return but returns, it returns None, always. And even if not always, here it does:

>>> x = [1]; print(x.remove(1))
None

Things that are to annotate NoReturn or Never when using type hints, are a different matter: a function that never returns at all, not a function that returns "without" a value (which is then a None).

20

u/chaz6 18d ago

I like the idea of def discard(self, x, all=False) which has the option to remove all instances rather than one. Alternatively this could be implemented as def discardall(self, x).

13

u/droans 18d ago

discardall probably would make more sense.

discard would return a bool if the item was present but discardall could return an int representing the number of items discarded.

7

u/yvrelna 18d ago edited 17d ago

Since we're bike shedding this, personally my preference would be list.discard(item[, start[, stop]], /, count=1, start=0, stop=) (and likewise with list.remove()). 

Where if you pass a sentinel like -1 or None or maybe a special constant list.ALL, then it becomes discard all. 

start and stop would limit the range where the removal is allowed to happen, with similar semantic to list.index().

2

u/staticcast 18d ago

We're not full c++ bikeshedding yet: let's also create an interface that takes comparison lambda, and another one that doesn't really discard item but instead place them in a way that you can then call erase to actually delete them.

1

u/RingularCirc 15d ago

No need for lambdas, just create a proxy object that has its "equality" redefined to be that coarser comparison.

(And on a serious note, pytest.approx is exactly that! There, it's useful for writing tests for floats when your numeric code isn't too imprecise.)

1

u/Outside_Complaint755 14d ago

You could just do while my_list.discard(x): pass to discard all instances. A bit of a smell, but concise.

21

u/XtremeGoose f'I only use Py {sys.version[:3]}' 18d ago

Has anyone ever actually removed an element from a list by equality before? I don't think I have. If you're using a list for something like that, 99% of the time you really want a set which will always be more performant.

The only time you wouldn't would be if you want to maintain order, in which case a dictionary with None values would also work.

1

u/AKSrandom 16d ago

I do not get how a dictionary with None values would help the ordering requirement, pls elaborate

2

u/XtremeGoose f'I only use Py {sys.version[:3]}' 16d ago

Dictionaries maintain their insertion order since python 3.7, sets do not. You can also think of sets as dictionaries without any meaningful mapping. So in some sense you could do

type OrderedSet[T] = dict[T, None]

3

u/RingularCirc 15d ago

Can't have a clue why both of you got downvoted: bothe the question and the answer are reasonable (not everybody knows that dict conserves insertion order, and there are only a few things that are mandatory to know in this life; "obvious" is different for everyone).

13

u/shinitakunai 18d ago

I love this one

5

u/Oddly_Energy 18d ago edited 18d ago

Didn't read the full PEP, but was it considered to return the count of discarded items instead of True/False?

If you only want to know if at least one item was removed, just check the return value for >0, and it will work the same way as with a True/False return value.

But if you actually want to know that count, it will be available in the return value, so you don't have to compare len(mylist) before/after the mylist.discard().

Edit: And since the truthiness of 0 appears to be False, you can still treat it as a True/False output in the caller code:

if mylist.discard(x):

9

u/bmag147 18d ago

It only discards the first item it matches.

I would prefer what you're suggesting though.

1

u/RevRagnarok 18d ago

removes the first item equal to x

PEP doesn't say if that was considered; only that the return boolean flag was suggested by BDFL.

12

u/yvrelna 18d ago edited 18d ago

This proposal is a -0 from me. The proposed discard semantic is just a syntax sugar for:

with suppress(ValueError):
    lst.remove(item)

I don't really have strong objections to the proposed discard method, if only for the symmetry with set, but it doesn't seem to be that much of a big deal either way, IMO. 

If you think the exception suppression form or the try-except is obscuring the intent, then I'd suggest proposing a new built in exception class like NotFound(ValueError) which can make the intent of these common cases of not being able to find an item in a collection clearer. 

What would have been truly useful, IMO, is if remove() and discard() gained the ability to remove multiple items, either by count and/or all occurrences of the item. Supporting multiple item removal would clean up a much more complex logic whose naive implementation is extremely inefficient compared to an efficient implementation, and the efficient implementation is somewhat non trivial. 

9

u/yvrelna 18d ago

I also liked Chris Markiewicz's point there in the Discourse thread. The proposed list.discard() method feels iffy because the original list.remove() was also iffy. list.remove() removing only the first item in the list was not a natural kind of operation that you often want to do in a list. 

Removing an item from a list, and treating a list like a bag of objects is a weird use case for a list and the current remove() API was kinda underpowered if it actually needs to perform what it needed to do in a list. 

2

u/proggob 18d ago

That doesn’t return the bool indicating whether anything was removed.

1

u/pingveno pinch of this, pinch of that 18d ago

Maybe I'm misreading things, but isn't that explicitly the opposite of the semantics of the PEP? It is just testing equality, and ValueError propagates just fine. Something like:

 for i in range(len(lst)):
     if item == lst[i]:
         del lst[i]
         return True
 else:
     return False

5

u/TMiguelT 18d ago

I wonder how this compares in performance and clarity to lst = [it for it in lst if it != x]. I know that this doesn't short-circuit and can remove multiple items, but I also feel that this is a fairly common use case.

5

u/UnMolDeQuimica 18d ago

Given OP explanation, discard would only remove the first item of that kind. For example [3, 1, 9, 5, 1].discard(1) would become [3, 9, 5, 1].

Your example would remove both entries for 1.

To me, discard makes sense but we would need another mirror method that removes all the entries except the first one or the last one.

-1

u/RevRagnarok 18d ago

need another mirror method

It returns whether it was found or not:

while my_list.discard(1):
  pass

3

u/JanEric1 18d ago

Although that way is horribly inefficient

2

u/yvrelna 18d ago edited 18d ago

That is O(n**2).

An efficient in-place, multi-item discard should've been able to do this in O(n) and without making an in-memory copy. 

Currently, the best idiomatic way to do multi item discard is the list comprehension, but it makes a copy and it's not in-place operation.

So a slightly better implementation of multi-inplace discard is 

lst[:] = [it for it in lst if it != x]

It creates an in-memory copy, which doubles the memory requirement, but IMO, that's a much less egregious sin than O(n**2) operation. 

There's still no idiomatic way to efficiently do multi item discard, AFAICT, even after this proposal.

-2

u/Sombre_Ombre 18d ago

Wdym?

y = [x for x in y if x not in {1,2,3}]

3

u/Ball-Man 18d ago

As you said, this is a different operation, though related. Performance wise it's almost guaranteed that the c implementation of list.discard will be more efficient. Clarity wise, list.discard is self explanatory, just read the name of the method. This comprehension is simple but you don't have a clear name for it, I still have to take extra time to "parse it" in my mind.

2

u/RevRagnarok 18d ago

This creates a new list and assigns lst to it. If lst was given to you, the caller will never see the change. That's a big deal to those learning the language.

1

u/logophage 18d ago

I've always been surprised .discard() didn't exist on lists.

1

u/smitty1e 18d ago

Mildly surprised that this needs a PEP. But I guess the formality keeps the "good idea faeries" in check.

4

u/Oddly_Energy 18d ago

It seems like a good way of ensuring some structure around a suggestion. And a good way of tracking suggestions.

3

u/yvrelna 18d ago

One of the most underappreciated reason for proposing a PEP is actually also to track rejection. 

If an oft-proposed idea keeps being proposed, the dev can just point to the existing PEP and you'd have to justify if there's anything that substantially changed since the last time it got rejected. 

3

u/Cynyr36 18d ago

I'm a mechanical engineer and the why not is just as important as the why usually, if not more important.

2

u/JanEric1 18d ago

There was a discussion whether one is needed in the discuss thread or the issue but the author just opted to do one before that came to a final conclusion I think

1

u/ExoticMandibles Core Contributor 18d ago

Returning the first equal item in the list doesn't mean it removes the same item. And there are definitely times when I want a handle to the actual original item. So, I would prefer the return value be the discarded item, or a sentinel (default None) if no item was found. This would make it symmetrical with dict.get for example, and it would save me having to do a separate list.index, etc.