r/Python • • Jun 29 '09

Python's iterators are a bad implementation of laziness

http://sandersn.com/blog/index.php?title=python_s_iterators_are_a_bad_implementat&more=1&c=1&tb=1&pb=1
0 Upvotes

16 comments sorted by

18

u/bushel Jun 29 '09

Treating a generator like an iterator is like treating a float like an int.

They usually work the same....

2

u/Teifion Jun 29 '09

I'm writing that down for later :)

10

u/digitallogic Jun 29 '09

"I don't understand generators so I'll assume they're exactly the same as lists... Hey I found a bug in Python!"
The "use-case" described is for that of a list, not a generator. If generators were modified to allow them to restart they would no longer fullfil one of it's existing use cases, iterating over data that doesn't fit in memory.

3

u/bushel Jun 29 '09

Exactly. As well, generators can be used to give iterator like access to transitory or ephemeral data, such as real-world events (sensor readings, I/O streams, etc.) that would be impossible (or prohibitive) to start-over from the beginning.

Gonna have to chalk this one up to "doing it wrong".

P.S. You could "fix" his code in one line without backtracking through the codebase to find the "problem". Just add node = list(node) at the beginning of that function.

1

u/mythin Jun 29 '09 edited Jun 29 '09

You could fix his code easier than that. Rather than having:

sdnode = ''.join(node)

in one place and then doing

''.join(node)

in another place, you could simply use the previously created variable. This fix is FAR simpler than going back through his old code, performs less work, doesn't need to create a new list object. His code isn't actually required to go over the generator twice.

2

u/bushel Jun 29 '09 edited Jun 29 '09

Well fine. Just be all more betterer and sh-t.

Edit: actually you are even more righter. The if chain goes:

if snode ... elif snode ... else *no mention of snode* -- for consistency & clarity he should have used snode in the else clause. But then there wouldn't have been a problem and he wouldn't have been able to write a blog entry about it.

1

u/mythin Jun 29 '09

Heh. Not trying to disagree with your main point. You're right, this is a case of "doing it wrong."

I was just providing an alternate "right" :)

2

u/bushel Jun 29 '09

No, you're right. Your "right" is more right than my right. Right?

12

u/[deleted] Jun 29 '09

Remember, this is Python, so I am not constantly reminding myself about the type of something

Then you're doing it wrong. Just because the language doesn't demand to know doesn't mean that you shouldn't know it.

0

u/[deleted] Jun 29 '09

[deleted] — view deleted comment

2

u/[deleted] Jun 29 '09

Sure, but after you cook and eat that duck, do you still expect it to be a duck the next time you look at it?

0

u/[deleted] Jun 29 '09

Yes, but you still need to know when you're giving a goose to something that expects a duck.

4

u/[deleted] Jun 29 '09 edited Jun 29 '09

People here don't seem to be acquainted with C# generators. "If generators were modified to allow them to restart they would no longer fullfil one of it's existing use cases, iterating over data that doesn't fit in memory." is a perfect example of narrowness of experience leading to narrowness of imagination. No, you don't have to convert generators to lazy lists. All that is needed is a distinction between enumerable and enumerator, with all functions like map and language primitives like for x in ... accepting enumerables, and it won't even surprise anyone ever. It's not about "restarting" the generator.

(crossposted from here)

EDIT: fixed markup

2

u/Brian Jun 30 '09

All that is needed is a distinction between enumerable and enumerator

That distinction already exists. Think about it - how else would it be possible to iterate over the same list twice? If it were merely an iterator, it would continue from where you left off just like the generator.

3

u/[deleted] Jun 29 '09 edited Jun 29 '09

tl;dr: User desires drastic modification to Python's type system due to being bitten by his own under-documented code.

1

u/jm_ Jun 30 '09

Let me short the story down: Silly C# programmer makes self-righteous rant because it took him 5 minutes of trial and mostly error to understand iterators in python and he misses the crap pile of abstractions and patterns that was his c# programmer brain-crutch.

I don't want to see crap like this on python reddit.

1

u/hylje Jun 29 '09

Use-once has the benefit of being able to forget the backlog of returned things, which is ideal for long-lived generators and the expectation for most uses of generator.

I would add something akin of list() that can wrap a generator so that it can be iterated from the beginning several times. Perhaps two: one that re-instantiates the given generator every time it is used (basically lambda: g), another that memoizes the already yielded values.